P.-A Versini

M Al Sayah

F Bordignon

D Schertzer

How the concept of nature-based solutions for climate adaptation could be introduced in Master's curricula. Insights from France

Keywords: Nature-based solutions, Education, Master program, Climate change, Adaptation

While the importance of Nature-Based Solutions for climate change adaptation (NBaS) is being increasingly acknowledged, very few studies investigate the status of these solutions in higher education syllabi. In an effort to bridge this gap, this study assessed the position of the NBaS notion in French Master's programs.

Based on a survey questionnaire, several interviews and a text mining analysis of the Master's contents, this paper identified, localised and analysed the French Masters that presented links with NBaS related concepts.

This analysis revealed that the notion of NBaS is far from being mastered. This might be caused by a conceptual misunderstanding of definitions. While text mining revealed that the building blocks of the NBaS framework are present, much still needs to be done for connecting them together. A combined analysis of the findings derived from this study led to recommendations to better introduce NBaS in Masters programs in the future.

Introduction

From nature-based solution (NBS) to nature-based adaptation solution (NBaS)

"Nature-Based Solution" (NBS) is a recent expression characterizing a worldwide practice that uses natural features and processes for increasing resilience to climatological and environmental challenges, while providing environmental, social and economic benefits [START_REF] Nelson | Challenges to realizing the potential of nature-based solutions[END_REF]. Still emerging, the NBS term regroups diverse existing terminologies that promote the conservation and rehabilitation of natural ecosystems, namely: Ecosystem-Based Adaptation (EbA), Sustainable Urban Drainage Systems (SuDS), Urban Green Infrastructure (UGI), and others (e.g., [START_REF] Castellanos | A text-mining approach to compare impacts and benefits of nature-based solutions in Europe[END_REF]. Recently, the International Union for Conservation of Nature (IUCN) proposed a global standard to qualify NBS. According to the IUCN, NBS are "actions to address societal challenges through the protection, sustainable management and restoration of ecosystems, benefiting both biodiversity and human well-being" (IUCN, 2020). Previously, and with a focus on urban environment, the European Commission (EC) introduced the NBS concept in its Horizon 2020 Program as "living solutions inspired and continuously supported by nature, which are designed to address various societal challenges in a resource-efficient and adaptable manner and to provide simultaneously economic, social, and environmental benefits" (European Commission, 2016). These initiatives have clearly promoted the expression in development and rehabilitation projects.

The implementation of NBS in diverse geographical and sociocultural environments has shown their potential for catalysing extensive reframing of classical approaches, and for promoting the integration of eco-system services and infrastructure resilience concepts (UN Water, 2018;[START_REF] Browder | Integrating Green and Gray: Creating Next Generation Infrastructure[END_REF]. Several works have already illustrated this purpose in agriculture [START_REF] Peter | Nature-based agricultural solutions: scaling perennial grains across Africa[END_REF], urban areas [START_REF] Babí Almenar | Nexus between nature-based solutions, ecosystem services and urban challenges[END_REF][START_REF] Bush | Building urban resilience with nature-based solutions: how can urban planning contribute?[END_REF][START_REF] Raymond | A framework for assessing and implementing the co-benefits of nature-based solutions in urban areas[END_REF], or maritime environments [START_REF] Arkema | Linking social, ecological, and physical science to advance natural and naturebased protection for coastal communities: advancing protection for coastal communities[END_REF][START_REF] Ruckelshaus | Harnessing new data technologies for naturebased solutions in assessing and managing risk in coastal zones[END_REF]. Among the socio-environmental challenges targeted by NBS, climate change is gaining increasing importance. Indeed, for several years, NBS have been promoted as climate change adaptation solutions as they make use of living organisms, soils or landscapes to reduce or mitigate climate change hazards [START_REF] Kabisch | Nature-Based Solutions to Climate Change Adaptation in Urban Areas: Linkages between Science[END_REF]. They have gained prominence both in scientific literature (IUCN French Committee, 2019) and in practical applications, particularly in Europe [START_REF] Seddon | Nature-based Solutions in Nationally Determined Contributions: Synthesis and Recommendations for Enhancing Climate Ambition and Action by 2020[END_REF]. Several works have already illustrated the importance of developing scientifically-based knowledge on climate change impacts as a precondition for designing and implementing NBS (see [START_REF] Hobbie | Nature-based approaches to managing climate change impacts in cities[END_REF] for the urban environment, or Gomez [START_REF] Go´mez Martín | Assessing the longterm effectiveness of Nature-Based Solutions under different climate change scenarios[END_REF] for the rural one). Accordingly, the term "Nature-Based adaptation Solution" (NBaS) is currently emerging [START_REF] Sayah | H2020 projects and EU research needs for nature-based adaptation solutions[END_REF], and will be used hereafter in this paper.

It regroups NBS targeting adaptation measures to sea level rise, extreme weather conditions and changes in precipitations, rise in temperatures, food production, health and economic sectors … More recently, NBaS were defined as one of the three cross-cutting priorities in the EU's 2021 strategy on climate change adaptation (European Commission, 2021).

Sustainable development and education

In parallel to the emergence of the NBaS term, academic teachings and programs devoted to sustainable development have increased during the last decade [START_REF] O'byrne | Teaching and learning sustainability: an assessment of the curriculum content and structure of sustainability degree programs in higher education[END_REF][START_REF] Vermeulen | Higher education level teaching of (master's) programmes in sustainable development: analysis of views on prerequisites and practices based on a worldwide survey[END_REF][START_REF] Keeler | Utilizing international networks for accelerating research and learning in transformational sustainability science[END_REF]. This increase can be attributed to the United Nations Decade of Education for Sustainable Development (2004Development ( -2014) ) [START_REF] Beynaghi | Future sustainability scenarios for universities: moving beyond the United Nations decade of education for sustainable development[END_REF]. Additional International initiatives have also encouraged the dissemination of this topic in the academic sphere. It is particularly the case of the United Nations Sustainable Development Goals 1 which are a set of 17 interlinked global goals designed to be a "blueprint to achieve a better and more sustainable future for all". SDG 4, "Quality of Education", directly relates to this purpose, specifically under target 4.7 that seeks to ensure knowledge and skills required for the promotion of sustainable development by the year 2030. The large media coverage of the Conference Of the Parties (COP) 21 and the Paris agreement on Climate Change have also contributed to the widespread inclusion of sustainable development into education programs. The global development of "green job" opportunities can also be considered as a contributing factor (OECD, 2014). In the same manner, more traditional sectors have shown growing interests in sustainable development, as employers are increasingly searching for sustainability expertise within the skill sets of their candidates [START_REF] Vincent | Environmental reviews and case studies: in search of common ground: exploring identity and core competencies for interdisciplinary environmental programs[END_REF].

Therefore, questions about how environment-oriented courses are given appear relevant. This necessity has been revealed and illustrated in several papers [START_REF] Clark | Education for sustainable development, curriculum reform and implications for teacher education in a small island developing state[END_REF][START_REF] Le | Education for sustainable development through curricular themes of environmental knowledge: an analysis on vietnam's biology curriculum[END_REF][START_REF] Lozano | Analysing the factors affecting the incorporation of sustainable development into European Higher Education Institutions' curricula[END_REF][START_REF] Farinha | Universities speak up regarding the implementation of sustainable development challenges: the case of Portugal[END_REF], where this aspect was usually introduced through the notion of "Education for Sustainable Development". However, very few studies were dedicated specifically on Masters programs [START_REF] Trencher | Evaluating core competencies development in sustainability and environmental master's programs: an empirical analysis[END_REF] and NB(a)S. Indeed, only one paper [START_REF] Oral | Recommendations for promoting environmental education through nature-based solutions (NBS) perspective at Turkish higher education institutes[END_REF] was focussed on NBS to provide recommendations for increasing ecological education in Turkish higher education institutions. Climate change is introduced more and several papers have focussed on this topic [START_REF] Hess | Climate change belief, sustainability education, and political values: assessing the need for higher-education curriculum reform[END_REF][START_REF] Milovanovic | Senior engineering students in the USA carry misconceptions about climate change: implications for engineering education[END_REF][START_REF] Tolppanen | Changes in students' knowledge, values, worldview, and willingness to take mitigative climate action after attending a course on holistic climate change education[END_REF]. They illustrated that knowledges and practical aspects must be taught together and included within general education curriculum to correct misconceptions and biases related to climate change [START_REF] Hess | Climate change and higher education: assessing factors that affect curriculum requirements[END_REF][START_REF] Tang | A model of behavioral climate change education for higher educational institutions[END_REF]. Understanding the science of climate change appears as a first step towards taking action to address it.

Although the EU and IUCN are pushing NBaS to become a mainstream term, there are no specific studies regarding NBaS and its related concepts are introduced -if introduced -into education programs. It looks particularly important for master's programs, because they usually lead to management jobs. Future graduates will therefore be required to make decisions in land use planning, urban planning, risk and resource management for example. It therefore seems very important that they be made aware of NBS during their studies, as an alternative to traditional "grey" solutions. Moreover, as underlined by recent IPCC (IPCC, 2021) and IPBES (Portner et al., 2021) reports, climate change adaptation and biodiversity conservation/protection must be considered together. Making the bridge between both objectives, NBaS appear to be particularly relevant for this purpose.

Consequently, a comprehensive overview on how NBaS (term and concepts) are being disseminated to students is still lacking. Accordingly, this paper reports insights on the way NBaS are addressed in Master's degrees training programs. It was done in the framework of the LIFE ARTISAN project (Achieving Resiliency by Triggering Implementation of nature-based Solutions for climate Adaptation at a National scale 2 ), one of whose objectives is to identify the obstacles and levers related to the implementation of NBaS.

To fulfil its purpose, this research work aimed to identify, localize, and analyse the current Master's level offer in relation with NBaS. This study was conducted in France, as NBaS are trusted to be the way forward for leveraging conjointly the second national climate change adaptation plan (PNACC-2 3 ) and the Biodiversity plan 4 .It is under this context that the LIFE ARTISAN project aims to promote the implementation of NBaS throughout the French (metropolitan and overseas) territory. Based on the inventoried Master programs, some recommendations were proposed to improve the dissemination of NBaS in higher education curricula, and to promote the uptake of these modules among future graduates.

Methodology

The analysis conducted in this study was built using a sequential five stepped approach, mixing open information collection, text-mining analysis and interviews: (i) to inventory all the environmental Masters with a possible link with the NBS, (ii) to design a unified questionnaire and send it to Master supervisors to select those really related to NBS, (iii) to conduct individual interviews with some of the supervisors to obtain qualitative information, (iv) to analyse Master's contents and descriptions via text mining to highlight similarities and differences, and (v) to collect and synthetize results to make recommendations.

Inventory and survey questionnaire

In France, since 2014, Masters are classified by fields and mentions (A mention defines the sectorial, disciplinary and professional approaches of the master program). To select the more appropriate mentions, two databases were used: one developed by the France government,5 and one created by a media group specializing in information on training, studies and professions for young people. 6Based on the titles and theoretical contents of the 253 existing mentions, those related to environmental disciplines were selected. It was assumed that they possess possible links with NBaS related concepts. To this end, 13 mentions were selected7 :

• Biodiversity, Ecology and Evolution (BEE) A first inventory containing 210 "Environmental" Master was created. For each Master, basic information (location, mention, competences, and courses fields) was compiled from the corresponding website. Then, a survey questionnaire was built in Excel form (Fig. 1) to obtain additional information: objectives, target audience, skills developed, career opportunities, detailed courses, teaching methods, etc. Specific fields were dedicated to NBaS to determine the importance that this notion occupies in the considered course, by whom are they introduced (discipline, specialization, background) and how (in which medium/environment, type, and concepts).

The Master supervisors of these 210 Masters were contacted by email by the authors to complete the questionnaire. In addition to the initial email, two to three reminders were sent a few weeks apart to ensure the highest possible response rate. Additional contacts were created based on the mail feedbacks and the conducted interviews.

Interviews

35 phone interviews were then conducted by the authors with some of the Master supervisors. They aim to obtain additional information about Master's program closely related to NBaS, or provide details to the contacted supervisor to explain the objective of this work and how they can participate. An interviews guide was specially developed for this purpose, and can be found in additional materials. It was organized in 5 groups of questions to: (i) identify of the interviewee and the master program, (ii) assess the interviewee knowledge and activities on NBS, (iii) understand how NBS is introduced and studied, (iv) have feedback from the direct and indirect beneficiaries of the course, (v) analyse the needs for future development and possible collaboration within the ARTISAN project.

Classification

Following the completion of the first two steps, a two-level classification was made to facilitate the analysis. The first level was related to the strength of the link between the Master's program and NBaS (called "relevancy"). Three categories were then defined according to the importance given to NBaS concepts in the programs' presentation and contents:

• Low: NBaS often named differently are approached in a very marginal way, and the notions of "adaptation" and "climate change" are absent or disconnected. Only one or two courses introduce NBaS, and hence these solutions do not appear as a focal point of the program. A second level of classification was proposed based on the environment that the Master's program focused on. This classification was initially proposed by the consortium of the ARTISAN project to be used as a common reference for every action. As masters' programs are usually focused on a particular environment (urban, agricultural, coastal…), the objective was to see if an environment was more familiar to the notion of NBS than another. The following classification was used:

• Natural environment (reserves, park, etc.)

• Agricultural environment • Forest environment • Water environments (wetlands, rivers, water bodies, streams, etc.)

• Maritime and coastal environment

• Urban environment • Mountainous environment • Multi-environments
It is worthwhile mentioning that the "multi-environments" category was introduced for generalist training programs. The latter offer conceptual or theoretical knowledge rather than a given application in a specific area (e.g. economic training or ecological engineering). In this case, several environments are covered, hence the inclusion of the prefix multi.

Text mining: meta-analysis using CorTexT

A meta-analysis of the inventoried Masters contents was performed using the CorTexT text-mining tool.8 CorText is a platform for methodological development, software engineering and analytical support for textual corpora. It allows the processing, characterization, analysis and quantification of slightly structured or unstructured textual data. For this purpose, a co-word analysis was conducted on the explored corpus by capturing the frequency of word pairs. When two words are often associated within the same textual element (sentence, paragraph), it means that they are strongly related, hence, these co-occurrences are counted. CorTexT was recently applied for the analysis of scientific literature of different disciplines: permafrost research [START_REF] Bordignon | A scientometric review of permafrost research based on textual analysis (1948-2020)[END_REF], food security [START_REF] Cardon | Bounded futures: growing a boundary foreknowledge infrastructure in food security research[END_REF], research and education in agroecology [START_REF] Nicot | The European networks of research, education and training stakeholders in agroecology[END_REF], and ecoservices in agriculture [START_REF] Tancoigne | The place of agricultural sciences in the literature on ecosystem services[END_REF].

In this study, the meta-analysis was carried out on the information compiled from the Master inventory, especially for the objectives, skills and courses fields, to understand and characterize the current master offer. This analysis was carried out on the whole sample of environmental Master's degrees (n = 210). Two sub-groups (corpora) were then identified: the first contains all of the inventoried masters that have a more or less strong link with NBaS, identified by the questionnaire returns ("NBaS" group), while the second groups all of the remaining Masters ("Non-NBaS" group).

Once uploaded, CorTexT performs an extraction of terms using the dedicated script available on the tool's platform. Using a distributional measure, CorTexT then automatically groups close terms (e.g., "flood risks" and "risk of flood"). However, the list of terms can also be manually fine-tuned so that synonyms can be grouped together, (e.g., "GIS" and "geographic information system") or to group any terms that the user considers to be close or associated (e.g., "professional environment" and "professional world"). Irrelevant terms can be then removed (e.g., "training offer", "various fields"), and others can be added despite their low frequency of occurrence in the corpus (e.g., "nature-based solution"). On this basis, several steps/analysis can follow.

• Similarity and specificity: by determining the number of occurrences for each term, and listing words that are found in the presentation of all Master's mentions or conversely those that are only found uniquely in one. For that purpose, the skills, objectives or courses sections were investigated. • Definition of clusters: by representing the network of term co-occurrences (whether they appear in the skills, objectives or courses fields of the survey).

CorTexT uses a distributional measure to quantify term co-occurrence [START_REF] Weeds | Co-Occurrence retrieval: a flexible framework for lexical distributional similarity[END_REF]. Within the plotted network, the nodes' size is proportional to the number of co-occurrences they have with other nodes and is thus highly correlated with their global frequency. Edges between the nodes represent the proximity between terms. The program identifies clusters, which are sets of strongly related terms and represents the topics contained in the corpus (using the same colour in the graph). For each cluster, CorTexT automatically suggests a name that can be changed.

Results of the surveys

Returns and encountered difficulties

Among the 210 solicited Master's supervisors, 118 responded to the survey with answers regarding the link between their training programs and NBaS (either positive or negative). Among the 118 received answers, 68 Masters were finally retained given their different degrees of relationship with NBaS.

It is important to mention that this inventory was built in 2020, at the same time when the COVID-19 pandemic was casting its weight on the world. Under this context, all educational institutions were forced to ensure a pedagogical continuity, by adopting new methods such as remote or distant learning. Naturally, this reduced the availability/ response of Masters' supervisors given their occupation with saving the academic year. Nevertheless, this response rate is clearly higher than that obtained (~10%) for similar approach at the European scale [START_REF] Lozano | Analysing the factors affecting the incorporation of sustainable development into European Higher Education Institutions' curricula[END_REF]. To ensure the integrity of this study's inventory, Master's programs whose supervisors did not respond were investigated separately. This was done by analyzing the relationship of their contents with NBaS using their respective websites, description and courses presentation. If these programs were considered unrelated to NBaS, they were excluded from the selection.

Classification by relevance to NBaS and by environment (or medium)

The classification of Masters by environments is presented in Fig. 2. These findings highlight the predominance of generalist training courses that focus on several environments (i.e. the "multi-environment" category). These Masters are mainly related to ecological engineering and its application in various contexts, whether natural (continental or maritime) or anthropogenic. The urban environment is also the subject of several Masters in connection with NBaS. In most of them, the renaturation of cities and green and blue corridors are particularly promoted. It is worthwhile mentioning that the urban environment is often suitable to multidisciplinary work bringing together hard sciences (hydrology, climatology, mechanics, etc.) and economic and social sciences (urban planning, sociology, law, etc.), hence its widespread presence in syllabi. Conversely, some environments have very few Master programs that are specifically dedicated to them. This is the case of the maritime, mountainous, natural or forest environments. However, despite this small sample, the NBaS concept seems to occupy an important position within these Masters when their programs include adaptation to global changes.

Geographical distribution

In order to geolocalise the selected Master programs on the French territory, an interactive Google Map (https://www.google.com/ maps/d/edit?mid=1EyuaacWDe5wY_4mSDfMORXsu0-8DPD1 Q&ll=47.235732892962446%2C-0.44982916441731025&z=6) was built. To focus on the most NBaS related Masters, only those belonging to the middle and high relevance categories are shown on this map. For display purposes, the mapped Masters were classified according to their medium of interest (environment), using a specific symbology (Fig. 3). This type of maps allows an evaluation of the program's adequacy with respect to the local geographical context (e.g. a mountain related program in a mountainous area); and can also serve as a decision support tool by helping local actors to identify the Master programs from which they can recruit employees according to their needs.

General Masters targeting multi environments cover the territory in a heterogeneous manner. As can be seen, the North, the Central Region and the Mediterranean coastal areas lack any real trainings in connection with NBaS. On the other hand, one third of the Masters offer is located in the Paris region. Concerning Master programs dedicated to very specific environments, these seem to be derived from their geographical contexts, for instance: courses dedicated to the maritime environment are concentrated near the coastal areas, courses related to the mountainous environment are located near the mountain belts, and those relevant to water environments are rather concentrated in the inland parts of the country. Urban related NBaS Master programs are particularly concentrated in the north-west of France. Surprisingly, no urban related Masters exist along the Brest-Grenoble diagonal, despite the presence of large metropolises such as Nantes, Bordeaux, Toulouse, and Marseille. Regarding Master programs dedicated to the agricultural sector, these are all located west of the Lille-Marseille axis. This distribution seems logical, given the fact that this part of the territory includes the country's largest useful agricultural area.

Results of the meta-analysis

A list of 152 terms was selected for text-mining using CorTexT. Common terms between different Master's mentions (fields) were analysed. The same was done for unique terms that are specific to each field. Tables 1 and2 illustrate the objectives and courses that are common to the majority of Master mentions included in the NBaS and Non-NBaS corpuses, respectively. Green cells refer to NBaS Masters, red cells to Non-NBaS Masters, while striped cells refer to both. Based on findings from Table 1, the terms climate change, law, policies and regulations appear to be mentioned in many descriptions and objectives of NBaS Masters. However, these are often employed as attracting conceptual elements for positioning the Master mentions into their operational environment, without necessarily having specific teachings on these subjects. This is the case of climate change, where the terms appear to be frequently used but with very little related teaching material. These terms also refer to different scientific domains that seem complementary in an integrated management framework. For this purpose, a multidisciplinary approach is often required. Table 1 also highlights the particularity of certain mentions (E, STAAE) that do not share any objectives with the other mentions of the NBaS corpus (all Masters contained in the EEET and SOAC mentions belong to the Non-NBaS corpus). This is due to the fact that these courses are far from the listed topics, and are oriented more towards biology (STAAE mention) and the energy sectors (E mention). Many teachings, tools and methods (statistics, GIS, project management), as well as institutional disciplines (law, public policies and regulations) are selected as common courses of the inventoried Master's programs. They represent a framework to assess solutions and projects from an engineering and political point of view. Within the whole sample, "water" is the only "geophysical" or hard science element addressed in several mentions. This highlights the essential role of this resource regardless of the Master's background. It is also interesting to note that physics, mathematics or biology for instance do not represent common teachings among these environment-related Masters.

Table 3 presents teaching disciplines that are unique to a particular mention. It illustrates that some of the listed courses could directly be linked to NBaS, such as the case of Agroforestry for the AETPF Masters or Landscape management for the RE Masters. This reflects an adaptive orientation geared towards sustainable development both for agricultural and risk management disciplines. Here again, no mention highlights specific teachings in geosciences. They are all majorly characterized by some technical or socio-economical specialities.

Regarding the analysis of the NBaS corpus, 8 clusters were identified by CorTexT and then named based on user interpretation (Fig. 4). Once again, there is no cluster directly related to geoscience. Some fields (hydrology, environmental sciences) are scattered among the different clusters, which are more oriented toward stakeholders and operational management. Adaptation is mentioned, but connected to few items (conservation biology, environmental changes). The most important ones along with their most frequent terms are listed below:

• Tools and systemic approach (law, GIS, statistical tools and analysis, public policies, climate change): this cluster regroups courses devoted to tools, methods and disciplines mentioned above and is not specific to NBaS. It should rather be linked to the "Public policies" and "Tools and analysis" clusters found in the Non-NBaS corpus. As already noted, this cluster also refers to the toolbox required to assess solutions and projects from an engineering/political point of view. • Water management (Aquatic system, hydrology, resource management, watershed): This cluster, also present in the Non-NBaS corpus, refers to courses related to water. The latter forms a common theme for many Masters, hence demonstrating the importance of this topic regardless of its field of application (agriculture, risk management, urban environment, etc.). The scarcity of this essential resource, particularly under the intensification of extreme events, may be a reason behind the predominance of the "water" theme. • Environmental management and impacts (environmental impacts, environmental management, water, pollution): this cluster, endowed with numerous connections to the previous one, deals with the threats that ecosystems both natural and anthropogenic have to face. While this cluster is more engineering-oriented, it highlights the importance of monitoring and analysing environmental impacts, for building proper management plans.

Table 1

Common objectives between the different Masters' mentions: Green cells refer to NBaS Masters, red cells to Non-NBaS Masters, striped cells refer to both categories.

Table 2

Common teachings of the Masters' mentions: Green cells refer to NBaS Masters, red cells to Non-NBaS Masters, striped cells refer to both categories.

• Natural resources and environments (management of natural environments, global changes, environmental sciences, conservation biology): this cluster, also connects with the previous one, and is more focused on natural environments. It refers more to ecology than engineering. It is also more oriented towards the implementation of solutions. It is in this cluster that NBaS can be introduced. It is also important to mention that this cluster is connected to that of adaptation, hence its importance (for the "a" in NBaS). • Regional development and planning (project management, spatial analysis, territorial development project, urban spaces): this cluster relates to the territorial scale. It highlights the different spatial scales that must be taken into consideration for implementing a project (development, management, etc.), while accounting for the heterogeneity of ecosystems or environments (urban, natural, agricultural, etc.).

Regarding the analysis of the Non-NBaS corpus, 7 clusters were defined (Fig. 5). Some of them look similar to those defined for the NBaS corpus, however they are more generic. The most important clusters along with their frequent terms are listed below:

• Public policies (Policies and regulations, project management, public policies, urban projects) • Tools and analysis (Geographic information systems, statistical tools and analysis, environmental challenges and projects, analysis tools) • Water management (water, hydrology, water treatment, aquatic systems, wetlands) • Risk and environmental management (Law, risk management, environmental management, risks and environments, natural risks)

Table 3

Teachings unique to a particular mention: Green cells refer to NBaS Masters, red cells to Non-NBaS Masters, striped cells refer to both categories.

Discussions

On the NBaS terminology

First of all, it has to be underlined that from the 118 analysed Master's programs, the term "Nature-Based adaptation Solution" is only mentioned once in the courses, descriptions or contents. The analysis of the Masters presentations (via questionnaires and interviews) showed that the term and concepts of NBaS (or simply NBS) are far from being well known and mastered by everyone. Although the IUCN definition, as well as many examples, were recalled in the questionnaire, this terminology had to be redefined and explained to many people. Even when this term was known, its definition was not perfectly clear. For instance, such comments were made during an interview: "the term NBS so fashionable that it must be employed in all our European proposals is not very explicit in French and is subject to various interpretations". For others, the NBaS concepts are not "standalone" features, but are rather related to other emerging terminologies (resilience, climate change, etc), which in turn are not well mastered.

In other instances, NBaS were associated with biomimicry or renewable energies concepts. More surprisingly, the concept was sometimes belittled with statements like: "I don't think we integrate this idea of blah blah solutions". For others, this concept is embedded in all nature-related activities such as agriculture, regardless of the type of production (intensive or organic).

Indeed, this confirmed that the emerging concept of NBaS encompassed old notions, that are more commonly used based on the field of application or the studied environment. Thus, blue green solutions (or blue green infrastructures) are used in urban hydrology as a tool for storm water management (alternative techniques), "soft solutions" (sand recharging, revegetation) are used in coastal environments, and "agroforestry" practices are used in agricultural media. The concept of "ecological engineering" also often appeared as a synonym for NBaS as both aim to increase the resilience of ecosystems by: promoting the reconstitution of natural environments, restoring degraded environments and optimizing functions provided by ecosystems. Finally, it seems that every university has their own strategies to introduce these notions. Note that similar results were obtained by Farinha et al. (2020) about and[START_REF] Wezel | Agroecology in Europe: research, education, collective action networks, and alternative food systems[END_REF] about agroecology. The Authors attributed this difference to the inner structures/frameworks of the concerned universities.

Position of the NBaS concept in the master programs

The in-depth analysis of the text-mining results provides further details. Concerning NBaS-related Masters (refer to Fig. 4), the occurrence of the "adaptation" cluster shows that the concept is present to a certain degree. Nonetheless, the size, position and connections of the cluster reveal a relatively shy presence. When considering the NBaS framework, a reference to one or more societal challenges should be made, and an active engagement of stakeholders with various backgrounds must be ensured (IUCN, 2020). As can be seen from Fig. 4, the adaptation cluster is almost entirely disconnected from these societal challenges, as it is only connected to the terms "global changes", "conservation biology" and "agrifood", and show no apparent direct links with stakeholders. The concept of adaptation is implicitly present in the Water and Environment clusters, but related to climate impacts related to hydrology (flood and drought). Adaptation to other types of risk (sea level rise, forest fires and dieback for instance) is not mentioned.

While global changes are a relatively vast field, its occurrence in the "natural resources and environment" cluster alone shows that it is being viewed from a narrow angle. Its disconnection from the term "climate change" supports the above-mentioned observation. As the NBaS framework clearly highlights the importance of adapting to global changes, the specificity of the latter should be better addressed. As conservation biology is related to both biodiversity and climate change, the connection of the term "adaptation" to conservation biology reveals a particular focus on biodiversity. While agri-food clearly relates to food security, the connection between adaptation and agri-food is not sufficient to determine if food security is being holistically studied.

This analysis shows that elements of the NBaS framework are clearly present but are very scattered, often without any multidisciplinary approach. This was confirmed by the questionnaires: NBaS are implicitly mentioned or diffused as marginal ideas in one or more courses. They are usually introduced as case studies where a solution considered as an adaptation measure is presented. Seemingly, workshops and projects related to such case studies are increasingly referring to NBaS.

Perceptions of NBaS by environment

Named differently throughout the different Master's programs, the concept of NBS is introduced in a diffuse manner, cited within more general modules or given as an example during lectures, projects, workshops or site visits. Clearly, this inclusion targets the transmission of theoretical knowledge with very few cases of concrete implementation for actual climate change adaptation efforts. Accordingly, the NBaS concept is introduced by environment as follows:

• Multi-environments: ecological engineering is taught for implementing environmental transition tools. The most concerned environments are natural spaces, terrestrial environments, aquatic environments and cities. Biodiversity and landscape integration are put forward for promoting sustainable development based on the "avoid, reduce, compensate" sequence, and thus for counterbalancing negative (degrading) driving forces. • Urban environment: NBaS concept is presented via the ecosystem services provided by nature in city. Blue green corridors appear as the most mentioned tool.

Challenges that are usually addressed are storm water management, urban heat island mitigation and biodiversity conservation. • Agricultural environment: NBaS concept is introduced through the • concepts of agroecology and phyto-valorization. These solutions are used for promoting the use of plants as solutions to environmental problems (phytopurification, restoration of degraded environments, etc.). • Mountainous environment: Ecological engineering is mostly introduced for the conservation of this medium's particular biodiversity and hydro systems. Ecoengineering is mainly used to help these environments adapt to anthropization (including agriculture and silviculture). • Water environment: Courses in these domains mainly focus on the interactions between aquatic environments and landscapes via blue green corridors and biodiversity. Wetland restoration is also often addressed in these courses. • Maritime environment: NBaS concept is mainly presented through eco-design methods for coastal management related structures. • Natural environment: the notions of ecology and biology are introduced for focusing on the management, conservation or restoration of natural spaces.

• Forest environment: only one training course related this very specific topic was found. It corresponds to the formation of the Guyana University and targets specifically tropical forests (Guyana). Within this course, conservation biology occupies a considerable volume for adapting forest management to global changes.

5.4. Is there a "public" demand for NBaS?

According to the interviewed Masters supervisors, students are asking for more courses that advocate more virtuous solutions for the environment. They seem to be receptive to Nature-Based Solutions when they are presented as examples in projects or workshops. The current emergence of youth political and social movements, such as Youth For Climate 9 favours this shift. However, Masters supervisors also notice that some students have controversial points of views, as they favour grey building solutions over the presence of nature.

According to several Master supervisors, territorial collectivities request that NBaS (appearing under other names) become more integrated into training programs. They consider that young graduates who are aware of these concepts will be in a better position to propose more biodiversity friendly/conservative solutions. This interest is translated by proposals for tutored projects subjects that integrate NBaS for different objectives (wetland restoration, floodplain management, etc.).

Nevertheless, there is no dedicated "environmental engineering" diploma and no specific jobs mentioning explicitly the term of NBaS for now. Therefore, it is important to pursue the creation of new "green" jobs for promoting a dynamic progression towards NBaS both in the education and market sectors. As suggested by several interviewees, the establishment of new regulations that companies can adopt could help. This is particularly important, since to date the absence of regulations related to NBaS severely hampers the development of the concept despite its importance. Once these regulations are created, the "Quality Safety Environment" manager can be in charge of ensuring their implementation. In order to go further on this demand point of view, an additional study could be carried out.

Another path often mentioned for leveraging the use of NBaS, and the emergence of dedicated jobs, is the creation/application of labels and certifications. In fact, according to the Master supervisors, many planners seek to enhance their ecological transition process by obtaining some kind of certification. Some common certifications such as: organic or biodynamic labels for agriculture, or eco-districts for urban neighbourhood, can be more readily obtained using NBaS. As a corollary for obtaining a label or a certification, and to overcome any suspicion of greenwashing, many interviewees highlighted the need for robust evidence on the efficiency of NBaS. Therefore, it seems that there is also a demand for tools and methods that quantify the effectiveness of the planned NBaS.

A consensus around the lack of knowledge

As mentioned above, many interviewees underlined the absence of evidence to 9 https://youthforclimate.fr justify the use and dissemination of NBaS. Indeed, from an operational point of view, a quantitative assessment of NBaS eco-system services is required. To implement such a solution, one must be sure to obtain satisfactory results for a particular problem. Behind this quantitative assessment, many disciplines are concerned: physics, biology, hydrology, mechanics, ecology, etc. In these domains, some significant research efforts seem to be required to better understand the NBaS performances over time, and especially under changing climate. These investigations seem necessary to better understand, reproduce and anticipate the behaviour of these NBaS, as well as the physical and biological related processes. Research results, and their applications in terms of decision support tools for instance, should represent relevant material to create new courses devoted to NBaS. These conclusions are corroborated by the absence of hard sciences in the teachings listed in the masters related to the NBS, that are currently dominated by technical or socioeconomical topics.

Recommendations to update masters programs

Based on the analysis of the results from the text-mining, questionnaires and interviews, some recommendations are proposed to update the Masters programs contents on NBaS issues.

NbaS: A concept and a term to promote

For unifying the contrasting terminologies that result from different backgrounds or media of interest, the term of Nature-Based (adaptation) Solution should be given further visibility. Little or not known, poorly mastered or outlined with uncertainties, significant pedagogical efforts must be made for familiarizing instructors, students and future employers with NBaS. To overcome this problem of terminology, the concepts behind NBaS should also be better explained. With reference to the IUCN's definition, these concepts include the direct objectives for their implementation: i.e. "to protect, sustainably manage and restore natural or modified ecosystems" and the indirect objectives "while ensuring human good-being and producing benefits for biodiversity". In order to properly illustrate these purposes, concrete examples of case studies are essential. These case studies must cover the different types of environments (urban, agricultural, forest, water, maritime, etc.) to include the largest audience possible.

Broaden the range of studied NBaS to underrepresented environments

The inventories of NBaS related Master's programs showed a predominance of general training material, often based on ecological engineering. At the same time, the urban and agricultural media appeared to be the most represented environments, leaving other areas with few or no specific courses. This is particularly the case of the mountainous, forested and maritime environments. Note that similar results on understudied environments were reported for ESD in small islands [START_REF] Clark | Education for sustainable development, curriculum reform and implications for teacher education in a small island developing state[END_REF]. Therefore, it is necessary to introduce further the concept of NBaS in these areas. This should be translated by the creation of teaching modules specially dedicated to these environments, based on past or ongoing activities.

Climate change: a need for deeper insights

Although the term appears in many Masters presentations, very few courses are actually dedicated to climate change, its causes, manifestations, consequences, and how to adapt to it. Climate change appears more as a currently hot topic that describes an ongoing situation and that justifies current legislations. Therefore, climate change should not be treated as a trending idea, but rather should be taught/studied as a complex process. Within its related courses, climate change should be addressed by explaining its genesis and the complexity of the global climatic system. The works of the IPCC should also be presented, namely those related to greenhouse gas emissions scenarios and subsequent consequences on climate and ecosystems. These courses could be locally adapted to the medium of interest (urban, agricultural, etc …), by illustrating the consequences of climate change induced events through concrete examples. Particular attention should also be paid to the uncertainties of climatic models' outputs, and to their difficult spatiotemporal representation of complex phenomena (such as precipitation), [START_REF] Shepherd | Storylines: an alternative approach to representing uncertainty in physical aspects of climate change[END_REF][START_REF] Versini | Climate change impact on water resources availability: case study of the Llobregat River basin (Spain)[END_REF].

Enlarging the role of geosciences

Throughout the different programs, NBaS are often approached from a biodiversity and landscape approach or agronomy perspective. Usually, given by ecologists, these courses confer an important place to ecology or conservation biology.

Although many of the ecosystem services provided by NBaS are linked to physical or chemical processes (eg. infiltration, retention, evapotranspiration), these are rarely detailed. Therefore, the introduction of geosciences into these courses is necessary for meeting the needs raised by the operational end-users, and reporting quantitative evidence on NBaS efficiency. As already conclude in [START_REF] Oral | Recommendations for promoting environmental education through nature-based solutions (NBS) perspective at Turkish higher education institutes[END_REF], the presentation of simulation models could be useful for this purpose. For enforcing the integration of quantitative assessments, the obtention of labels/certifications for Master's programs can also be conditioned by the obligation for robust evidence reporting. To this end, the ARTISAN project also dedicates an action for the formulation of new training modules that can serve the purpose.

Regarding this shift in interest towards geosciences, it is also important to better take into account the complexity of the studied environments across different scales. Spatial scales on one hand, to move from understanding the processes occurring at small scale (soil, vegetation …), to those of infrastructures, and its arrangement in a heterogeneous environment. Temporal scale on the other hand with the variability of the studied processes to consider, given the dynamic nature of climate change and the useful lifetime of the proposed solutions, and the persistence of the NBaS performance over time.

Towards a multidisciplinary and multi-environmental approach

The dynamic nature and complexity of the environmental, geophysical, biological and social challenges require a transition from silo approaches to multidisciplinary collaborative frameworks.

In fact, certain challenges are common to all kinds of environments despite their differences. This is particularly the case of climate change whose impacts are transversal across different media. The same can be said for urbanization, where urban sprawls expand into different media and create "mosaic" patterns. Given the interconnectivity of the different environments, a silo approach at a scale of a single environment does not fit the real-world conditions. Accordingly, approaches that take into consideration the existing interactions among the different entwined ecosystems are needed.

On this basis, a transformation towards more multidisciplinary training courses should be planned. Nonetheless, this task is not simple, but requires significant collaboration and pedagogical exchanges. However, ensuring multidisciplinary will ensure a transmission of knowledge, and know-hows far beyond the limit of monodisciplinary approaches.

Conclusions and perspectives

This study assessed the position of the NBaS notion in French Master's programs. It is particularly important as Climate change adaptation and biodiversity protection should be considered together. Students need to be educated and develop some skills in this area.

Based on an inventory, a survey questionnaire, several interviews and a text mining analysis of the Master's contents, this paper identified, located and analysed the French Masters that presented links with NBaS related concepts. The results presented above have revealed that the notion of NBaS along with climate change are far from being mastered. Many gaps have to be filled in to better introduce the concept of NBaS in higher education. Some recommendations have been made to fill these gaps: (i) better explain the concept behind the term of NBaS, (ii) broaden the range of studied NBaS to underrepresented environments (mountainous, maritime), (iii) improve knowledge on climate change consequences, impacts and mitigation measures, (iv) put more focus on geosciences, (v) promote a multi-disciplinary and multi-environmental approach.

This work could help to move forward to promote a better integration of NBaS in the Masters programs in the future. As the UNFCCC states, adaptation is an iterative, progressive and dynamic process. Therefore, progress can only happen by learning. Accordingly, stimulating formal and particularly higher education curricula to integrate more adaptation solutions, here NBaS, can contribute to progress towards adaptation.

•

  Middle: NBaS appear in several courses. The link with climate change can be mentioned, without being directly emphasized. • High: NBaS are mentioned in almost all courses and represent the bulk of the Master's program. Climate change (or global change) is put forward to justify the use of these solutions as adaptation tools.

Fig. 1 .

 1 Fig. 1. A sample of the survey sent to the Master supervisors.

Fig. 2 .

 2 Fig. 2. Inventoried Masters classified by environment and relevancy to NBaS.

Fig. 3 .

 3 Fig. 3. Geographical distribution of the NBaS related Masters in France.

Fig. 4 .

 4 Fig. 4. Clusters identified for the NBaS Masters from text mining analysis.

Fig. 5 .

 5 Fig. 5. Clusters identified for the Non-NBaS Masters from text mining analysis.

https://sdgs.un.org/goals.

www.life-artisan.fr https://www.ecologie.gouv.fr/adaptation-france-au-changement-climatique https://www.ecologie.gouv.fr/plan-biodiversite

https://www.trouvermonmaster.gouv.fr

https://www.letudiant.fr/etudes/annuaire-enseignement-superieur/formation/diplome-mastersprofessionnels.html

7 Note that Acronyms refer to the French terminologies

https://www.cortext.net/about-us

Acknowledgments

This research was developed in the framework of the LIFE ARTISAN project (Achieving Resiliency by Triggering Implementation of nature-based Solutions for climate Adaptation at a National scale, LIFE18 IPC/FR/000007) that aims to promote the implementation of NBaS in the French national plan for adaptation to climate change (PNACC).

Data availability

Data will be made available on request.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Appendix A. Supplementary data

Supplementary data to this article can be found online at https://doi. org/10.1016/j.jclepro.2023.136364.


J De

J Browell
email: jethro.browell@glasgow.ac.uk

Y Goude

Adaptive Probabilistic Forecasting of Electricity (Net-)Load Joseph de Vilmarest, Jethro Browell, Matteo Fasiolo, Yannig Goude and Olivier Wintenberger

Abstract-Electricity load forecasting is a necessary capability for power system operators and electricity market participants. The proliferation of local generation, demand response, and electrification of heat and transport are changing the fundamental drivers of electricity load and increasing the complexity of load modelling and forecasting. We address this challenge in two ways. First, our setting is adaptive; our models take into account the most recent observations available, yielding a forecasting strategy able to automatically respond to changes in the underlying process. Second, we consider probabilistic rather than point forecasting; indeed, uncertainty quantification is required to operate electricity systems efficiently and reliably. Our methodology relies on the Kalman filter, previously used successfully for adaptive point load forecasting. The probabilistic forecasts are obtained by quantile regressions on the residuals of the point forecasting model. We achieve adaptive quantile regressions using the online gradient descent; we avoid the choice of the gradient step size considering multiple learning rates and aggregation of experts. We apply the method to two data sets: the regional net-load in Great Britain and the demand of seven large cities in the United States. Adaptive procedures improve forecast performance substantially in both use cases for both point and probabilistic forecasting.

Index Terms-Adaptive forecasting, net-load, probabilistic, time series.

I. INTRODUCTION

F ORECASTING electricity demand is fundamental in the process of maintaining supply-demand balance. This permanent equilibrium is necessary to maintain a reliable supply of electricity and to avoid damaging infrastructure. As electricity cannot be stored on a large scale, forecasts are crucial to informing production planning. This necessity explains why energy forecasting has gathered so much attention from the time series and forecasting community [START_REF] Hong | Energy forecasting: A review and outlook[END_REF]. The recent increase in electricity prices in Europe further emphasizes the importance of demand forecast quality.

The literature historically focused on point forecasting. However, the expected value of the load is not sufficient for risk management. Forecasting models cannot be perfect and therefore the production plans cannot match exactly the demand; therefore, grid operators need to call reserves to maintain the equilibrium and traders need to hedge their market positions. It is thus essential to have some information on the distribution around the mean to schedule the reserve needed. In Great Britain, for instance, reserves are tailored to be sufficient in all but 4 hours per year; this corresponds to the estimation of 99.95% confidence intervals. The importance of probabilistic forecasts was highlighted in the last two Global Energy Forecasting Competitions [START_REF] Hong | Probabilistic energy forecasting: Global energy forecasting competition 2014 and beyond[END_REF], [START_REF] Hong | Global energy forecasting competition 2017: Hierarchical probabilistic load forecasting[END_REF]. Indeed, the objective in both competitions was to forecast certain quantiles, and the submissions were evaluated through the sum of quantile losses.

Demand and supply characteristics are both evolving with time. On the demand side, unexpected events such as the coronavirus crisis or the recent increase in electricity prices in Europe can affect it significantly and abruptly [START_REF] Iea | Covid-19 impact on electricity[END_REF], [START_REF]World energy outlook 2022[END_REF]. From a longer-term perspective, the evolution of consumption habits such as those induced by the expected growth in electric vehicle penetration will change the demand patterns. On the production side, the increasing penetration of intermittent electricity production strongly changes the forecasting task. Indeed, controllable production units must be employed to meet the difference between the demand and the intermittent electricity production (mainly wind and solar energy), denoted by net-load. These changes in the behavior of the net-load can not be captured by classical offline probabilistic methods and motivate the need for adaptive methods, taking into account observations in a streaming fashion to improve the forecasting model.

The recent point forecasting literature has highlighted the improvements yielded by adaptive methods able to learn regime changes. Adaptation is especially crucial to forecast wind and solar power, and it has been successfully applied to online variable selection [START_REF] Messner | Online adaptive lasso estimation in vector autoregressive models for high dimensional wind power forecasting[END_REF] and online forecast reconciliation [START_REF] Di Modica | Online forecast reconciliation in wind power prediction[END_REF]. State-space representations have well captured the recent changes of pattern due to the coronavirus crisis [START_REF] Obst | Adaptive methods for shortterm electricity load forecasting during covid-19 lockdown in france[END_REF], [START_REF] Vilmarest | State-space models for online post-covid electricity load forecasting competition[END_REF]. While the majority of recent works have focused on point forecasting, [START_REF] Álvarez | Probabilistic load forecasting based on adaptive online learning[END_REF] consider adaptive probabilistic load forecasting based on hidden Markov models and is found to perform well for a range of loads; however, this approach relies on Gaussian predictive distributions which leads to relatively poor calibration (quantile bias) on the data we consider here. In [START_REF] Gaillard | Additive models and robust aggregation for gefcom2014 probabilistic electric load and electricity price forecasting[END_REF], [START_REF] Fasiolo | Fast calibrated additive quantile regression[END_REF], quantile GAMs are proposed, as well as variants of GAMLSS (Location Scale and Shape) in [START_REF] Gilbert | Probabilistic load forecasting for the low voltage network: Forecast fusion and daily peaks[END_REF], for probabilistic demand forecasting. These methods relax the Gaussian assumption but are not adaptive. [START_REF] Wang | A novel hybrid method for short-term probabilistic load forecasting in distribution networks[END_REF] propose a mixture model for short term probabilistic forecasting of individual load demand at different aggregation level, but the distribution of the data is supposed to be invariant with time. [START_REF] Arora | Remodelling state-space prediction with deep neural networks for probabilistic load forecasting[END_REF] propose, similarly to [START_REF] Obst | Adaptive methods for shortterm electricity load forecasting during covid-19 lockdown in france[END_REF], a deep neural network probabilistic forecasting method where the features extracted with a multi-layer RNN-LSTM are mapped to a state-space model for online adaptation. Their work is limited to time-varying Gaussian distributions as they use a Gaussian state-space model to generate density forecasts.

We propose in this article to apply the adaptive framework to quantile regression for probabilistic forecasting. There has been scarce work in that direction. We mention the exciting use of aggregation of experts to obtain an online forecaster [START_REF] Gaillard | Additive models and robust aggregation for gefcom2014 probabilistic electric load and electricity price forecasting[END_REF], [START_REF] Berrisch | Crps learning[END_REF]. The principle is to combine various predictions (experts) with a weighted average where the weights evolve over time. We exploit the idea of aggregation, but we introduce a more structural adaptation in the sense the experts themselves are adaptive.

The methodology employed and our contributions are summarized in Figure 1. We adopt a two-step procedure. First, we employ a widely used model class, namely that of Generalized Additive Models (GAMs), to forecast the expected value of the variable of interest. This is a non-adaptive model fitted to several years of training data. We consider a state-space representation to obtain an adaptive variant, relying on the Kalman filter for the inference similarly to [START_REF] Obst | Adaptive methods for shortterm electricity load forecasting during covid-19 lockdown in france[END_REF], [START_REF] Vilmarest | State-space models for online post-covid electricity load forecasting competition[END_REF]. Second, we focus on forecasting the quantiles of the net-demand. While the Kalman filter already models it as a Gaussian, we observe that this forecast is not always well-calibrated. Therefore, we use a set of quantile regressions on the residuals of the mean estimates. We apply the online gradient descent (OGD) to obtain adaptive quantile regressions. We select the best gradient step by exploiting an expert aggregation algorithm, Bernstein Online Aggregation [START_REF] Wintenberger | Optimal learning with bernstein online aggregation[END_REF]; it means we compute the OGD with several step sizes (multiple arrows in Figure 1) and our final forecast is a combination of these forecasts; the quantile loss is optimized online as proposed by [START_REF] Zaffran | Adaptive conformal predictions for time series[END_REF].

In Section II we present adaptive methods for probabilistic forecasting, and we discuss their evaluation in Section III. Then we apply our approach on two data sets. In Section IV, we consider the regional net-load in Great Britain, and we extend the work of [START_REF] Browell | Probabilistic forecasting of regional netload with conditional extremes and gridded nwp[END_REF] to the adaptive setting, also extending the data set with more recent data (including the coronavirus crisis). Furthermore, we show in Section IV-D that adaptive models have fewer needs for good explanatory variables; indeed, removing two (difficult to obtain) variables from the model has a lower impact on the adaptive variant. In Section V, we apply the methods to the load of seven major cities in the United States [START_REF] Ruan | A cross-domain approach to analyzing the short-run impact of covid-19 on the us electricity sector[END_REF]. In the latter application, we predict daily, rather than half-hourly, data to determine whether our approach works well at this time granularity.

II. THEORETICAL FRAMEWORK

We aim to forecast a variable of interest y t ∈ R given some explanatory variables x t ∈ R d , d > 0. We formally introduce the impact of time dependence on our approach. Let F t = σ(x 1 , y 1 , . . . , x t , y t ) be the natural filtration; it models the information contained in the observations up to time t. We discriminate between two settings represented with two diagrams in Figure 1: • Offline or Batch (Section II-A). The model is learned on a training period, for instance on data up to time n train . We estimate L(y t | x t , F ntrain ), the conditional distribution of y t given x t , F ntrain . • Online or Adaptive (Sections II-B and II-C). The model is learned sequentially. We estimate L(y t | x t , F t-1 ) at each time t.

A. Offline Model

Motivated by [START_REF] Browell | Probabilistic forecasting of regional netload with conditional extremes and gridded nwp[END_REF], we decompose our model into two steps. We forecast the conditional mean with a generalized additive model (GAM), and then quantile regressions on the GAM residuals yield quantile forecasts. 1) We model y t as a Gaussian random variable whose mean is a sum of the effects of the covariates:

y t = d j=1 f j (x t,j ) + ε t , ε t ∼ N (0, σ 2 ) , (1)
where the effects f 1 , . . . f d are either linear or nonlinear.

In the latter case, the effects are built using linear combinations of spline bases [START_REF] Wood | Generalized additive models: an introduction with R[END_REF]. The optimization of these effects on the training set yields a mean forecast

ŷt = E[y t | x t , F ntrain ].
2) The Gaussian assumption with fixed variance is violated in practice. Therefore, we fit a set of quantile regressions [START_REF] Koenker | Regression quantiles[END_REF] on the residuals to predict the distribution around the mean. We use an intermediate vector of covariates z t ∈ R d0 , d 0 > 0 derived from the original vector x t ; depending on the application, z t may contain the GAM prediction, the GAM effects f j (x t,j ) ... For any quantile level q, we define a vector β q ∈ R d0 via the following minimization problem:

β q ∈ arg min β∈R d 0 ntrain t=1 ρ q (y t -ŷt , β z t ) , (2)
ρ q (y, ŷq ) = (1 y<ŷq -q) (ŷ q -y) .

Then, we predict the quantile for probability level q with ŷt + β q z t . This is justified by the following well-known lemma [START_REF] Koenker | Quantile regression[END_REF].

Lemma 1. Let Y be an integrable real-valued random variable. For any 0 < q < 1 the q-quantile of Y denoted by

Y q satisfies Y q ∈ arg min E[ρ q (Y, Y q )].
This two-step procedure is motivated by computational time and by the fact that the data is sparse in the tails (sparser than around the mean) as in [START_REF] Gaillard | Additive models and robust aggregation for gefcom2014 probabilistic electric load and electricity price forecasting[END_REF]. Indeed, we could also use quantile GAM [START_REF] Fasiolo | Fast calibrated additive quantile regression[END_REF], but it is more time-consuming than simple quantile regressions. Hence the need to use simple linear adjustments to correct the (more richly-parametrized) mean GAM. Also, we explain in the following paragraph that the mean GAM may be adapted using the Kalman filter, and that is not the case for quantile models, for which we adopt a different strategy, see II-C. The first natural adaptive forecaster consists in re-training the offline model at each time step. We call this procedure incremental offline.

B. Adaptation of Generalized Additive Model

We adapt the GAM as done by [START_REF] Obst | Adaptive methods for shortterm electricity load forecasting during covid-19 lockdown in france[END_REF]. More precisely, we freeze the nonlinear effects learned on the training set (Section II-A); with these effects, we define a new covariate vector f (x t ) = (f 1 (x t,1 ), . . . f d (x t,d ), 1) , where f j (x t,j ) is the standardized version of f j (x t,j ). Then we consider the linear Gaussian state-space model:

θ t -θ t-1 ∼ N (0, Q) , (4)
y t -θ t f (x t ) ∼ N (0, σ 2 ) , (5)
where Q is the state noise covariance matrix, and σ 2 the space noise variance. In the preceding set of equations, we assume implicitly that the noises are independent. Estimation in a linear Gaussian state-space model with known variances has been optimally solved by [START_REF] Kalman | New results in linear filtering and prediction theory[END_REF]:

Theorem 1 (Kalman Filter). Provided that the datagenerating process is the state-space model with variances Q and σ 2 , and if the prior distribution of θ 1 is N ( θ1 , P 1 ), then the posterior distribution of the state is a Gaussian whose mean and covariance matrix have analytical forms. Precisely, we have θ t | F t-1 ∼ N ( θt , P t ) with:

P t|t = P t - P t f (x t )f (x t ) P t f (x t ) P t f (x t ) + σ 2 , (6)
θt+1 = θt - P t|t σ 2 f (x t )( θ t f (x t ) -y t ) , (7)
P t+1 = P t|t + Q . ( 8
)
The unsolved issue of linear Gaussian state-space models concerns the estimation of its variances Q and σ 2 .

In our setting, as well as in many applications, the variances are unknown. The choice of the variances can be seen as a parametrization of a gradient algorithm. Indeed, f (x t )( θ t f (x t ) -y t ) is the gradient of the quadratic loss with respect to θ; therefore, the update ( 7) is a gradient step with a pre-conditioning matrix P t|t /σ 2 which depends crucially on the choice of σ 2 and Q. We propose different settings:

• Static. We set Q = 0 (constant state vector), and σ 2 = 1.

This yields a detailed link with the gradient community [START_REF] Vilmarest | Stochastic online optimization using kalman recursion[END_REF]; the constant state is equivalent to the i.i.d. assumption of gradient analyses in the setting P t|t → 0 (annealing step size). • Dynamic. The natural approach aims at maximizing the likelihood; in that vein, we apply an iterative greedy procedure implemented in the R package viking [START_REF] Vilmarest | viking: State-Space Models Inference by Kalman or Viking[END_REF], that was applied on electricity load forecasting by [START_REF] Obst | Adaptive methods for shortterm electricity load forecasting during covid-19 lockdown in france[END_REF], [START_REF] Vilmarest | State-space models for online post-covid electricity load forecasting competition[END_REF]. It yields a sparse matrix Q assumed diagonal. In that setting, the gradient step does not converge to 0. Note that we apply this procedure to maximize the likelihood on a training set; therefore the variances σ 2 and Q are learned in a batch fashion, but the underlying model assumes a time-varying state θ t leading to an adaptive θt .

C. Adaptation of the Quantile Forecasts

The novelty of the present approach lies in the adaptation of probabilistic forecasts. We compare different methods.

1) Gaussian Posterior from Kalman: We first recall that the Kalman filter does not only yield a mean forecast but already a probabilistic forecast. Indeed, at each time step t, Theorem 1 yields L(θ t | F t-1 ) = N ( θt , P t ). From the state posterior distribution and the observation distribution (5), we deduce

L(y t | x t , F t-1 ) = N ( θ t f (x t ), f (x t ) P t f (x t ) + σ 2
) . (9) It yields a readily computable probabilistic forecast; for any quantile level q, denoting by N q the quantile of the standard normal distribution, we predict the q-quantile of y t as

ŷt,q = θ t f (x t ) + N q f (x t ) P t f (x t ) + σ 2 . ( 10
)
However, this first adaptive forecaster would be an adaptive variant of the offline Gaussian distribution on the offline GAM residuals, where only the GAM is adapted but the variance of the Gaussian distribution is still fixed. The quantile regressions were specifically introduced because the Gaussian assumption with fixed variance is violated in practice. We see in the experiments that this Gaussian forecaster is not always wellcalibrated.

2) Quantile Regressions on Kalman Residuals: We then remark that the offline quantile regressions should work well in the case where the conditional distribution of the residual y t -ŷt given the covariates z t is fixed. The need for adaptive quantile forecasts is motivated only by changes in the residual distribution. Yet a critical property of the Kalman filter is that the residuals are stationary, provided that the state-space model is well-specified. Consequently, the dependence of the Kalman residuals on the quantile covariates should be more stable than that of the offline GAM. Therefore, we combine state-space adaptation of the GAM and offline quantile regressions.

3) Online Gradient Descent on the Pinball Loss: We consider an online quantile regression applying online gradient descent (OGD) on the pinball loss. Precisely, we defined in Section II-A the offline quantile regression for the q-quantile with a vector of covariates z t ∈ R d0 , d 0 > 0:

β q ∈ arg min β∈R d 0 ntrain t=1 ρ q (y t -ŷt , β z t ) . (11)
Motivated by the gradient interpretation of the Kalman filter (Section II-B), we apply the OGD to estimate recursively a vector β t,q . We start from any β 1,q ∈ R d0 and at each step we update it with a step in the direction opposite to the gradient of the loss:

β t+1,q = β t,q -α ∂ρ q (y t -ŷt , β z t ) ∂β βt,q . ( 12
)
The gradient step size α is the important parameter, analogous to the variances that are determining the gradient step in the gradient interpretation of the Kalman filter. We use a constant α in the OGD, and we standardize the covariates z t . Note that the gradient is not well defined for the singular point y t = ŷt + β t,q z t ; we choose to set it to 0 in the remote case that this happens in practice.

4) Bernstein Online Aggregation to Choose the

Step Size: In order to choose the step size in the previous method, we rely on aggregation of experts as proposed by [START_REF] Zaffran | Adaptive conformal predictions for time series[END_REF] and similarly by [START_REF] Van Erven | Metagrad: Adaptation using multiple learning rates in online learning[END_REF]. Our procedure is run separately for each quantile level q using the R package opera [START_REF] Gaillard | opera: Online Prediction by Expert Aggregation[END_REF]. First, we run the OGD with different possible values (α k ) 1≤k≤K ; at each time t, it yields a forecast ŷ(k) t,q for each step size α k . Second, we combine these forecasts using Bernstein Online Aggregation [START_REF] Wintenberger | Optimal learning with bernstein online aggregation[END_REF]; the principle of aggregation is to forecast ŷt,q = K k=1 p (k) t,q ŷ(k) t,q , where the weights p (k)

t,q are obtained sequentially. The properties obtained by the online learning literature guarantee that the total pinball loss of the aggregation has a small regret compared with the total pinball loss of the best expert. A similar approach was applied by [START_REF] Berrisch | Crps learning[END_REF], where the aggregation weights are estimated jointly for all quantiles; the authors assume the weights are smoothed functions of the quantile level and they optimize directly the CRPS instead of point quantile functions.

D. Computational Complexity of the Methodology

The computations of the Kalman filter are separated in two. The estimation of the variances in the dynamic setting is costly (more costly than a single GAM estimation). However, the updates detailed in Theorem 1 are very efficient and their computation on the whole data set is lighter than a single GAM estimation. On the other hand, the cost of the incremental offline GAM is much bigger because each day, we have a GAM estimation. Therefore, the cost of the mean adaptive mean forecast is lower than that of the incremental offline GAM, and is concentrated at the calibration of the variances.

Considering our probabilistic forecasts, note that the OGD and the BOA algorithm are also obtained with a single pass on the data, therefore they are much lighter than re-estimating quantile regressions every day.

Numerical values of computational times are given in the supplementary material [START_REF] Vilmarest | Supplementary materials for "Adaptive Probabilistic Forecasting of Electricity (Net-)Load[END_REF].

III. EVALUATION

For both mean and probabilistic forecasting tasks, we evaluate the forecasts qualitatively as well as quantitatively.

A. Mean Forecast

We evaluate through the root-mean-square-error (RMSE) and the mean absolute error (MAE), defined on a test set T by

RMSE = 1 |T | t∈T (y t -ŷt ) 2 , ( 13
) MAE = 1 |T | t∈T |y t -ŷt | . ( 14
)
Forecasting the mean of the variable of interest is a way to minimize the quadratic loss, thus the RMSE is natural. The MAE is known to be more robust to outliers.

We don't use the mean absolute percentage error (MAPE) because in the case of the net-load the variable may be close to 0 or even negative, see Section IV-A. However, our data sets are composed of different time series to predict and to obtain global evaluation we use relative metrics. We choose to divide by the same metric applied to the mean. Formally, we have N time series (y t,i ) t∈T ,1≤i≤N and corresponding forecasts (ŷ t,i ) t∈T ,1≤i≤N . We estimate the means y i = 1 |T | t∈T y t,i . Our aggregate metrics are defined by

nRMSE = 1 N 1≤i≤N t∈T (y t,i -ŷt,i ) 2 t∈T (y t,i -y i ) 2 , ( 15
) nMAE = 1 N 1≤i≤N t∈T |y t,i -ŷt,i | t∈T |y t,i -y i | . (16)
Our normalized metrics may be interpreted as unexplained variations, in the opposite way of the R-squared.

B. Probabilistic Forecast

There are different ways to evaluate quantile forecasts. However, a necessary condition for the forecast to be meaningful is reliability, also known as calibration. A quantile forecast is reliable if the observed frequency of exceedance coincides with the quantile level. The forecast of a q-quantile is expected to be empirically higher than the quantity of interest for a fraction q of the data set and smaller for a fraction 1 -q. Therefore, we compare the observed frequencies with the quantiles in reliability diagrams.

Reliable forecasts may then be ranked by sharpness. Numerical evaluation is obtained by the pinball loss ρ q defined by Equation [START_REF] Hong | Global energy forecasting competition 2017: Hierarchical probabilistic load forecasting[END_REF]. A way to combine the pinball losses at different quantile levels is to use the continuous ranked probability score (CRPS) [START_REF] Gneiting | Strictly proper scoring rules, prediction, and estimation[END_REF], defined equivalently by the two following expressions:

CRPS(F, y) = +∞ -∞ (F (x) -1 y≤x ) 2 dx = 2 1 0 ρ q (y, F -1 (q))dq , ( 17
)
where y is the observation and F the predicted cumulative distribution function. Remark that for any finite y we have ρ 0 (y, F -1 (0)) = ρ 1 (y, F -1 (1)) = 0, therefore the CRPS is not a good performance indicator for the tail estimation. We use the discrete approximation of the integral in Equation [START_REF] Wintenberger | Optimal learning with bernstein online aggregation[END_REF]. We have a set of forecasted quantiles ŷq1 , . . . , ŷq l , and we define the RPS as the integral of the piecewise linear function interpolating 0, ρ q1 (y, ŷq1 ), . . . ρ q l (y, ŷq l ), 0 at the points 0, q 1 , . . . q l , 1. This yields RPS (ŷ q1 , . . . , ŷq l ), y = l i=1 ρ qi (y, ŷqi )(q i+1 -q i-1 ), [START_REF] Zaffran | Adaptive conformal predictions for time series[END_REF] where we define q 0 = 0, q l+1 = 1. Then we simply use the RPS averaged over time on a test set.

As for mean evaluation, we then aggregate this metric over the multiple time series (regions in IV, cities in V). We divide by the mean absolute error of the mean, which is also the CRPS of a Dirac distribution centered at the mean:

nRPS = 1 N 1≤i≤N t∈T RP S (ŷ t,i,q1 , . . . , ŷt,i,q l ), y t,i t∈T |y t,i -y i | . (19)

IV. REGIONAL NET-LOAD IN GREAT BRITAIN

We first study the data set created by [START_REF] Browell | Probabilistic forecasting of regional netload with conditional extremes and gridded nwp[END_REF]. They studied the data from 2014 to 2018, and we augment the period to range from 2014 to 2021 [START_REF] Vilmarest | Supplementary materials for "Adaptive Probabilistic Forecasting of Electricity (Net-)Load[END_REF], allowing us to integrate the unstable covid period. We refer to [START_REF] Browell | Probabilistic forecasting of regional netload with conditional extremes and gridded nwp[END_REF] and [START_REF] Browell | Supplementary Material for "Probabilistic Forecasting of Regional Net-load with Conditional Extremes and Gridded NWP[END_REF] for more details on the data set. Due to Brexit, the day-ahead electricity price definition changed; we decided to remove this explanatory variable from the data set.

A. Data Presentation and Offline Model

We are interested in forecasting the electricity net-load, defined as the difference between electricity consumption and embedded generation (mostly wind and solar production). We consider the regional data from Great Britain. Great Britain is divided into 14 regions called Grid Supply Point Groups. The time granularity is half an hour and we assume a oneday delay for data availability: each day at midnight we can update the model with data up to 24 hours before. We display in Figure 2 the evolution of the net-load in each region, as well as the daily profiles. We observe different behaviors. In particular, region P (North Scotland) often has a negative netload, meaning the consumption is smaller than the embedded production.

For each region independently, the model designed by [START_REF] Browell | Probabilistic forecasting of regional netload with conditional extremes and gridded nwp[END_REF] to predict quantiles of the net-load is decomposed into three steps. First, a GAM is fitted to forecast the mean. Second, a set of quantile regressions are fitted to the GAM residuals (between 2.5% and 97.5%). Third, extreme quantiles are modeled via a generalized Pareto distribution. The first two match our framework introduced in Section II-A; we apply the same additive model for the mean, then the same quantile regression.

1) We use the formula of the GAM Point of [START_REF] Browell | Probabilistic forecasting of regional netload with conditional extremes and gridded nwp[END_REF], where electricity prices are removed. The normalized electricity net-load is modeled by [START_REF] Hong | Energy forecasting: A review and outlook[END_REF] with the following covariates: a trend, the time of year, the time of day, the day of the week, school holidays, a moving average of the net-load with one-day delay, the temperature at maximum population density, a moving average of that temperature, solar radiation combined with the embedded solar generation capacity, the wind speed combined with the embedded wind generation capacity, and the precipitation.

Weather variables are weather forecasts. All nonlinear effects are built using cubic regression splines. 2) For the quantile regressions, we also use the model of [START_REF] Browell | Probabilistic forecasting of regional netload with conditional extremes and gridded nwp[END_REF]: the GAM residuals are modeled via linear functions of the GAM prediction, the squared prediction, the product of solar radiation and embedded solar capacity, the wind speed, the temperature, as well as of the categorical versions of the time of day and day of the week. For these two latter variables, we have an additive constant defined for each value of the categorical variable. Note that the GAM and the quantile regressions are not trained per half an hour; they directly incorporate the time of day as a covariate.

B. Performances of Mean Forecast

We display the evolution of the error in Figure 3. We improve the mean forecast performance in almost all regions and all years using state-space adaptation, considering the nRMSE, see Figure 4. We display the aggregate nRMSE and nMAE in Table I for different methods and different years. When we compare the dynamic Kalman filter to the incremental offline GAM (model re-trained each day), we reduce the nRMSE by approximately 4% in 2019, 7% in 2020, and 8% in 2021; furthermore, we have a much lower computational cost per day.

C. Performances of Quantile Forecast

We display reliability diagrams in Figure 5. We observe that the offline model is not reliable, probably because of the biases in the mean model. The adaptation of the mean model by the Kalman filter yields Gaussian quantiles that are much more reliable. Reliability is then improved further by the adaptation of the quantile regressions. Similarly to the evolution of the bias of the model (Figure 3), we display in Figure 6 the evolution of the observed frequency for a specific quantile. Our adaptive model has a much better evolution of calibration in time than the offline version. Fig. 2. On the left: evolution of the net-load of the 14 regions. On the right: daily profiles. We observe that in region P (North Scotland) the embedded generation often exceeds the consumption and the daily profile is close to 0. Also, this high level of renewables implies higher weather-driven volatility, and the net-load does not have a clear yearly profile as in the other regions.

A B C D E F G H J K L M N P GAM 2019 GAM 2021 GAM Incremental GAM KF Static GAM KF Dynamic
Fig. 4. nRMSE for each region in Great Britain. We compare the offline GAM to the incremental offline GAM (trained each day on incremental training data), the Kalman filter in the static setting (degenerate covariance matrix of the state noise Q = 0), and the Kalman filter in the dynamic setting. We divide the test set in three (2019, 2020 and 2021). For the last two years, we compare also with the GAM re-trained each year: GAM y is the GAM trained with the data up to year y -1 included. Fig. 6. Evolution of the calibration. We select a specific time of day (midday) and a quantile level 0.8; we display the observed frequency that the net-load exceeds the quantile forecast. The 95% confidence interval is based on the assumption that the residuals are iid. We see that for the offline model (GAM then quantile regression) the confidence interval is not satisfied. We provide in Table II the RPS averaged on the 14 regions for the different models. We obtain an important gain in nRPS by adapting the GAM using the Kalman filter and keeping an offline quantile regression, even for the stable period (the year 2019). Adapting the quantile regression with an OGD has a very tenuous difference. The step size of this OGD may be selected by Bernstein Online Aggregation, combining step sizes in {10 i , -8 ≤ i ≤ 0}. This choice of step size does not only perform as well as the best expert as suggested in Section II-C4, but it outperforms this oracle. Combining both levels of adaptation yields the best performances.

D. Learning the Embedded Capacities

A remarkable advantage of model adaptation is that it reduces the need for good explanatory variables, which may be difficult to obtain. In particular, we show that our Kalman GAM can learn the embedded generation capacities. That is removing these variables does not significantly change the predictions, contrary to the offline method.

More precisely, in the GAM and in the quantile regressions presented in Section IV-A, we remove the solar and wind generation capacities. We consider solar radiation only instead of its product with solar capacity. In the nonlinear GAM effect of the wind speed, we remove the dependence on the embedded wind capacity. We thus evaluate removing the capacities in the offline model, as well as in the Kalman adaptation of the GAM. The evolution of the capacities may be captured by the adaptation of the state coefficient.

We evaluate during 2019 to not include the coronavirus crisis. For mean prediction, removing the capacities increases the nRMSE of the offline method by more than 10%, while it reduces the nRMSE by 0.4% for the Kalman adaptation of the GAM. For probabilistic prediction, removing the capacities increases the nRPS by more than 10% for the offline model and reduces it by 0.02% for the offline quantile regression on the residuals of GAM Kalman.

V. CITY-WIDE LOAD IN THE UNITED STATES

We test our framework also on US data during the coronavirus crisis [START_REF] Ruan | A cross-domain approach to analyzing the short-run impact of covid-19 on the us electricity sector[END_REF]. To demonstrate the generality of the approach, we consider a very different setting. We forecast the standard load at a daily granularity and with no delay in data availability. Each day, we can adapt on the data up to the current day in order to forecast the next.

A. Data Presentation and Offline Model

The data set covers seven U.S. electricity markets, and the major city for each one: Boston, Chicago, Houston, Kansas City, Los Angeles, New York City, Philadelphia. Consider the city-wide loads shown in Figure 8 and note that load patterns are different across cities.

We follow the forecasting structure presented in Section II.

1) The generalized additive model we consider has the following form:

Load t = 7 i=1 α i 1 W eekDayt=i + β 1 BH t + β 2 W B t + β 3 LoadD t + f 1 (LoadW t ) + f 2 (t) (20) + f 3 (T emp t ) + f 4 (Hum t ) + f 5 (T oy t ) + ε t ,
where ε t is a Gaussian i.i.d. noise and for each day t,

• W eekDay t is the day of the week, • BH t (respectively W B t ) is a Boolean denoting if the day is a bank holiday (respectively in the winter break), • LoadD t and LoadW t are lags of the load with a one-day and one-week delays, • t is the day (variable growing linearly with time),

• T emp t and Hum t are the temperature and humidity, • T oy t is the time of year (variable growing linearly from 0 on January 1 st to 1 on December 31 st ). The nonlinear effects f 1 , f 2 , f 3 , f 4 , f 5 are built using thin-plate splines for the first four, and cubic cyclic splines for f 5 . 2) The quantile regression we fit on the residuals uses as covariates the effects of the GAM, the mean prediction, and the square of the mean prediction as in Section IV-A:

z t = ŷt , ŷ2 t , 7 i=1 α i 1 W eekDayt=i , β 1 BH t , β 2 W B t , β 3 LoadD t , f 1 (LoadW t ), f 2 (t), f 3 (T emp t ), (21)
f 4 (Hum t ), f 5 (T oy t ), 1 .

B. Evaluation

We display the nRMSE and nMAE aggregated on the seven cities in Table III. The nRPS is displayed in Table IV.

The best models are still the dynamic setting of the Kalman filter for mean forecasting and the combination of Kalman adaptation of GAM with the adaptive quantile regression for the probabilistic task. However, the performances are significantly different from the results of Section IV. In particular, adapting the mean model with Kalman and then applying offline quantile regression models behaves poorly. It is crucial to adapt the quantile models.

VI. CONCLUSION

In this article, we applied an adaptive procedure for probabilistic net-load forecasting. The proposed methodology relies on several steps: an adaptive mean forecast is obtained by the Kalman filter, and adaptive quantile regressions are derived from Online Gradient Descent. To solve the choice of the step size in the OGD, the algorithm is estimated with multiple learning rates and the estimates are combined using Bernstein Online Aggregation.

Evaluation has been performed based on the electricity net-load in the fourteen regions of Great Britain, and on electricity demand in seven major cities in the United States. Online adaptation of the mean model is always important, yielding reductions in RMSE of around 10% in all case studies compared to updating the model periodically. Additional adaptation of quantile regressions added marginal benefits for regional load forecasting in GB, but substantial benefits for US cities, reducing the RPS by over 20%. This may be due to the fact that the coronavirus crisis has had more impact on daily lives in big cities. Therefore, in our applications, adaptive methods have more interest for big cities. Indeed, we also observe a more substantial gain for region C (London) in the GB data set.

A natural extension would be the adaptation of multivariate probabilistic forecasts. Previous works have considered adaptive dependency structures, but how models for marginal distributions and dependency structures could (and should) be adapted simultaneously has not been studied. Also, we focused on non-extreme quantiles and adaptive estimation of extreme values is an exciting and very challenging topic. A special treatment for extremes may be useful to refine reliability as well as numerical performance.

Fig. 1 .

 1 Fig. 1. Our two-step procedure starts from explanatory variables, computes an intermediate mean forecast, and then estimate the quantile forecast. On the left we describe the offline model (Section II-A and [19]). Our contributions are represented by the right diagram, introducing our adaptive model (Sections II-B to II-C4), in which both steps are adapted.

Fig. 3 .

 3 Fig. 3. Evolution of the standardized error for the offline model trained on data up to December 31, 2018 (left), the incremental offline model trained each day (middle), and the GAM adapted by the Kalman filter (right).

Fig. 5 .

 5 Fig. 5. Reliability diagrams for different methods in 2019 and in 2021.

Fig. 7 .

 7 Fig. 7. Weights optimized by BOA for the forecast of region A in Great Britain at quantile level 0.1.

Fig. 8 .

 8 Fig. 8. Daily electricity load in Boston (left) and Houston (right). The consumption has two annual peaks in summer and winter in Boston, whereas there is only a summer peak in Houston.

TABLE I AGGREGATE

 I METRICS ON THE 14 REGIONS FOR THE DIFFERENT MODELS. WE COMPARE TO PERSISTENCE BENCHMARKS (LAGS OF THE VARIABLE OF

									INTEREST).
									2019		2020	2021
						Forecast		nRMSE nMAE	nRMSE nMAE	nRMSE nMAE
					Persistence (7 days)		0.691	0.589		0.710	0.599	0.737	0.639
					Persistence (2 days)		0.767	0.686		0.755	0.668	0.736	0.668
						Offline GAM		0.356	0.327		0.485	0.453	0.635	0.601
				Incremental offline GAM (yearly)	-	-		0.407	0.376	0.387	0.378
				Incremental offline GAM (daily)	0.338	0.307		0.370	0.344	0.377	0.365
					Kalman GAM (Static)		0.337	0.307		0.374	0.347	0.380	0.368
					Kalman GAM (Dynamic)	0.324	0.292		0.328	0.301	0.332	0.307
				Offline GAM + Offline QR: 2019				Kalman GAM (Gaussian Quantiles): 2019			Kalman GAM + QR OGD (BOA): 2019
		1.0						1.0						1.0
		0.8						0.8						0.8
	Observed frequency	0.4 0.6					Observed frequency	0.4 0.6					Observed frequency	0.4 0.6
		0.2				A B	H J	0.2				A B	H J	0.2				A B	H J
						C	K					C	K					C	K
						D	L					D	L					D	L
						E	M					E	M					E	M
		0.0				F G	N P	0.0				F G	N P	0.0				F G	N P
		0.0	0.2	0.4	0.6	0.8	1.0	0.0	0.2	0.4	0.6	0.8	1.0	0.0	0.2	0.4	0.6	0.8	1.0
				Quantiles						Quantiles						Quantiles
				Offline GAM + Offline QR: 2021				Kalman GAM (Gaussian Quantiles): 2021			Kalman GAM + QR OGD (BOA): 2021
		1.0						1.0						1.0
		0.8						0.8						0.8
	Observed frequency	0.4 0.6					Observed frequency	0.4 0.6					Observed frequency	0.4 0.6
		0.2				A B	H J	0.2				A B	H J	0.2				A B	H J
						C	K					C	K					C	K
						D	L					D	L					D	L
						E	M					E	M					E	M
		0.0				F G	N P	0.0				F G	N P	0.0				F G	N P
		0.0	0.2	0.4	0.6	0.8	1.0	0.0	0.2	0.4	0.6	0.8	1.0	0.0	0.2	0.4	0.6	0.8	1.0
				Quantiles						Quantiles						Quantiles

TABLE II AGGREGATE

 II NRPS ON THE 14 REGIONS FOR THE DIFFERENT MODELS.

		2019	2020	2021
	Offline Method	0.231	0.338	0.454
	GAM Kalman (Gaussian Quantiles)	0.212	0.217	0.222
	GAM Kalman + Offline QR	0.206	0.214	0.217
	Offline GAM + QR OGD (10 -3 )	0.218	0.270	0.293
	Offline GAM + QR OGD (10 -2 )	0.207	0.221	0.218
	Offline GAM + QR OGD (10 -1 )	0.250	0.248	0.293
	Offline GAM + QR OGD (BOA)	0.204	0.211	0.216
	GAM Kalman + QR OGD (10 -2 )	0.205	0.204	0.212
	GAM Kalman + QR OGD (BOA) Saved from:	0.202	0.201	0.209
	http://localhost:10389/session/viewhtml758049ac9a91/index.html?
	viewer_pane=1&capabilities=1&host=http%3A%2F%2F127.0.0.1%3A59503

TABLE III AGGREGATE

 III METRICS IN THE SEVEN US CITIES.

		2020				2021
	Forecast	nRMSE nMAE	nRMSE nMAE
	Persistence (7 days)	0.777	0.688		0.852	0.745
	Persistence (1 day)	0.455	0.417		0.464	0.414
	GAM	0.407	0.400		0.666	0.616
	Incremental GAM (daily)	0.223	0.213		0.214	0.185
	GAM KF Static	0.206	0.195		0.204	0.178
	GAM KF Dynamic	0.194	0.168		0.198	0.166
		TABLE IV
	AGGREGATE NRPS IN THE SEVEN US CITIES.
	Forecast		2020	2021
	Offline method		0.319	0.539
	GAM Kalman (Gaussian Quantiles)	0.122	0.123
	GAM Kalman + Offline QR		0.230	0.391
	Offline GAM + QR OGD (10 -4 )	0.232	0.157
	Offline GAM + QR OGD (10 -3 )	0.145	0.109
	Offline GAM + QR OGD (10 -2 )	0.109	0.196
	Offline GAM + QR OGD (10 -1 )	0.612	1.75
	Offline GAM + QR OGD (BOA)	0.099	0.097
	GAM Kalman + QR OGD (BOA)	0.094	0.094