Clément Elbaz
email: clement.el-baz@def.gouv.fr

Louis Rilling
email: louis.rilling@irisa.fr

Christine Morin
email: christine.morin@inria.fr

Automated Risk Analysis of a Vulnerability Disclosure Using Active Learning

Keywords: Security, CVE, Machine Learning, Active Learning

Exhaustively listing the software and hardware components of an information system is non-trivial. This makes it even harder to analyze the risk created by a vulnerability disclosure in the context of a specific information system. Instead of basing the risk analysis of a newly disclosed vulnerability on a possibly obsolete list of components, we focus on the security team members tasked with protecting the information system, by studying how Chief Information Security Officers (CISOs) and their subordinates actually react to vulnerability disclosures. We propose to use active learning to extract the conscious and unconscious knowledge of an information system's security team in order to automate the risk analysis of a newly disclosed vulnerability for a specific information system to be defended.

Introduction

Disclosure is the most critical part of a vulnerability life cycle. Bilge et al. [START_REF] Bilge | Before We Knew It: An Empirical Study of Zero-day Attacks in the Real World[END_REF] showed a five orders of magnitude increase in the usage of vulnerabilities before and after their disclosure. In the midst of this increasing risk, most standard defense mechanisms do not work at disclosure: software patches have not been deployed and sometimes are not even available yet. Security experts do not understand the vulnerability well enough to author a proper signature rule for an IDS at this early stage. These factors contribute in making the disclosure a dangerous time, leaving many systems vulnerable in practice.

We propose an automated system evaluating, for a newly disclosed vulnerability, the necessity to alert an on-call security operator regarding a new risk in the context of a specific information system to be defended. To this end we intend to let an expert (a CISO or her subordinates) participate in the training of the prediction system by annotating past vulnerabilities to indicate whether she would have wished being alerted about a similar recent disclosure. Unlike a SIEM [START_REF]US10616258B2 -Security information and event management[END_REF], our system does not need to be connected to the monitored information system, as we focus exclusively on public events (vulnerability disclosures) and how they match our expert's interests. We should however fulfill the following constraints:

1. We want to make the best use of past historical vulnerability data in order to predict how to react to future vulnerabilities. 2. We want to make the best use of the limited time of security experts by allocating a fixed annotation budget for the training and selecting the most relevant vulnerabilities to be annotated. [START_REF]Common Vulnerabilities and Exposures (CVE)[END_REF]. In order to keep the operator's trust in the prediction system we want both the vulnerability selection process during the training stage, and the vulnerability analysis process during the inference stage, to be explicable.

In Section 2 we discuss background and related work. In Section 3 we present our approach. In Section 4 we propose a preliminary evaluation of our technique. We discuss our results in Section 5 and conclude in Section 6.

Background and Related Work

The process of disclosing new vulnerabilities is coordinated by the Common Vulnerabilities and Exposures (CVE) system overseen by Mitre's Corporation [START_REF]Common Vulnerabilities and Exposures (CVE)[END_REF]. Newly disclosed vulnerabilities are first published on the CVE List data feed (managed by Mitre) then forwarded to other security databases such as NIST's NVD database [START_REF]National Vulnerability Database[END_REF] or SCAP data feeds [START_REF]Security Content Automation Protocol[END_REF]. Only then they will be annotated by multiple security experts. As of September 2021, more than 160 000 vulnerabilities have been disclosed by CVE and analyzed by NVD. This has allowed the research community to explore the vulnerability life cycle at scale [START_REF] Frei | Large-scale Vulnerability Analysis[END_REF][START_REF] Clark | Familiarity Breeds Contempt: The Honeymoon Effect and the Role of Legacy Code in Zero-day Vulnerabilities[END_REF][START_REF] Shahzad | A Large Scale Exploratory Analysis of Software Vulnerability Life Cycles[END_REF][START_REF] Frei | Modeling the security ecosystem -the dynamics of (in)security[END_REF] and converge on three insights. First, the relationship between attacks and vulnerabilities follows a power-law, with a minority of vulnerabilities being exploited in a majority of attacks, and most vulnerabilities never being exploited at all [START_REF] Nayak | Some vulnerabilities are different than others[END_REF][START_REF] Allodi | Economic factors of vulnerability trade and exploitation[END_REF]. The actual rate of vulnerability exploitation has been subject to discussion, with figures going from 15 % [START_REF] Nayak | Some vulnerabilities are different than others[END_REF] to 1.3 % [START_REF] Sabottke | Vulnerability disclosure in the age of social media: Exploiting twitter for predicting real-world exploits[END_REF] having been reported in the literature. Second, too many vulnerabilities are disclosed at any time for them to be treated with an equal level of urgency. More than 18000 vulnerabilities were disclosed by CVE [START_REF]Common Vulnerabilities and Exposures (CVE)[END_REF] in 2018 and 2019, around 50 vulnerabilities per day on average. Prioritization is required. Third, as mentioned in Section 1, usage of a vulnerability in the wild can increase as high as five orders of magnitude between before and after vulnerability disclosure [START_REF] Bilge | Before We Knew It: An Empirical Study of Zero-day Attacks in the Real World[END_REF].

These three facts taken together led the community to a conclusion: vulnerability exploitation in the wild, both in the present and the future, is the most important property of a vulnerability to predict. By properly predicting exploitation in the wild, one can efficiently prioritize limited mitigation resources while still preventing most attacks. Starting from 2010, most efforts in the community were devoted to finding accurate predictors of exploitation, with attempts focusing on CVSS [START_REF]Common Vulnerability Scoring System[END_REF][START_REF] Houmb | Quantifying security risk level from cvss estimates of frequency and impact[END_REF][START_REF] Allodi | A preliminary analysis of vulnerability scores for attacks in wild: The ekits and sym datasets[END_REF][START_REF] Allodi | Comparing vulnerability severity and exploits using case-control studies[END_REF], OSVDB [START_REF]OSVDB Shut Down Permanently -Securityweek.com[END_REF][START_REF] Bozorgi | Beyond heuristics: Learning to classify vulnerabilities and predict exploits[END_REF], exploit black-markets [START_REF] Allodi | Comparing vulnerability severity and exploits using case-control studies[END_REF], social networks [START_REF] Sabottke | Vulnerability disclosure in the age of social media: Exploiting twitter for predicting real-world exploits[END_REF][START_REF] Mittal | Cybertwitter: Using twitter to generate alerts for cybersecurity threats and vulnerabilities[END_REF] and online forums [START_REF] Tavabi | Darkembed: Exploit prediction with neural language models[END_REF]. These prediction efforts culminated with the release in 2019 of the Exploit Predicting Scoring System (EPSS) [START_REF] Jacobs | Exploit Prediction Scoring System (EPSS)[END_REF], which uses many data sources to train a regression model answering a simple question: what is the probability of a vulnerability being exploited in the wild in the twelve months following its disclosure?

All of these models are based on data usually generated after disclosure, such as human discussion or exploit trading happening after the vulnerability has already gone public for some time. While some of these models can provide reasonable results days after the disclosure, the ability to predict the danger posed by a vulnerability in the minutes or seconds after its disclosure has not improved significantly. This is a problem as some major vulnerabilities such as Shellshock [START_REF] Wheeler -Shellshock | [END_REF] were exploited within one hour of their disclosure. Therefore our own previous work [START_REF] Elbaz | Reacting to "n-day[END_REF] has focused on predicting multiple properties of a vulnerability, such as the affected software [START_REF] Elbaz | Automated keyword extraction from "one-day" vulnerabilities at disclosure[END_REF] or its CVSS vector [START_REF] Elbaz | Fighting n-day vulnerabilities with automated cvss vector prediction at disclosure[END_REF] immediately after disclosure. Moreover, for a specific organization, exploitation in the wild is not a sufficient criterion to act, as many vulnerabilities affect software that is not used in the organization's information systems.

Cataloging an information system exhaustively is hard. Software has many forms and many locations: while software binaries reside on hard disks, firmware is embedded inside hardware, and uncompiled source code can be interpreted at run time. Software can be downloaded on the fly over the network without touching the disk, such as when browsing the web with Javascript enabled. Hardware can be vulnerable [START_REF] Kocher | Spectre attacks: Exploiting speculative execution[END_REF][27] and is not easily updated or replaced. Databases such as the Common Platform Enumeration (CPE) [START_REF]NVD -CPE[END_REF] (which references all software and hardware ever affected by a vulnerability) can in theory be used to track and monitor all these components but in practice discrepancies between the real world and CPE entries (which are authored manually) quickly accumulate to the point of becoming unmanageable [START_REF]The Myth of Software and Hardware Vulnerability Management[END_REF]. This problem also impacts software versioning: as an example the Debian security team [START_REF]Debian -Security Information[END_REF] is known to backport security fixes into Debian packages without waiting for official releases of the upstream software, creating a Debian-specific version number while doing so. While this practice should be lauded for improving the timeliness of Debian security updates, it adds confusion about how a given version of a software is supposed to behave.

The evolution of cybersecurity threats led organizations to pursue continuous protection through the creation of Security Operation Centers (SOCs) dedicated to ensuring their ongoing safety. Ideally, every organization would benefit from being protected by a SOC, either internal or outsourced. However, this is a challenging goal in practice, as hiring security analysts is difficult and expensive [START_REF]The 2019 Official Annual Cybersecurity Jobs Report[END_REF][START_REF]The Mad Dash to Find a Cybersecurity Force -The New York Times[END_REF] and the profession has a documented history of elevated stress and burnout [START_REF] Dykstra | Cyber operations stress survey (coss): Studying fatigue, frustration, and cognitive workload in cybersecurity operations[END_REF][START_REF] Sundaramurthy | An anthropological approach to studying csirts[END_REF][START_REF] Sundaramurthy | A human capital model for mitigating security analyst burnout[END_REF][START_REF] Ogbanufe | Burnout in cybersecurity professionals, WISP 2019 -Workshop in Information Security and Privacy[END_REF][START_REF] Chappelle | Sources of Occupational Stress and Prevalence of Burnout and Clinical Distress Among U.S. Air Force Cyber Warfare Operators[END_REF]. Therefore, at a global level it is neither possible to hire more security analysts nor to give them much more work to do. These facts taken together mean that the human component of cybersecurity defense is globally saturated, and the only way to meet the ever-increasing demand for real-time cybersecurity defense is through automation.

Active learning [START_REF] Settles | Active learning literature survey[END_REF] is a hybrid class of machine learning where training is an interactive process between an automated learner and an oracle such as a human expert. The learner starts with a set of unlabeled data. It then has to solve two learning problems: the first is to select the next sample to be submitted to the oracle for labeling, and then to use both the labeled and unlabeled data to make predictions. An important characteristic of active learning is training interactivity: each label provided by the oracle should impact which sample should be submitted next. Active learning is an interesting choice for information security as the same channel between the learner and the oracle can be used for both training and evaluation: once the system is trained, random unlabeled samples can be sent to both the learner and the oracle to compare their answers. The oracle, the learner, the data and its annotations can all remain in one security context, avoiding the need to share sensitive datasets between multiple security contexts. An example of active learning for security is ILAB [START_REF] Beaugnon | Ilab: An interactive labelling strategy for intrusion detection[END_REF], a framework designed by Beaugnon et al aiming to facilitate the annotation of intrusion detection datasets.

Proposed Approach

We propose an active learning architecture based on the work by Settles et al [START_REF] Settles | An analysis of active learning strategies for sequence labeling tasks[END_REF], which we selected because of its simplicity and explicability properties. Our architecture is depicted in Figure 1. It is composed of a training phase during which an expert and the system train together and an inference phase during which the system monitors new vulnerabilities and evaluates the risk that they create. Since the active learning architecture originally proposed by Settles et al is deterministic and only handles static datasets, we made slight modifications to the scheme (detailed in Sections 3.6 and 3.7) to handle the dynamic nature of an ongoing flow of vulnerability disclosures and support a controllable amount of randomness during the training process.

During the training phase the system iteratively selects a relevant past vulnerability, submits it to the expert through the user interface, who then annotates it by choosing an appropriate alert level for the vulnerability. The training phase is actually split in two parts. First, the offline training phase happens during the deployment of a production prediction system at a fixed point in time, and lets the prediction system and the expert create together an initial knowledge base by letting the prediction system select any past vulnerability at that point in time, with the system having full control on the vulnerability selection process. Second, the online training phase is a continuous interaction between the prediction system and the expert over time, discretized into periods such as weeks. Every time a new period starts, the prediction system iteratively selects new vulnerabilities among the ones disclosed during the last period and makes the expert annotate them. Each of these phases has a separate annotation budget, with the offline phase requiring a one-off effort from the expert and the online phase requiring a continuous effort over time. These two phases of the training are important, as they allow the prediction system to acquire both long-term context on past vulnerability data as well as awareness of the latest vulnerability disclosures.

Once the expert considers that the system has been made reliable enough through offline and online training (we come back to this in Section 4), the inference phase can begin. The system now monitors new vulnerability disclosures in near real time and chooses an appropriate alert level for them using the knowledge base it has constructed together with the expert.

The processing of a vulnerability is made of several building blocks, some of them common to both the training and inference phases, and some specific to one or the other, as detailed below. During the inference phase, evaluating the risk created by a new vulnerability is done by comparing it to similar, annotated vulnerabilities. To this end a similarity metric between vulnerabilities is needed. This similarity metric is made possible by viewing vulnerabilities as euclidean vectors through a featurization stage.

During the training phase, selecting relevant vulnerabilities to be annotated by the expert is based on the same building blocks. Our similarity metric is used to compute a similarity matrix for the entire past vulnerabilities dataset, which then lets the learning system compute an information density metric for every past vulnerability. Selecting a vulnerability to be annotated by the expert is done by combining this information density metric with an uncertainty metric computed by simulating an alert decision process for every vulnerability while measuring the confidence of the prediction.

In the rest of this section we first present the concepts necessary for both training and inference: in Section 3.1 we describe the possible alert levels for a vulnerability and the user interface used by the expert. In Section 3.2 we discuss the featurization stage of our architecture. In Section 3.3 we discuss our choice of similarity metric. The actual alert decision process used during the inference phase is then presented in Section 3.4. We finally cover the remaining concepts necessary for the training phase. In Section 3.5 we present our choice of uncertainty metric. We then discuss concepts proposed by Settles et al [START_REF] Settles | An analysis of active learning strategies for sequence labeling tasks[END_REF] that we use: first the similarity matrix and information density metric in Section 3.6, then in Section 3.7 the process used by the learning system to select the next vulnerability to be annotated by the expert during the training phase.

User Interface and Alert Levels

An alert level scale is needed to let both the system and the expert translate their risk analysis of a vulnerability into an actionable decision. The recipient of this alert is a human, so it makes sense for the alert scale to be human-centered. We use the scale proposed by Google in their Site Reliability Engineering doctrine [START_REF] Beyer | Site Reliability Engineering: How Google Runs Production Systems[END_REF]:

• LOG: The disclosure is logged in an activity log that can be consulted afterward, but no alert is actively raised to a human security operator. • TICKET: The disclosure causes the creation of a ticket in an issue tracking system. A ticket is expected to be solved in a non-urgent manner during working hours. • PAGE: The disclosure causes an on-call security operator to be paged immediately, even outside regular working hours. A page is expected to be treated immediately.

As shown in Figure 2, in the training phase the system selects a vulnerability to be labeled by the human expert then shows its description on screen. The expert then selects the alert level she deems the most appropriate for the vulnerability, in the context of the system she is tasked to protect. In the inference phase, the system automatically selects the appropriate alert level for new vulnerabilities using the same alert scale.

Vulnerability as a Euclidean Vector

In order to transform the raw description of a vulnerability into a euclidean vector, we reuse the same keyword extraction pipeline as described in our past work [START_REF] Elbaz | Automated keyword extraction from "one-day" vulnerabilities at disclosure[END_REF]: we retain only words present in past CPE URIs then weight them using an algorithm called TF-IDF [START_REF] Jones | A statistical interpretation of term specificity and its application in retrieval[END_REF].

TF-IDF is a numerical statistic reflecting the importance of a word in a document, in the context of a corpus. It is defined in Equation 1where 𝑡 is a word and 𝑑 is a document belonging to a corpus of documents 𝐷.

TF-IDF(𝑡, 𝑑, 𝐷) = TF(𝑡, 𝑑) × IDF(𝑡, 𝐷) (1)
TF is based on the number of occurrences of the word in the document. Several formulas have been proposed, such as using the raw count directly or treating the presence or absence of a word as a boolean event. Applying a logarithm on top of the raw count of occurrences has been argued [START_REF]Term Frequency -Inverse Document Frequency statistics[END_REF] to better reflect the diminishing returns of repeating the same term several times.

IDF is defined in Equation 2:

IDF(𝑡, 𝐷) = log |𝐷| |𝑑 ∈ 𝐷 ∶ 𝑡 ∈ 𝑑| ( 2
)
where |𝐷| is the number of documents in the corpus, and |𝑑 ∈ 𝐷 ∶ 𝑡 ∈ 𝑑| is the number of documents of the corpus containing the word 𝑡. TF-IDF therefore allows more specific words to have a bigger impact on the mapping than common words. This weighting is then improved using domain-specific heuristics presented in [START_REF] Elbaz | Automated keyword extraction from "one-day" vulnerabilities at disclosure[END_REF]. We finally use a euclidean truncation scheme to reduce the number of non-zero components for every vector. This pipeline lets us identify the affected software for a vulnerability in the form of a sparse euclidean vector (with a low number of non-zero vector components). Vulnerabilities affecting the same software share many non-zero components, making them similar in the vector space.

Measuring Vulnerability Vectors Similarity

As a similarity metric between vulnerability euclidean vectors we choose the cosine similarity that can be computed using the formula shown in Equation 3:

cosine similarity(x, x ′ ) = 𝑛 ∑ 𝑖=1 x 𝑖 x ′ 𝑖 ‖x‖‖x ′ ‖ (3)
Cosine similarity returns a similarity between 0 and 1 for two vectors. Its main avantage is to return a similarity of zero for vectors having no common non-zero components. This is helpful to avoid creating artificial similarity between unrelated vulnerabilities. In particular, distancebased similarity metrics tend to overestimate the similarity between vectors close to origin. In our case, a vector close to origin indicates a vulnerability for which we have not extracted any meaningful keyword and therefore have no strong understanding of. Therefore having two vectors being close to origin does not necessarily means that the two related vulnerabilities are actually similar to each other. Another advantage of cosine similarity is to not require any hyperparameter, making it more explicable and stable than similarity metrics that do.

Alerting Decision Score

For every new vulnerability disclosure, the prediction system has to make a risk estimate by choosing an appropriate alert level for the vulnerability, among LOG, TICKET and PAGE. In order to do that, we had to make several hypotheses. These hypotheses can be challenged, as we discuss in Section 5.

Our first hypothesis is that the TICKET alert level is a reasonable default reaction for a vulnerability about which nothing is known. This is based on the assumption that security operators are risk averse and want to eventually review any vulnerability for which the prediction system could not make a meaningful decision. In the light of this hypothesis, we propose an alert decision score in range [-1; +1]. It can be converted into an alert decision for a vulnerability 𝑣 using Equation 4.

decision(𝑣) = ⎧ ⎨ ⎩ 𝐿𝑂𝐺, if score(𝑣) ∈ [-1; -1 3 [ 𝑇 𝐼 𝐶𝐾 𝐸𝑇 , if score(𝑣) ∈ [-1 3 ; + 1 3 [ 𝑃𝐴𝐺𝐸, if score(𝑣) ∈ [+ 1 3 ; +1] (4)
Conversely, a default alert decision score is set for all annotated vulnerabilities. The alert decision score of an annotated vulnerability 𝑎𝑣 is set following Equation 5.

score(𝑎𝑣) = ⎧ ⎨ ⎩ -1, if the vulnerability is annotated as LOG 0,
if the vulnerability is annotated as TICKET +1, if the vulnerability is annotated as PAGE [START_REF]Security Content Automation Protocol[END_REF] The formula to compute the decision score of a non-annotated vulnerability 𝑣 is shown in Equation 6, assuming a knowledge base of 𝑘 annotated vulnerabilities 𝑎𝑣 0 to 𝑎𝑣 𝑘-1 :

score(𝑣) = ⎧ ⎪ ⎪ ⎨ ⎪ ⎪ ⎩ 𝑘-1 ∑ 𝑖=0 score(𝑎𝑣 𝑖 ) × similarity(𝑣, 𝑎𝑣 𝑖 ) 𝑘-1 ∑ 𝑖=0 similarity(𝑣, 𝑎𝑣 𝑖 ) , if ∃𝑖 ∈ {0, ..., 𝑘 -1} similarity(𝑣, 𝑎𝑣 𝑖 ) > 0 0, otherwise (6)
Put another way, the alert decision score is the average decision score for all annotated vulnerabilities weighted by their similarity to the analyzed vulnerability, with a special case of returning 0 when there are no similar annotated vulnerabilities yet.

This formula implies that any vulnerability for which there are few or no similar annotated vulnerabilities gets an alert decision score close or equal to 0, resulting in a TICKET alert, in line with our hypothesis that this is the correct default reaction. Conversely, LOG and PAGE alerts are only emitted if the vulnerability is strongly similar to known vulnerabilities that were annotated as such.

Uncertainty Score

The active learning architecture we use [START_REF] Settles | An analysis of active learning strategies for sequence labeling tasks[END_REF] requires an uncertainty metric for each vulnerability, as part of the vulnerability selection process used to submit vulnerabilities to be annotated by the expert (which we discuss in Section 3.7). However they do not specify how to implement it.

We make the hypothesis that since TICKET is our default reaction, the closer to TICKET a vulnerability is, the more uncertain our decision is. Conversely, the closer to LOG or PAGE a decision is, the more certain our decision is. Therefore we compute uncertainty using Equation 7.

uncertainty(𝑣) = 1 -|score(𝑣)| (7)
Therefore the uncertainty score of a vulnerability reaches 1 the closer its decision score is to 0 (TICKET), and reaches 0 the closer its decision score is to -1 or +1 (LOG or PAGE, respectively).

Similarity Matrix to Measure Information Density

We first describe as background the concepts of similarity matrix and information density proposed by Settles et al [START_REF] Settles | An analysis of active learning strategies for sequence labeling tasks[END_REF], as they are part of our architecture. We then present a slight modification to the scheme we add in the context of a dynamic dataset such as ongoing vulnerability disclosures.

Background: Similarity Matrix and Information Density

Assuming a set of 𝑛 past vulnerabilities 𝑣 0 to 𝑣 𝑛-1 , a similarity matrix 𝑆 of size 𝑛 × 𝑛 can be constructed using Equation 8.

∀𝑖, 𝑗 ∈ {0, ..., 𝑛 -1} 𝑆 𝑖,𝑗 = similarity(𝑣 𝑖 , 𝑣 𝑗 )

From the similarity matrix 𝑆 we compute an information density metric for every vulnerability by computing the average similarity of a vulnerability to every other vulnerabilities in the dataset, including itself (therefore no vulnerability can have an information density of zero). This is described by Equation 9.

∀𝑖 ∈ {0, ..., 𝑛 -1} density(𝑣 𝑖 ) = 𝑛-1 ∑ 𝑗=0 𝑆 𝑖,𝑗 𝑛 (9)
Information density is a measure of how typical a vulnerability is. When a lot of vulnerabilities resemble each other, they all have a high similarity metric when compared to each other, increasing the information density of each of them. Conversely, an outlier vulnerability with few or no similar vulnerabilities has a low information density.

Vulnerability Dataset Truncation

Settles et al proposed the concept of similarity matrix with a static dataset in mind. However, as many new vulnerabilities are disclosed every day, in our case the dataset is dynamic and the creation and update of a similarity matrix are quadratic through time and space. To limit the cost, we modify the scheme proposed by Settles et al and decide to truncate our vulnerability dataset once it goes over a certain size. We define a hyperparameter 𝑁 for the maximum number of vulnerabilities we find acceptable, then remove excess vulnerabilities once 𝑛 > 𝑁. For choosing the vulnerabilities to be truncated we simply remove the oldest vulnerabilities first.

Selecting a Vulnerability to be Evaluated

The main decision task of the training phase is to select the next vulnerability to be annotated by the expert. This raises several challenges. Selecting only the most typical vulnerabilities can lead to submit many vulnerabilities similar to each other, with diminishing returns after the first few ones. Conversely, selecting vulnerabilities only based on decision uncertainty leads to submit many outliers to the expert, which does not help with getting a broader understanding of the entire vulnerability set. Another question is how much the selection process should be based on randomness. A solely deterministic process could get the learning stuck in local maxima of the vulnerability euclidean space, while leaning too much on randomness might lead to submit too many uninteresting vulnerabilities to the expert.

To address all these challenges, we propose an approach based on the work of Settles et al [START_REF] Settles | An analysis of active learning strategies for sequence labeling tasks[END_REF] which we present as background. We then propose a modification to incorporate randomness in the selection strategy.

Background: Selection Score and Selection Strategy

Settles et al [START_REF] Settles | An analysis of active learning strategies for sequence labeling tasks[END_REF] proposed that every vulnerability eligible for annotation should be given a selection score according to a selection strategy. They proposed several strategies for the selection score and we choose the simplest one as a starting point, shown in Equation 10. This strategy does not include randomness yet, which we describe in the next section. select(𝑣) = density(𝑣) × uncertainty(𝑣) [START_REF] Nayak | Some vulnerabilities are different than others[END_REF] As seen in Section 3.5, when a vulnerability has no similar annotated vulnerabilities, it gets an uncertainty score of 1. Therefore, at the beginning of the training (when there are no annotated vulnerabilities yet) all vulnerabilities get the same uncertainty score, and the selection process is based on information density only (we show below how we add randomness). A large group of vulnerabilities similar to each other (and therefore with high information density) is called a vulnerability cluster. Once more vulnerabilities get annotated inside the biggest clusters, the uncertainty score of the unannotated vulnerabilities in the clusters progressively decreases as there are similar annotated vulnerabilities in the knowledge base. At some point, other smaller clusters with lower information density but higher uncertainty (as they are unexplored yet) reach higher selection scores and get picked up first.

Adding Randomness to the Selection Process

With no randomness at all, the vulnerability selection is entirely deterministic, assuming a fixed set of vulnerabilities and an expert always choosing the same alert level for the same vulnerability. This can create problems as some vulnerability clusters are much denser than others, leading the selection process to waste too much of the expert's time on too few clusters. However, selecting a vulnerability from a smaller pool of randomly chosen vulnerabilities allows for more exploration of the vulnerability dataset as the denser clusters are not always present in the random pool. Therefore in our work we slightly modify the selection strategy proposed by Settles et al [START_REF] Settles | An analysis of active learning strategies for sequence labeling tasks[END_REF] to incorporate a certain amount of randomness by randomly selecting 𝑟 vulnerabilities from the set of all candidates vulnerabilities to be annotated, before applying the rules described in Equation 10 to choose a vulnerability from this random subset. 𝑟 is a hyperparameter. We claim that taken together, randomness, uncertainty, and information density provide the foundation for a robust annotation selection process.

Our complete training algorithm, including the vulnerability selection process is described in Algorithm 1.

Input: V: set of all past vulnerabilities

Preliminary Evaluation

Our evaluation goal is to check the real-world applicability of our prediction system, designed and configured with preliminary hypotheses, by confronting it to actual security experts in charge of real information systems. In our experimental protocol, these experts annotate vulnerabilities to assert their risk levels, in the context of the systems they are responsible for. The experiment is divided in two steps: in the first step, security experts train the prediction system, through an offline training followed by an online training, by simulating the passing of time. Second, they evaluate the prediction system: this is done by randomly selecting vulnerabilities, then submitting them to the prediction system and the expert simultaneously while comparing their answers. The main difference between the two steps is that during training the system controls which vulnerabilities are submitted to the expert, and the resulting annotations are added to the system's knowledge base. On the contrary during evaluation the vulnerability selection is completely random and the expert's annotations are only used as an evaluation tool without being added to the system's knowledge base.

A major challenge of this experiment is the requirement for busy security experts to commit a certain amount of time to participate, such as half a day. Moreover, as the training's vulnerability selection process simultaneously depends on the active learning architecture, its hyperparameters, and the expert previous choices, any iteration in the design or configuration of the prediction system requires to undertake a brand new experiment (including the expert full participation) to properly evaluate the changes.

In the end we have been able to complete two experiments, which we denote Experiment A and B. The information system studied in experiment A is a server-side software development environment, including a git server, an issue tracker, NTP and DNS servers, as well as Windows and Linux machines using a Kerberos-based authentication protocol. The information system studied during experiment B was also an information system for software development, with Linux machines only. It includes a git server, a Jenkins server, an issue tracker, a wiki system, and is administered through SSH.

A shortcoming of this preliminary evaluation is that both experiments were done as part of a first experimental campaign, and we have not been able to secure the launch of a second experimental campaign based on what we learned during the first one. This campaign was based on initial hypotheses we took, some of which turned out to be questionable, resulting in mixed evaluation results. This is further discussed in Section 5.

We present our evaluation protocol in Section 4.1 and the results of the experiments in Section 4.2.

Experimental Protocol

Our experimental protocol uses past CVE vulnerabilities to simulate the production deployment of an alert system at a random past date. The protocol starts with an initial offline training session, then simulates the passing of time for an online training session, followed by an online evaluation session.

A major experimental constraint was that vulnerability annotations authored by our participants were actually sensitive themselves, as they provided a lot of insights into the related information systems. This required the whole campaign to be carried out without us ever seeing those annotations, or storing them on our own hardware. Therefore experiments were done exclusively on security experts' own machines (usually laptop workstations), preventing the use of dedicated server-grade computing resources. For this reason the vulnerability selection process, which is computationally intensive, had to be bounded in time and resources, as the experience had to stay interactive enough for the security expert to remain engaged while running entirely from a lightweight workstation. This is not straightforward as a similarity matrix for the entire CVE dataset requires hundreds of megabytes of storage, takes minutes to compute, and is only valid for a specific time step in the simulation as new vulnerabilities and keywords are added and discarded. We solved this engineering problem by storing all similarity matrices needed for every time step of the simulated timeline on the experts' workstations, ensuring the interactivity of the experience at the cost of each experiment requiring several hundreds of gigabytes of storage.

The experimental protocol was prepared in 2019, giving us access to all CVE vulnerabilities disclosed between 2007 and 2018. However, as explained in Section 3.6, the computation and storage of the vulnerability similarity matrix is quadratic in the number of vulnerabilities in our database, and differs at every time step (as more vulnerabilities are disclosed and more CPE URIs are added to the featurization word list). The hyperparameter 𝑁, first described in Section 3.6, is the maximum number of vulnerabilities the prediction system retains in its database. We empirically found that setting 𝑁 = 10000 was close to the highest value we could set while keeping the user experience interactive using our current prototype. As we discard older vulnerabilities first and more than 5000 vulnerabilities have been disclosed every year for the last decade, in practice only vulnerabilities disclosed between 2014 and 2018 are actually used in the experiment.

Regarding the annotation budget, we had initial discussions with potential participants about the amount of time they would be able to commit to the experiments. From early participant feedback and preliminary empirical tests, we converged on an experimental duration of half a day consisting in annotating 300 vulnerabilities, an amount to be divided into offline training, online training, and online evaluation. This budget was the best compromise between participant's availability and experimental validity.

Our first intuition was to divide the annotation budget evenly between offline training, online training, and online evaluation, providing a budget of 100 annotations for each step. For online training and evaluation, we also have to divide the budget into a number of weeks of simulated time and a number of vulnerabilities to be annotated per week. For online evaluation we settled on an even divide of 10 annotations per week during 10 weeks of simulated time. However early informal experiments highlighted a potential problem for online training: it is common for many vulnerabilities affecting the same software or hardware to be disclosed simultaneously, creating a risk of wasting part of the training budget as most vulnerabilities disclosed in a single period are affecting the same software. For this reason, we decided to spread the online training budget on more weeks, with only 8 annotations per week during 12 weeks of simulated time, for a total of 304 vulnerability annotations in the entire experiment.

The last hyperparameter to set was 𝑟, the number of randomly picked vulnerabilities to be ranked during the vulnerability selection process. Early informal experiments highlighted the risk of getting stuck into local maxima when carrying a mostly deterministic vulnerability selection process. Therefore we decided to let randomness have an important role in vulnerability selection by setting 𝑟 = 10. We hypothesized that this setting would let the training process propose varied vulnerabilities to the expert, while still having a high probability of having at least one interesting vulnerability to submit to the expert among the ten random ones.

Results

Evaluation results for experiments A and B are shown in Table 1 and2. Correct decisions are shown in the diagonal row. False positives (for which the system chose a higher alert level than the expert) are shown above the diagonal, while false negatives (for which the system chose a lower alert level than the expert) are shown below.

Results show that our approach can be improved: less than half of the decisions are identical between the system and the expert, with a very high number of false positives yet a significant amount of false negatives. The most common type of false positives is a vulnerability deemed benign by the expert (LOG, no alert raised) for which the system created a non-urgent TICKET for inspection during working hours. This is a consequence of the low base rate of non-LOG vulnerabilities (experts for experiment A and B respectively classified 85 % and 77 % of evaluation vulnerabilities as LOG) coupled with our design choice of favoring TICKET as a default response.

As we said in Section 4.1, we did not get access to the expert's annotations for neither experiment A or B. However, for experiment B we had the opportunity to follow up with the expert's team to get some insight into the evaluation process. An important discovery made during this follow-up is that at least five false negatives from experiment B are actually human errors instead of prediction errors (the five vulnerabilities annotated as PAGE by the expert were incorrectly annotated during the evaluation and the participant actually agrees with the system's decisions). According to the participant, this is partly due to a specific keyword in the vulnerability description that incorrectly startled him, and partly due to annotation fatigue, which we discuss in Section 5. We chose not to correct our results following this information for two reasons: first, there have been no comprehensive review of all evaluation vulnerabilities for neither experiments, leaving the possibility of even more undetected human errors. Moreover, we consider expert error a real-world condition that should be acknowledged by a robust evaluation protocol.

Discussion

As said previously, for both experiments A and B we did not get access to the 304 vulnerabilities submitted to the expert, nor the annotations provided by the expert, for confidentiality reasons. Under these conditions we can only speculate about what went right or wrong during the automated decision process, and can outline some design changes we would propose in the advent of another experimental campaign.

Experimental Limitations

It is likely that a budget of 304 vulnerabilities is not enough for both training and evaluating our prediction system. It is noteworthy that in the context of a production prediction system, having a security operator annotate vulnerabilities one hour a week for a year would result in an order of magnitude increase in annotation budget compared to our experimental campaign. We do not know how such an increase in budget would affect decision accuracy.

Decision Quality

For this work we only used dimensions generated by the keyword extraction pipeline described in our previous work [START_REF] Elbaz | Automated keyword extraction from "one-day" vulnerabilities at disclosure[END_REF], with past CPE URIs used as a filtering whitelist to prevent too many noisy keywords. This raises two risks: the quality of the extracted names may not be good enough for accurate decisions and comparing vulnerabilities by affected software alone may not be enough for proper risk analysis. As we could not study ourselves the training and evaluation data, we delivered a debugging tool to experimenters allowing them to look back at any evaluation vulnerability. Using this tool they could list the training vulnerabilities that were considered similar, and the keywords from which the similarity arose. While they did not have time to do a comprehensive analysis of all evaluation decisions, they did report some valuable insights. Many decision mistakes were due to incorrect links between vulnerabilities, due to noisy keywords that are nevertheless present in the CPE URI whitelist: this is the case for common technical words such as "internet" (present in the whitelist because of many software names starting with Internet Explorer), "api" (because of software names such as Broker API or API connect), "credentials" (because of software names such as Credential Bindings). It could be worthwhile to improve the filtering capabilities of our keyword extraction pipeline, as well as giving security experts the opportunity to blacklist keywords that led to incorrect decisions in the past. Last but not least, it would be interesting to evaluate how adding other vulnerability properties would help risk analysis.

Expert Expectations

When explaining our experimental protocol to participants we did our best to convey that the current prediction system is only based on affected software names, and does not take into account other properties such as affected versions or vulnerability severity. However we never met directly the participant for experiment A (we presented the protocol to his colleagues) and more than a year had passed between initial project presentation and the starting of experiment B. This means participants may have forgotten (or were never aware to begin with) of this limitation while participating in the experiment, and participant's feedback following the experiments left this point unclear. If it was confirmed that security experts tend to include other information such as version or severity into their vulnerability analysis, this would be a strong indicator that more vulnerability properties are needed to increase prediction accuracy.

Another point where our system may have differed from participants' expectations is our choice to consider TICKET as a default choice. Our initial hypothesis was that false negatives are worse than false positives, as they imply the presence of an unmitigated risk for the information system. Many security experts we talked to disagreed with us: most of them told us that they had too many alerts from too many monitoring systems to handle them all, and they did not have time to check all newly disclosed CVE vulnerabilities manually. Therefore they strongly prefer some false negatives (which is not worse than their current situation) to false positives (which create more alerts for them to handle). A future version of this prediction system could allow the possibility for the security expert to choose a default alert behavior she feels appropriate.

Hyperparameters Tuning

Our learning system and evaluation protocol both require many hyperparameters, creating a hyperstate space too vast to be explored comprehensively. This is a problem shared by many machine learning endeavors, as changing a hyperparameter value requires the training to be started over from scratch. However active learning is especially vulnerable to this phenomenon as training requires the active participation of a human, and the choice of a sample to be annotated during training is impacted by the hyperparameters settings. As security experts' time is expensive, any hyperparameter mistake is very costly. Many big and small decisions must be taken to prepare such an evaluation protocol, sometimes without the ability to justify these decisions appropriately. When the participating human is expected to be a domain-knowledge expert (as in our case), retraining the model is very expensive and doing it repeatedly is not possible in practice.

Decision Explicability

In our opinion, our lack of access to this campaign's experimental data validates our design choice to consider decision explicability as a critical property of a security decision system. This enables us to design self-serving debugging tools usable by participants, giving them the ability to understand by themselves the reasons why predictions succeeded or failed. The participants were able to give us all the valuable insights we discussed in this section while still keeping the sensitive details of their information system confidential. As discussed in Section 5.2, a natural next step would be to give the expert the tools to improve by themselves the quality of a decision after having diagnosed its root cause. A proposal that is simple for experts to understand and easy for us to implement would be to allow the expert to manually blacklist a low-quality keyword, for instance after having debugged an incorrect alert decision.

Conclusion

In this article we presented the initial design of an automated vulnerability risk analysis prediction system that can immediately choose an appropriate alert level for a just-disclosed vulnerability. It is interactively trained by a security operator using active learning: by mimicking the alert decisions of the human operator, the prediction system can gradually learn which vulnerabilities are considered important in the context of a specific information system. The system is based on an active learning architecture proposed by Settles et al [START_REF] Settles | An analysis of active learning strategies for sequence labeling tasks[END_REF] that includes the computation of a cosine similarity matrix and an information density vector. We modified this architecture to handle dynamic datasets such as the flow of vulnerability disclosures and to add a controllable amount of randomness in the vulnerability selection process. The architecture also uses our own work on vulnerability keyword extraction (described in our previous work [START_REF] Elbaz | Automated keyword extraction from "one-day" vulnerabilities at disclosure[END_REF]) as a featurization scheme.

This contribution is still at an early stage. Our experimental results show some of our initial design choices could be revisited. However, security experts we collaborated with expressed great interest in the concept and their collaboration was a crucial aspect of this contribution. Notably, this partnership highlighted how decision explicability is an important factor for real-world applicability of machine learning for security alerting. Going forward we would like to carry on this relationship onward as well as extend it to include even more participants, opening the door to a more iterative research process and more sound experimental protocols. Validating such a system would be an important step forward in the defense against newly disclosed vulnerabilities, allowing organizations to set up an around-the-clock vulnerability monitoring system while only requiring their security operators to work during regular hours to train the system.

Figure 1 :

 1 Figure 1: Proposed active learning architecture to estimate the risk-level evolution.

Figure 2 :

 2 Figure2: A screen capture of the user interface (UI) of our prototype Firres (FIRst RESponder) during the active learning phase. For a vulnerability to be labeled, the UI displays its CVE ID, its description, the weighted keyword list extracted from the description. The expert can then select an appropriate alert level by entering 1 (for LOG), 2 (for TICKET) or 3 (for PAGE).

1 :

 1 Input: B: annotation budget for the current training phase Input: ONLINE: boolean set to true if the current training phase is online, false if offline Input: r: size of the random subset of vulnerabilities while B > 0 do C = get_all_unannotated_vulnerabilities(V); if ONLINE then C = retain_only_vulnerabilities_disclosed_in_last_period(C); end R = select_random_subset_of_vulnerabilities(C, r); v = select_vulnerability_with_highest_selection_score(R); submit_vulnerability_to_expert(v); B = B -1; end Algorithm Training algorithm, including the vulnerability selection process.

Table 1

 1 Evaluation results for experiment A. Correct decisions: 48. False positives: 39. False negatives: 13.

			System decision
	Expert decision LOG TICKET PAGE
	LOG	46	39	0
	TICKET	4	2	0
	PAGE	5	4	0
			System decision
	Expert decision LOG TICKET PAGE
	LOG	35	36	6
	TICKET	5	12	1
	PAGE	2	3	0

Table 2

 2 Evaluation results for experiment B. Correct decisions: 47. False positives: 45. False negatives: 10.


L Lacour
email: leo.lacour@obs-vlfr.fr.

M Ardyna

K F Stec

H Claustre

L Prieur

A Poteau

M Ribera D'alcala

D Iudicone

Unexpected winter phytoplankton blooms in the North Atlantic subpolar gyre

published or not. The documents may come   L'archive ouverte pluridisciplinaire

and the emergence of winter blooms. Restratification also caused a phytoplankton community shift from pico-and nanophytoplankton to phototrophic diatoms. We conclude that transient winter blooms can maintain active diatom populations throughout the winter months, directly seeding the spring bloom and potentially making a significant contribution to over-winter carbon export.

The Sverdrup paradigm [START_REF] Sverdrup | On conditions for the vernal blooming of phytoplankton[END_REF] postulates that deep winter mixing prevents phytoplankton biomass accumulation since losses exceed light-limited production. However, few wintertime in situ observations revealed significant phytoplankton stock in deep mixed layers [START_REF] Backhaus | Convecion and primary production in winter[END_REF][START_REF] Dale | Seasonal development of phytoplankton at a high latitude oceanic site[END_REF] . Based on modelling simulations, orbital motions triggered by deep convection have been suggested to allow recurrent incursions of phytoplankton to the surface lit layer, and thus supporting a winter production [START_REF] Große | The influence of winter convection on primary production: A parameterisation using a hydrostatic threedimensional biogeochemical model[END_REF] . Furthermore, winter deepening of the mixed layer would dilute plankton concentration, thus reducing grazing pressure, and consequently, phytoplankton loss (disturbance-recovery hypothesis) [START_REF] Behrenfeld | Abandoning Sverdrup's Critical Depth Hypothesis on phytoplankton blooms[END_REF][START_REF] Behrenfeld | Annual cycles of ecological disturbance and recovery underlying the subarctic Atlantic spring plankton bloom[END_REF] . Despite low phytoplankton growth, biomass may accumulate throughout winter in deep mixed layers.

Modelling studies have shown that mixed layer eddies (MLEs) growing from horizontal density gradients can generate vertical restratification of the mixed layer at spatial scales of 1-10 km and time scales of days [START_REF] Boccaletti | Mixed Layer Instabilities and Restratification[END_REF][START_REF] Fox-Kemper | Parameterization of Mixed Layer Eddies. Part I: Theory and Diagnosis[END_REF] , thus allowing patchy blooms to be initiated before the vernal stratification [START_REF] Mahadevan | Eddy-Driven Stratification Initiates North Atlantic Spring Phytoplankton Blooms[END_REF] . MLEs being potentially much more frequent in winter [START_REF] Callies | Seasonality in submesoscale turbulence[END_REF] , such restratification mechanism may provide an additional mechanism for winter phytoplankton growth.

In addition, other studies have proposed that a reduction in turbulent mixing, due to a relaxation in atmospheric forcing, may allow phytoplankton to grow, before the vernal restratification (critical turbulence hypothesis) [START_REF] Huisman | Biodiversity of plankton by species oscillations and chaos[END_REF][START_REF] Taylor | Shutdown of turbulent convection as a new criterion for the onset of spring phytoplankton blooms[END_REF][START_REF] Ferrari | Shutdown of convection triggers increase of surface chlorophyll[END_REF] .

In this context, distinguishing the mixing layer depth from the mixed layer depth is important when dealing with phytoplankton dynamics [START_REF] Brainerd | Surface mixed and mixing layer depths[END_REF][START_REF] Franks | Has Sverdrup's critical depth hypothesis been tested? Mixed layers vs. turbulent layers[END_REF][START_REF] Brody | Changes in dominant mixing length scales as a driver of subpolar phytoplankton bloom initiation in the North Atlantic[END_REF] . In the following, we improved the determination of the mixed layer depth to be the closest to the mixing layer depth (see Supplementary Methods 1.4) given that there are no large scale, autonomous measurements of turbulence so far.

Winter mixing intermittency

The common view is that the North Atlantic subpolar ocean is continuously deeply mixed during winter [START_REF] Behrenfeld | Resurrecting the ecological underpinnings of ocean plankton blooms[END_REF][START_REF] Lindemann | A seasonal diary of phytoplankton in the North Atlantic[END_REF] . However, around 25% of Argo density profiles in this area were stratified (mixed layer depth (MLD) < 100 m, Supplementary Methods 1.4) during the winters of 2014 and 2015 (Fig. 1a and Supplementary Fig. 4). The vertical stratification was particularly strong in areas of significant horizontal density gradients (Fig. 1b), suggesting the potential role of MLEs. MLEs drive net horizontal transfer of lighter water above heavier water that can locally stratify the mixed layer. Similar temperature and salinity structure (i.e. Turner angle [START_REF] Johnson | Global Estimates of Lateral Springtime Restratification[END_REF] ) between horizontal and vertical density gradients in the subpolar gyre (Fig. 1c, Supplementary Methods 1.3) confirm the crucial role of lateral processes in restratifying the deep winter mixed layer in the North Atlantic Subpolar Gyre.

Restratification mechanisms compete with vertical mixing induced mainly by surface wind and cooling [START_REF] Mahadevan | Eddy-Driven Stratification Initiates North Atlantic Spring Phytoplankton Blooms[END_REF] . Thus, restratification events require sufficiently long calm periods with a relaxation of this atmospheric forcing. While analysing wind speed and heat fluxes, we observed that some regions of the subpolar gyre presented up to 30% of calm periods (Fig. 1d, Supplementary Fig. 5 and Methods 1.6). A proxy of the sea state (i.e. wave height, Supplementary Methods 1.7), derived from BGC-Argo floats, also confirmed evidence of calm periods (i.e. 20% of the winter profiles, Supplementary Fig. 7). Therefore, transient calm periods appear to be frequent across the North Atlantic Subpolar Gyre, allowing restratification to be a recurrent feature in winter.

Mixing intermittency triggers winter phytoplankton blooms

Restratification events triggered winter phytoplankton accumulations in the surface layer in the whole subpolar gyre (based on two different proxies of phytoplankton biomass: chlorophyll a (chla) and particulate optical backscattering (b bp ); Fig. 2 and Supplementary Fig. 9). These restratification events maintain phytoplankton cells in the euphotic zone (surface to Ze; see Fig. 2b and Supplementary Fig. 15), enhancing the light availability for their growth.

Indeed, the light level (i.e. the daily-integrated Photosynthetically Available Radiation (PAR) averaged over the MLD) increased ten-fold in average in stratified conditions (MLD < 100 m; 1.3 mol photons m -2 ) compared to deeply mixed conditions (MLD > 100 m; 0.13 mol photons m -2 ). For 70% of the stratified profiles (Fig. 2b, top panel), an increase in light exposure led to the occurrence of winter blooms, defined here as a biomass increase of 5% above the winter median value (75% of the stratified profiles using b bp instead of chla). These observations suggest that most of the calm periods lasted long enough to initiate locally transient winter phytoplankton blooms.

Importantly, distinct light environments between stratified and deeply mixed conditions impacted not only phytoplankton production but also the structure of the community [START_REF] Litchman | Growth rates of phytoplankton under fluctuating light[END_REF][START_REF] Walter | Interactive effects of temperature and light during deep convection: A case study on growth and condition of the diatom Thalassiosira weissflogii[END_REF] . An optical community index (chla to b bp ratio [START_REF] Cetinić | A simple optical index shows spatial and temporal heterogeneity in phytoplankton community composition during the 2008 North Atlantic Bloom Experiment[END_REF] ) derived from BGC-Argo float measurements, revealed a clear shift in the phytoplankton community structure during restratification events. In typical winter conditions (i.e., deep mixed layers with low surface daily-integrated PAR), pico-and nanophytoplankton clearly dominated the overall phytoplankton community in the subpolar gyre (Fig. 3a). By contrast, the proportion of diatoms temporarily increases during transient restratification events when their growth was evidently boosted by more favourable light conditions (Fig. 3a,b). These blooms became particularly frequent in late February and March (10-25% of the all profiles, n = 30, Fig. 3b), when frequency of restratification events increased (20-40% of the all profiles, n = 86; r = 0.94, p-value < 0.01) and surface dailyintegrated PAR was higher due to longer daylength (Fig. 3c). Variation of the cloudiness (dot size in Fig. 3c) also reinforced the critical role of light level, and its impact on changing the community composition in winter. During calm periods with clear sky conditions, higher surface daily-integrated PAR significantly boosted the growth of diatoms (Fig. 3c; r = 0.56, pvalue < 0.01, see supplementary Table .3). These findings have been corroborated by rare phytoplankton pigment samples collected during a winter cruise in the Newfoundland Basin (CATCH, January-February 1997, Fig. 2a and Supplementary Methods 1.10 and 2.5). While deep mixing layers were dominated by chryso-and pelagophytes, contributing to almost 50% of the total chla, diatoms became a relevant group contributing to almost 20% of the total chla when MLD shoaled above 100 m (Supplementary Fig. 16).

Reassessing the winter as an active phytoplankton period

Moderate to high light-adapted diatoms thriving in spring are not likely able to survive in deep winter mixed layers. A set of suitable overwintering strategies, e.g. formation of resting cells or spores [START_REF] Smetacek | Role of sinking in diatom life-history cycles: ecological, evolutionary and geological significance[END_REF][START_REF] Mcquoid | Diatom resting stages[END_REF] , have been highlighted as means to survive harsh winter conditions. Here, we show that diatoms can remain physiologically active throughout winter by benefiting from transient restratification events and thus more favourable light environments (Supplementary Fig. 17 and19). By contrast, the persistence of pico-and nanophytoplankton, especially small flagellates, and their dominance in deep mixing layers may be explained by mixotrophy, the combined used of phototrophy and heterotrophy within a single organism [START_REF] Caron | Mixotrophy stirs up our understanding of marine food webs[END_REF] . The shift to heterotrophy during long period of darkness at depth could confer to these organisms a serious ecological advantage and bring an alternative hypothesis to the disturbance-recovery hypothesis [START_REF] Behrenfeld | Abandoning Sverdrup's Critical Depth Hypothesis on phytoplankton blooms[END_REF] .

An extraordinary number of studies emerged during the last decade to understand the dynamics of the high-latitude spring bloom (i.e. timing and magnitude). However, the vigorous debate on spring bloom dynamics has remained mainly focused on physical controls of phytoplankton biomass accumulation. Our observations reveal that the physical environment plays a key role also in shaping the community structure during the preconditioning period of the spring bloom. Although transient winter blooms, supported by mixing intermittency, are not intense in term of biomass, they maintain an active diatom population throughout winter.

Unlike resting cells or spores that require a germination period [START_REF] Smayda | Dark survival of autotrophic, planktonic marine diatoms[END_REF] , active winter diatoms can directly seed the spring bloom, hence impacting the bloom timing and potentially its magnitude.

Implications for spring bloom and carbon export dynamics

Results of a parallel subpolar phytoplankton community model (Supplementary Methods 2.7) support the claim that intermittent winter stratification can affect spring bloom characteristics. Restratification events in winter boost diatom net growth rate (Supplementary Fig. 17), increasing the standing stock of diatoms towards spring. Mixing intermittency may increase by up to 2 fold the bloom magnitude and bring the bloom peak timing forward by up to 19 days (Supplementary Fig. 19 and Table 7). These results not only corroborate in situ observations (i.e. BGC-Argo and CATCH data) but further suggest a possible impact of winter mixing intermittency on the observed yet unexplained [START_REF] Barton | Physical controls of variability in North Atlantic phytoplankton communities[END_REF] interannual variability of spring blooms.

Unexpected wintertime occurrence of diatom blooms in the subpolar gyre could also challenge our perception of the spring bloom-dependent carbon export. Alternation of stratification and deep mixing allows transient diatom blooms along with non-sinking particles and dissolved organic and inorganic carbon to be transferred rapidly to depth (the so-called mixed layer pump [START_REF] Gardner | The oceanic mixedlayer pump[END_REF][START_REF] Dall'olmo | Carbon export by small particles in the Norwegian Sea[END_REF][START_REF] Koeve | Storm-induced convective export of organic matter during spring in the northeast Atlantic Ocean[END_REF] ). Even though a single event transfers only a low carbon stock compared to the spring bloom, cumulative events over the winter months may contribute significantly to the annual carbon export.

Methods

Methods and supplementary figures are available in the supplementary information.

Fig. 1 .Fig. 2 .

 12 Fig. 1. Winter mixing intermittency in the North Atlantic Subpolar Gyre. (a) 2022 Argo float

Fig. 3 .

 3 Fig. 3. The light environment impacts the phytoplankton community structure. Colour scale

Acknowledgements

We thank N. Briggs, M. J. Perry, E. D'Asaro, B. Gentili, E. Boss and F. Benedetti for fruitful discussions, C. Schmechtig for BGC-Argo float data management and Joséphine Ras for proofreading the manuscript. We also thank Simon Wright for sharing the CHEMTAX software v.1.95, and Clément de Boyer Montégut for providing the MLD climatology. This work represents a contribution to the remOcean project (REMotely sensed biogeochemical cycles in the OCEAN, GA 246777) funded by the European Research Council, the ATLANTOS EU project (grant agreement 2014-633211) funded by H2020 program and the Italian Flagship Program RITMARE.

Data availability

Argo data were collected and made freely available by the International Argo Program and the CORIOLIS project that contribute to it (http://www.coriolis.eu.org). BGC-Argo data are publicly available at ftp://ftp.ifremer.fr/ifremer/argo/dac/coriolis/. Net heat flux and wind data were extracted from the ECMWF ERA Interim dataset (reanalysis) freely available at http://apps.ecmwf.int/datasets/data/interim-full-daily/levtype=sfc. The MLD climatology was provided by Ifremer (http://www.ifremer.fr/cerweb/deboyer/mld/home.php).

Competing financial interests

The authors declare no competing financial interests.