Susanna M Badalyan

Françoise Fons

Morel Sylvie

Luis Portillo-Lemus

Sylvie Rapior

Knowledge on the biomedical and cosmetic usage of macrofungi

Fungi are ecologically diverse organisms distributed worldwide. From estimated 1.5-3 million species, about 150,000 are macrofungi (mushrooms). About 10% (14,000-16,000 species) of macrofungi are identified, of which 7000 are considered edible and 500 poisonous species. There may be as many as 700 species which are considered safe and possess around 130 pharmacological activities. The present work reviews the current state of knowledge and the main findings of previous studies on mycotherapeutics and is an update contribution to modern mycopharmacology and mycocosmetology.

Wild and cultivable edible, and non-edible medicinal macrofungi (e.g. Agaricus brasiliensis, Auricularia auricula-judae, Coprinus comatus, Cordyceps militaris, Ganoderma lucidum, Grifola frondosa, Hericium erinaceus, Flammulina velutipes, Inonotus obliquus, Lentinula edodes, Ophiocordyceps sinensis, Phellinus linteus, Pleurotus ostreatus, Trametes versicolor, Tremella fuciformis) are considered valuable bio-resources to develop functional food (nutraceuticals, nutriceuticals), drugs (mycopharmaceuticals), and cosmetic products (cosmeceuticals, nutricosmetics). The pathways of biosynthesis of bioactive compounds of macrofungi and the genes behind are less understudied compared to their available extensive lists and therapeutic effects.

The advances in fungal biotechnology and cultivation will assist in the production of novel biotech products and their biomedical usage. The clinical trials and in vitro animal studies justify the experience of traditional medicine and suggest a great potential of mycopharmaceuticals for the prevention and treatment of various diseases. However, developments of standardized biotech products for further clinical trials are needed to substantiate the pharmacological properties or side effects of consumption of macrofungi before their clinical application. Thus, advances in fungal biology and biotechnology, genomics and proteomics are required for further biomedical research and usage of macrofungi. Macrofungi have widely been appreciated all over the world for their nutritional value and medicinal properties. They have been used in traditional medicine for more than 3000 years. Despite ethnomycopharmacological information and scientific data, the extensive pharmacological and biotechnological potential offered by macrofungi has not been sufficiently investigated. Modern research has shown that they are producers of a broad spectrum of high-and low-molecular weight bioactive compounds (alkaloids, polysaccharides, phenolics, terpenoids, polyketides, cyclic peptides, lectins, ribosome-inactivating proteins, etc.) possessing analgesic, antidiabetic, anti-inflammatory, antimicrobial, antioxidant, anti-tumorous, antiviral, cardioprotective, hepatoprotective, hypocholesterolemic, antidiabetic/hypoglycemic, hypotensive, immunomodulatory, neuroprotective, mitogenic/regenerative and other pharmacological effects.

Fig. 2 Fig. 1

 21 Fig. 2 Fruiting bodies of edible potential neuroprotective macrofungi: (a) Hericium erinaceus (Photo Courtesy of Angelini C), (b) Grifola frondosa (Photo Courtesy of Angelini C), (c) Lentinula edodes (Photo Courtesy of Angelini C), (d) Tremella fuciformis (Photo Courtesy of Callac P)

TABLE 1 . Macrofungi incorporated in different cosmetic products (Badalyan et al, 2022) Mushroom species Cosmetic Product Cosmetic Effect

 1

	Agaricus subrufescens	Vitamega Facial Moisturizing Mask, Brazil	Renew and revitalize skin
	Albatrellus confluens	REN Clean Skincare Ever calm Ultra Comforting	Skin feels smooth with even tone
		Rescue Mask
	Cordyceps sinensis	Kose Sekkisei Cream, Japan	Moisturizer, suppress
			melanin production
	Ganoderma lucidum	Eminence Organic Skin Care Birch Water Purifying	Anti-aging, provides
		Essence	hydration
		Yves Saint Laurent Temps Majeur Elixir De Nuit	Anti-aging
		Dr. Andrew Weil for Origins™	Anti-aging
		Mega-Mushroom Skin-Relief Micellar Cleanser
		Shiseido: Ultimune Power	Anti-aging, anti-inflammatory
		Infusing Concentrate
		Tela Beauty Organics Encore	Hair protection
		Dri Styling Cream
	Inonotus obliquus	Supermood -The Perfect Day	Antioxidant, skin moisturizer
		Cream
		Root Science RS Reborn Organic Face	Anti-inflammatory, soothe
		Mask, U.S.	irritated skin
		Griffin+row naturally effective skincare	Anti-inflammatory,
			skin brightening
	Lentinula edodes	Aveeno Positively Ageless,	Anti-aging, skin lifting
		Daily Exfoliating Cleanser
		One Love Organics Vitamin D Moisture Mist, UK	Skin lightener, moisturizer and
			toner
	Pleurotus ostreatus	Hankook Sansim Firming Cream	Enhances elasticity, skin tight and
			vitalize
	Polyporus umbellatus	iS Clinical Active Serum	Brightens and revitalizes skin
		Alqvimia Eternal Youth	Brightens, unifies skin tone
		Maximum recovery face cream
	Schizophyllum commune	Alqvimia Eternal Youth Cream Facial Máxima	Anti-aging, skin lifting
		Regeneración, Spain
		Sulwhasoo Hydroaid, cream, Korea	Hydrates, revitalizes skin and
			promotes clear
		Eminence Organic Skin Care	Antioxidant, enhances skin
		Birch Water Purifying Essence	elasticity
	Tremella fuciformis	Dr. Dennis Gross Skincare Alpha Beta® Pore	Enhances skin elasticity, skin
		Perfecting Cleansing Gel	firming
		La Prairie Advanced Marine Biology Night Solution,
		Switzerland

Moisturizes, nourishes, revitalize

Tuber spp.

Black Orchid Tom Ford, Jo Malone Red Truffle 21, Estee Lauder Re-Neutriv' skin cream, Perigold Truffoire truffle magnetic face mask Perfumes with truffles' aroma, moisturizes, nourishes


Ekaterina Khaleva

Anna Rattu

Chris Brightling

Andrew Bush

Arnaud Bourdin

Apostolos Bossios

Kian Fan Chung

Rekha Chaudhuri

Courtney Coleman

Ratko Djukanovic

Sven-Erik Dahlén

Andrew Exley

Louise Fleming

Stephen J Fowler

Atul Gupta

Eckard Hamelmann13

Gerard H Koppelman

Erik Melén

Vera Mahler

Paul Seddon

Florian Singer

Celeste Porsbjerg

Valeria Ramiconi22

Franca Rusconi

Valentyna Yasinska

Graham Roberts
email: g.c.roberts@soton.ac.uk

Definitions of non-response and response to biological therapy for severe asthma: a systematic review

Background

Biologics have proven efficacy for patients with severe asthma but there is lack of consensus on defining response. We systematically reviewed and appraised methodologically developed, defined and evaluated definitions of non-response and response to biologics for severe asthma.

Methods

We searched four bibliographic databases from inception to 15 March 2021. Two reviewers screened references, extracted data, and assessed methodological quality of development, measurement properties of outcome measures and definitions of response based on COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN). A modified GRADE (Grading of Recommendations Assessment, Development and Evaluation) approach and narrative synthesis were undertaken.

Results

13 studies reported three composite outcome measures, three asthma symptoms measures, one asthma control measure and one quality of life measure. Only four measures were developed with patient input; none were composite measures. Studies utilised 17 definitions of response: 10 out of 17 (58.8%) were based on minimal clinically important difference (MCID) or minimal important difference (MID) and 16 out of 17 (94.1%) had high-quality evidence. Results were limited by poor methodology for the development process and incomplete reporting of psychometric properties. Most measures rated "very low" to "low" for quality of measurement properties and none met all quality standards.

Conclusions

This is the first review to synthesise evidence about definitions of response to biologics for severe asthma. While high-quality definitions are available, most are MCIDs or MIDs, which may be insufficient to justify continuation of biologics in terms of cost-effectiveness. There remains an unmet need for universally accepted, patient-centred, composite definitions to aid clinical decision making and comparability of responses to biologics.

Introduction

According to the European Respiratory Society/American Thoracic Society (ERS/ATS) guidelines, severe asthma is defined as asthma requiring treatment based on Global Initiative for Asthma (GINA) steps 4-5 for the previous year or oral corticosteroids for ⩾50% of the previous year either to prevent the disease becoming uncontrolled or disease which remains uncontrolled despite this therapy [START_REF] Chung | International ERS/ATS guidelines on definition, evaluation and treatment of severe asthma[END_REF]. Even though severe asthma only affects 5-10% of the total population with asthma [START_REF] Chung | International ERS/ATS guidelines on definition, evaluation and treatment of severe asthma[END_REF], it represents a significant socioeconomic [START_REF] Chen | Economic burden of multimorbidity in patients with severe asthma: a 20-year population-based study[END_REF][START_REF] Pamuk | The economic burden of severe asthma in children: a comprehensive study[END_REF][START_REF] Kerkhof | Healthcare resource use and costs of severe, uncontrolled eosinophilic asthma in the UK general population[END_REF][START_REF] Janson | Health care resource utilization and cost for asthma patients regularly treated with oral corticosteroidsa Swedish observational cohort study (PACEHR)[END_REF][START_REF] O'neill | The cost of treating severe refractory asthma in the UK: an economic analysis from the British Thoracic Society Difficult Asthma Registry[END_REF], psychological [START_REF] Foster | I have lost in every facet of my life": the hidden burden of severe asthma[END_REF][START_REF] Stubbs | Living well with severe asthma[END_REF] and treatment [START_REF] Efraij | A systematic review of the adverse events and economic impact associated with oral corticosteroids in asthma[END_REF] burden, and is also be associated with risk of mortality [START_REF] Bourdin | Mortality: a neglected outcome in OCS-treated severe asthma[END_REF][START_REF] Ekstrom | Oral corticosteroid use, morbidity and mortality in asthma: a nationwide prospective cohort study in Sweden[END_REF].

Over the past decades, new biological drugs have demonstrated a positive impact on the lives of many patients with severe asthma by reducing the frequency of exacerbations and dose of oral corticosteroids, and by improving lung function [START_REF] Agache | Efficacy and safety of treatment with biologicals (benralizumab, dupilumab, mepolizumab, omalizumab and reslizumab) for severe eosinophilic asthma. A systematic review for the EAACI Guidelinesrecommendations on the use of biologicals in severe asthma[END_REF][START_REF] Agache | Efficacy and safety of treatment with biologicals (benralizumab, dupilumab and omalizumab) for severe allergic asthma: a systematic review for the EAACI Guidelines -recommendations on the use of biologicals in severe asthma[END_REF][START_REF] Agache | Efficacy and safety of treatment with dupilumab for severe asthma: a systematic review of the EAACI guidelinesrecommendations on the use of biologicals in severe asthma[END_REF][START_REF] Brusselle | Biologic therapies for severe asthma[END_REF]. Recently, in addition to total IgE, blood eosinophil counts and fractional exhaled nitric oxide (FENO) have been suggested as a guide to initiate anti-IgE treatment in adolescents and adults [START_REF] Holguin | Management of severe asthma: a European Respiratory Society/ American Thoracic Society guideline[END_REF]. Furthermore, blood eosinophil counts have been used to select patients for anti-interleukin (IL)-5 in adults [START_REF] Holguin | Management of severe asthma: a European Respiratory Society/ American Thoracic Society guideline[END_REF], and FENO/blood eosinophil counts for dupilumab in adolescents and adults [START_REF]Dupilumab for treating severe asthma with type 2 inflammation[END_REF]. Several studies have described the characteristics of patients who started biologics [START_REF] Chipps | Baseline asthma burden, comorbidities, and biomarkers in omalizumab-treated patients in PROSPERO[END_REF][START_REF] Albers | Biologic treatment eligibility for real-world patients with severe asthma: the IDEAL study[END_REF] and the characteristics of responders to treatment [START_REF] Casale | Omalizumab effectiveness by biomarker status in patients with asthma: evidence from PROSPERO. A prospective real-world study[END_REF][START_REF] Corren | Tezepelumab improves patient-reported outcomes in patients with severe, uncontrolled asthma in PATHWAY[END_REF][START_REF] Probst | In search of clinically relevant parameters to monitor successful omalizumab therapy in allergic asthma[END_REF][START_REF] Drick | Anti-IL-5 therapy in patients with severe eosinophilic asthmaclinical efficacy and possible criteria for treatment response[END_REF]. It has been shown that some patients reached a "super-response" [START_REF] Eger | Long-term therapy response to anti-IL-5 biologics in severe asthmaa real-life evaluation[END_REF] or "partial response" [START_REF] Abdo | Small airway dysfunction as predictor and marker for clinical response to biological therapy in severe eosinophilic asthma: a longitudinal observational study[END_REF], whereas others experienced a "non-response" [START_REF] Eger | Long-term therapy response to anti-IL-5 biologics in severe asthmaa real-life evaluation[END_REF] or even deterioration [START_REF] Mukherjee | Suboptimal treatment response to anti-IL-5 monoclonal antibodies in severe eosinophilic asthmatics with airway autoimmune phenomena[END_REF] of clinical and patient-reported outcome measures (PROMs).

Although many studies have measured responses to different biologics, there are no universally accepted criteria for what constitutes response, and the absence of guidance on criteria is reported as a high-priority research gap in both children and adults [START_REF] Agache | EAACI biologicals guidelinesrecommendations for severe asthma[END_REF][START_REF] Golebski | Expert meeting report: towards a joint European roadmap to address the unmet needs and priorities of paediatric asthma patients on biologic therapy[END_REF]. Evidence about responder definitions is critical for understanding the effectiveness of treatment for patients [START_REF] Coleman | Narrative review to capture patients' perceptions and opinions about non-response and response to biological therapy for severe asthma[END_REF], clinicians and regulatory bodies, such as the European Medicines Agency [START_REF]Reflection paper on the regulatory guidance for the use of health-related quality of life (HRQL) measures in the evaluation of medicinal productsscientific guideline[END_REF] and the Food and Drug Administration [START_REF]Guidance for industry: patient-reported outcome measures: use in medicinal product development to support labelling claims[END_REF]. Minimal clinically important difference (MCID) [START_REF] Jaeschke | Measurement of health status. Ascertaining the minimal clinically important difference[END_REF] and minimal important difference (MID) [START_REF] Revicki | Recommended methods for determining responsiveness and minimally important differences for patient-reported outcomes[END_REF] are often used for assessing responses; these are defined as the smallest relevant within-person change or group differences between treatments, respectively. According to the Food and Drug Administration report, it is useful to report intra-subject responses based on an a priori responder definition [START_REF]Guidance for industry: patient-reported outcome measures: use in medicinal product development to support labelling claims[END_REF]. In November 2016, an ERS Task Force reached a consensus on a traffic-light system to classify patients as non-responders, intermediate responders or super-responders [START_REF] Buhl | Severe eosinophilic asthma: a roadmap to consensus[END_REF]. The Task Force suggested that patients need to be on biological treatment for at least 4 months before an initial assessment of response can be determined [START_REF] Buhl | Severe eosinophilic asthma: a roadmap to consensus[END_REF]. However, this proposal has neither been validated nor further developed.

Given the unmet need to use consistent definitions of response for paediatric and adult patients, we aimed to 1) synthesise evidence about definitions of non-response and response to biological therapy used in patients with severe asthma, 2) assess the quality of the evidence for these definitions, and 3) evaluate the development, measurement properties and quality of outcome measures as supporting evidence for the included definitions. We chose to restrict our systematic review to studies where definitions were methodologically developed, defined and evaluated. Comprehensive assessment of response in clinical practice and trials using prespecified consensus criteria should provide useful guidance for clinical decision making, allow comparison across studies, eliminate unnecessary treatment in patients with inadequate response and ensure that the high cost associated with biological therapies [START_REF] Mcqueen | Cost-effectiveness of biological asthma treatments: a systematic review and recommendations for future economic evaluations[END_REF] is justified [START_REF] Anderson Wc 3rd | Cost-effectiveness and comparative effectiveness of biologic therapy for asthma: to biologic or not to biologic?[END_REF].

Methods

This was a systematic review conducted by the 3TR (Taxonomy, Treatment, Targets and Remission) [START_REF] Tr | Treatment, Targets and Remission[END_REF] Respiratory Work Package members and external collaborators including academic clinicians, regulatory, patient and pharmaceutical representatives from across Europe. The study is registered at PROSPERO with identifier number CRD42021211249. Our aim was to look at response in severe asthma, but in anticipation that the evidence base would be limited, we initially included studies of all severities of asthma. However, given that there is evidence for definitions of response to biological therapy for severe asthma, the protocol was revised to restrict the systematic review to studies of severe asthma. The PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) checklist has been used to structure this article (supplementary appendix 1) [START_REF] Page | The PRISMA 2020 statement: an updated guideline for reporting systematic reviews[END_REF]. The methods are briefly described here. Details are available in the supplementary material.

Search strategy

Four databases were searched (Embase (OVID), MEDLINE (OVID), CINAHL (EBSCOhost, Cumulative Index to Nursing and Allied Health Literature) and ISI Web of Science (Thomson Web of Knowledge)) using a search strategy developed on Embase (OVID) and then adapted for other databases (supplementary appendix 2). In summary, the search strategy was designed to identify papers focused on "asthma" AND "a biological therapy" AND "response/treatment outcome/minimal important difference". Databases were searched from inception to 15 March 2021. Additional references were searched through the references cited by the identified studies, systematic reviews, reviews, guidelines or highlighted by experts in the field.

Inclusion criteria

Studies were eligible for inclusion if they met the following criteria. 1) Population: children/adolescents (6 -17 years) and/or adults (⩾18 years) with a diagnosis of severe asthma. 2) Intervention: any biological therapy which was investigated and/or currently used for severe asthma. 3) Comparator: any comparator, including placebo or no comparator. 4) Outcomes: any definitions of non-response and response to biological therapy for severe asthma which were methodologically developed, defined and evaluated. Sole or a composite of clinical, patient-reported, biological and/or imaging outcome measures were eligible for inclusion. Additional evidence about these outcome measures including development (undertaken in studies of any severity of asthma) and validation (conducted in studies with biologics for severe asthma) was included. 5) Study types: randomised controlled trials, crosssectional studies, controlled before-and-after studies, non-randomised controlled studies, case-control studies in humans, cohort studies and consecutive case series (with a minimum of 10 participants) published as full-text articles and letters published in English were eligible for inclusion. Additional evidence about development and validation of outcome measures was considered from qualitative and validation studies.

Exclusion criteria

The following were excluded from the analysis: systematic reviews and meta-analyses, narrative reviews, discussion papers, editorials, commentaries, case reports, animal studies, conference abstracts, studies not available in full form, studies published in a language other than English, unpublished material and non-asthma studies (e.g. viral bronchiolitis or viral-associated wheeze). Studies were also excluded if they only used outcome measures and definitions of response to assess treatment effectiveness or efficacy.

Study selection

All references were pooled and de-duplicated in Endnote version X9 (Thomson Reuters, Philadelphia, PA, USA) and subsequently uploaded to Rayyan (https://rayyan.qcri.org), where any remaining duplicates were removed. Titles, abstracts and full texts were screened independently by two reviewers (E.K. and A.R.) according to the predefined selection criteria and categorised as included, excluded or unsure. Any disagreements were resolved through discussion with a third reviewer (G.R.).

Data extraction, risk of bias assessment, quality and synthesis of the results

Data extraction was based on the COnsensus-based Standards for the selection of Measurement INstruments (COSMIN) guideline for outcome measures [START_REF] Prinsen | COSMIN guideline for systematic reviews of patient-reported outcome measures[END_REF]. Definitions of the measurement properties provided by COSMIN are provided in supplementary table S1 and criteria for good measurement properties (GMPs) are provided in supplementary table S2.

Risk of bias of individual studies was assessed using the COSMIN checklist for PROMs [START_REF] Mokkink | COSMIN Risk of Bias checklist for systematic reviews of Patient-Reported Outcome Measures[END_REF][START_REF] Terwee | COSMIN methodology for evaluating the content validity of patient-reported outcome measures: a Delphi study[END_REF] and composite outcome measures (COSMIN risk of bias for non-patient-reported outcomes) [START_REF] Mokkink | COSMIN Risk of Bias tool to assess the quality of studies on reliability or measurement error of outcome measurement instruments: a Delphi study[END_REF]. Risk of bias for each measurement property in the validation studies was rated as very good, adequate, doubtful or inadequate. The certainty of evidence was assessed using the modified GRADE (Grading of Recommendations, Assessment, Development and Evaluation) approach [START_REF] Prinsen | COSMIN guideline for systematic reviews of patient-reported outcome measures[END_REF][START_REF] Terwee | COSMIN methodology for evaluating the content validity of patient-reported outcome measures: a Delphi study[END_REF][START_REF] Guyatt | GRADE guidelines: 1. Introduction -GRADE evidence profiles and summary of findings tables[END_REF]. Data extraction, risk of bias assessment and modified GRADE were completed independently by two reviewers (E.K. and A.R.). Any discrepancies were resolved by discussion or by a third reviewer (G.R.). A descriptive synopsis with summary data tables was produced and results were summarised using narrative synthesis. Detailed methods are provided in supplementary appendix 3. The results were reviewed and discussed within the Core Outcome Measures for Severe Asthma (COMSA) initiative [START_REF] Khaleva | Development of Core Outcome Measures sets for paediatric and adult Severe Asthma (COMSA)[END_REF] that included a multidisciplinary, European group of academic clinicians, regulatory, patient and pharmaceutical representatives. The group aimed to select the core outcome measure sets for paediatric and adult severe asthma.

Results

Description of studies

Our search strategy identified a total of 11 588 papers; 11 553 articles were excluded after title and abstract screening. The full text of 35 papers was assessed for eligibility, including 20 articles identified through review of citations. 13 papers were included in the systematic review, of which three were about development of the outcome measures [START_REF] Globe | Symptoms and markers of symptom severity in asthmacontent validity of the Asthma Symptom Diary[END_REF][START_REF] Hyland | How patient participation was used to develop a questionnaire that is fit for purpose for assessing quality of life in severe asthma[END_REF][START_REF] Revicki | Integrating patient preferences into health outcomes assessment: the multiattribute Asthma Symptom Utility Index[END_REF], five were validation papers [START_REF] Lanario | Validation of subscales of the Severe Asthma Questionnaire (SAQ) using exploratory factor analysis (EFA)[END_REF][START_REF] Masoli | The Severe Asthma Questionnaire: sensitivity to change and minimal clinically important difference (MCID)[END_REF][START_REF] Shen | Psychometric properties of the Asthma Symptom Index in patients with severe asthma[END_REF][START_REF] Globe | Evaluating minimal important differences and responder definitions for the asthma symptom diary in patients with moderate to severe asthma[END_REF][START_REF] Krouse | Minimally important differences and risk levels for the Composite Asthma Severity Index[END_REF], and five reported development and validation data in the same paper [START_REF] Fitzpatrick | Development and initial validation of the Asthma Severity Scoring System (ASSESS)[END_REF][START_REF] Perez De Llano | Development of a tool to measure the clinical response to biologic therapy in uncontrolled severe asthma: the FEV1, exacerbations, oral corticosteroids, symptoms score[END_REF][START_REF] Hyland | The construction and validation of the Severe Asthma Questionnaire[END_REF][START_REF] Lloyd | Psychometric evaluation of Global Evaluation of Treatment Effectiveness: a tool to assess patients with moderate-to-severe allergic asthma[END_REF][START_REF] Wildfire | Development and validation of the Composite Asthma Severity Indexan outcome measure for use in children and adolescents[END_REF] (figure 1).

Development and quality of definitions of non-response and response

The approach to development of definitions and their characteristics are shown in tables 1 and 2. Definitions were developed for three composite asthma outcome measures [START_REF] Krouse | Minimally important differences and risk levels for the Composite Asthma Severity Index[END_REF][START_REF] Fitzpatrick | Development and initial validation of the Asthma Severity Scoring System (ASSESS)[END_REF][START_REF] Perez De Llano | Development of a tool to measure the clinical response to biologic therapy in uncontrolled severe asthma: the FEV1, exacerbations, oral corticosteroids, symptoms score[END_REF], three asthma symptom outcome measures [START_REF] Shen | Psychometric properties of the Asthma Symptom Index in patients with severe asthma[END_REF][START_REF] Globe | Evaluating minimal important differences and responder definitions for the asthma symptom diary in patients with moderate to severe asthma[END_REF], one asthma control outcome measure [START_REF] Lloyd | Psychometric evaluation of Global Evaluation of Treatment Effectiveness: a tool to assess patients with moderate-to-severe allergic asthma[END_REF] and one quality of life (QoL) measure [START_REF] Masoli | The Severe Asthma Questionnaire: sensitivity to change and minimal clinically important difference (MCID)[END_REF]. The following methods of development were used: consensus [START_REF] Perez De Llano | Development of a tool to measure the clinical response to biologic therapy in uncontrolled severe asthma: the FEV1, exacerbations, oral corticosteroids, symptoms score[END_REF][START_REF] Lloyd | Psychometric evaluation of Global Evaluation of Treatment Effectiveness: a tool to assess patients with moderate-to-severe allergic asthma[END_REF], anchor-based [START_REF] Masoli | The Severe Asthma Questionnaire: sensitivity to change and minimal clinically important difference (MCID)[END_REF][START_REF] Shen | Psychometric properties of the Asthma Symptom Index in patients with severe asthma[END_REF][START_REF] Globe | Evaluating minimal important differences and responder definitions for the asthma symptom diary in patients with moderate to severe asthma[END_REF][START_REF] Krouse | Minimally important differences and risk levels for the Composite Asthma Severity Index[END_REF] and distribution-based [START_REF] Fitzpatrick | Development and initial validation of the Asthma Severity Scoring System (ASSESS)[END_REF] methods. 10 definitions measured response based on MCID [START_REF] Masoli | The Severe Asthma Questionnaire: sensitivity to change and minimal clinically important difference (MCID)[END_REF][START_REF] Shen | Psychometric properties of the Asthma Symptom Index in patients with severe asthma[END_REF][START_REF] Krouse | Minimally important differences and risk levels for the Composite Asthma Severity Index[END_REF] or MID [START_REF] Globe | Evaluating minimal important differences and responder definitions for the asthma symptom diary in patients with moderate to severe asthma[END_REF][START_REF] Fitzpatrick | Development and initial validation of the Asthma Severity Scoring System (ASSESS)[END_REF] and seven [START_REF] Globe | Evaluating minimal important differences and responder definitions for the asthma symptom diary in patients with moderate to severe asthma[END_REF][START_REF] Lloyd | Psychometric evaluation of Global Evaluation of Treatment Effectiveness: a tool to assess patients with moderate-to-severe allergic asthma[END_REF] based on responder/non-responder levels. Omalizumab [START_REF] Masoli | The Severe Asthma Questionnaire: sensitivity to change and minimal clinically important difference (MCID)[END_REF][START_REF] Krouse | Minimally important differences and risk levels for the Composite Asthma Severity Index[END_REF][START_REF] Fitzpatrick | Development and initial validation of the Asthma Severity Scoring System (ASSESS)[END_REF][START_REF] Lloyd | Psychometric evaluation of Global Evaluation of Treatment Effectiveness: a tool to assess patients with moderate-to-severe allergic asthma[END_REF], brodalumab [START_REF] Globe | Evaluating minimal important differences and responder definitions for the asthma symptom diary in patients with moderate to severe asthma[END_REF], benralizumab [START_REF] Masoli | The Severe Asthma Questionnaire: sensitivity to change and minimal clinically important difference (MCID)[END_REF][START_REF] Perez De Llano | Development of a tool to measure the clinical response to biologic therapy in uncontrolled severe asthma: the FEV1, exacerbations, oral corticosteroids, symptoms score[END_REF], reslizumab [START_REF] Masoli | The Severe Asthma Questionnaire: sensitivity to change and minimal clinically important difference (MCID)[END_REF][START_REF] Perez De Llano | Development of a tool to measure the clinical response to biologic therapy in uncontrolled severe asthma: the FEV1, exacerbations, oral corticosteroids, symptoms score[END_REF] and mepolizumab [START_REF] Masoli | The Severe Asthma Questionnaire: sensitivity to change and minimal clinically important difference (MCID)[END_REF][START_REF] Shen | Psychometric properties of the Asthma Symptom Index in patients with severe asthma[END_REF][START_REF] Perez De Llano | Development of a tool to measure the clinical response to biologic therapy in uncontrolled severe asthma: the FEV1, exacerbations, oral corticosteroids, symptoms score[END_REF] were predominantly used in these studies. Response was evaluated at different time-points, including as early as 4 weeks [START_REF] Masoli | The Severe Asthma Questionnaire: sensitivity to change and minimal clinically important difference (MCID)[END_REF] and up to 60 weeks [START_REF] Krouse | Minimally important differences and risk levels for the Composite Asthma Severity Index[END_REF]. Most definitions were developed for adults [49-51, 53, 54], while three were for adolescents [START_REF] Shen | Psychometric properties of the Asthma Symptom Index in patients with severe asthma[END_REF][START_REF] Krouse | Minimally important differences and risk levels for the Composite Asthma Severity Index[END_REF][START_REF] Fitzpatrick | Development and initial validation of the Asthma Severity Scoring System (ASSESS)[END_REF] and one was for children [START_REF] Krouse | Minimally important differences and risk levels for the Composite Asthma Severity Index[END_REF] with severe asthma. Quality of evidence for definitions of response was rated as "high" for all except "moderate" for the Asthma Severity Scoring System (ASSESS) [START_REF] Fitzpatrick | Development and initial validation of the Asthma Severity Scoring System (ASSESS)[END_REF] due to a lower number of patients taking biologics.

Development and content validity of the outcome measures

An overview of the developmental process and its quality are shown in table 2 and supplementary table S3. The developmental process was predominantly rated as "sufficient", while quality of evidence was mainly "very low" to "low", but "moderate" for the Severe Asthma Questionnaire (SAQ) [START_REF] Hyland | How patient participation was used to develop a questionnaire that is fit for purpose for assessing quality of life in severe asthma[END_REF][START_REF] Hyland | The construction and validation of the Severe Asthma Questionnaire[END_REF]. Three composite outcome measures were developed by physicians, including FEOS (forced expiratory volume in 1 s (FEV1), exacerbations, oral corticosteroids, symptoms score) [START_REF] Perez De Llano | Development of a tool to measure the clinical response to biologic therapy in uncontrolled severe asthma: the FEV1, exacerbations, oral corticosteroids, symptoms score[END_REF] for adults and ASSESS [START_REF] Fitzpatrick | Development and initial validation of the Asthma Severity Scoring System (ASSESS)[END_REF] which was adapted from the Composite Asthma Symptom Index (CASI) [START_REF] Wildfire | Development and validation of the Composite Asthma Severity Indexan outcome measure for use in children and adolescents[END_REF] for adolescents/adults and children with asthma, respectively. The Global Evaluation of Treatment Effectiveness (GETE) [START_REF] Lloyd | Psychometric evaluation of Global Evaluation of Treatment Effectiveness: a tool to assess patients with moderate-to-severe allergic asthma[END_REF] scale was also developed by physicians. Only four outcomes were developed with patient input, including the SAQ [START_REF] Hyland | How patient participation was used to develop a questionnaire that is fit for purpose for assessing quality of life in severe asthma[END_REF][START_REF] Hyland | The construction and validation of the Severe Asthma Questionnaire[END_REF], Asthma Symptom Diary (ASD) [START_REF] Globe | Symptoms and markers of symptom severity in asthmacontent validity of the Asthma Symptom Diary[END_REF], Asthma Symptom Utility Index (ASUI) [START_REF] Shen | Psychometric properties of the Asthma Symptom Index in patients with severe asthma[END_REF] and Asthma Symptom Index (ASI) [START_REF] Revicki | Integrating patient preferences into health outcomes assessment: the multiattribute Asthma Symptom Utility Index[END_REF], which was adapted from the ASUI by excluding questions about medication side-effects. A summary of key instrument characteristics and feasibility is provided table 3 and supplementary table S4.

Risk of bias and quality of evidence for validation studies of outcome measures

Validation data including risk of bias are shown in supplementary tables S5-S7 and methodological quality of the outcome measures rated against criteria for GMPs is presented in table 4. Overall, almost all outcome measures had "inadequate" risk of bias due to lack of involvement of patients in the development, many measurement properties not being reported and none of the studies reporting cross-cultural validity including measurement invariance.

The GETE [START_REF] Lloyd | Psychometric evaluation of Global Evaluation of Treatment Effectiveness: a tool to assess patients with moderate-to-severe allergic asthma[END_REF] scale has patient and physician versions which demonstrated high quality of evidence for construct validity, although there was a positive skew towards "complete control of asthma" and "marked improvement of asthma" possibly due to the ceiling effect. The CASI [START_REF] Wildfire | Development and validation of the Composite Asthma Severity Indexan outcome measure for use in children and adolescents[END_REF] showed insufficient responsiveness but "high" quality of evidence. Sufficient measurement properties were rated for ASSESS, including test-retest reliability, construct validity and responsiveness to change, while the quality was mostly "very low". The ASUI [START_REF] Shen | Psychometric properties of the Asthma Symptom Index in patients with severe asthma[END_REF] and ASI [START_REF] Shen | Psychometric properties of the Asthma Symptom Index in patients with severe asthma[END_REF] performed similarly and showed sufficient rating against GMP criteria and "low" to "high" quality. The SAQ [START_REF] Lanario | Validation of subscales of the Severe Asthma Questionnaire (SAQ) using exploratory factor analysis (EFA)[END_REF][START_REF] Masoli | The Severe Asthma Questionnaire: sensitivity to change and minimal clinically important difference (MCID)[END_REF][START_REF] Hyland | The construction and validation of the Severe Asthma Questionnaire[END_REF] again showed sufficient properties and "very low" to "moderate" quality of evidence. Only responsiveness to change was evaluated for the ASD [START_REF] Globe | Evaluating minimal important differences and responder definitions for the asthma symptom diary in patients with moderate to severe asthma[END_REF] as assessment of other measurement properties was not performed in patients taking biologics for severe asthma. The FEOS [START_REF] Perez De Llano | Development of a tool to measure the clinical response to biologic therapy in uncontrolled severe asthma: the FEV1, exacerbations, oral corticosteroids, symptoms score[END_REF] scale only contains data about inter-rater agreement which was not possible to assess based on the COSMIN methodology.

Discussion

This study aimed to review the literature on definitions of response and non-response to biological therapy for severe asthma. To the best of our knowledge, the current systematic review is the first to synthesise methodologically developed, defined and evidenced definitions. We identified eight outcome measures: three composite outcome measures, three measuring asthma symptoms, one measuring asthma control and one measuring QoL. Studies utilised a variety of definitions of response criteria, mostly using MCIDs or MIDs where available and measured at different time-points for different biologics. Only GETE [START_REF] Lloyd | Psychometric evaluation of Global Evaluation of Treatment Effectiveness: a tool to assess patients with moderate-to-severe allergic asthma[END_REF] defined a non-response, while FEOS [START_REF] Perez De Llano | Development of a tool to measure the clinical response to biologic therapy in uncontrolled severe asthma: the FEV1, exacerbations, oral corticosteroids, symptoms score[END_REF] is a scale ranging from 0 to 100 (best), with no established cut-off for non-responders.

One of the aims of the review was to assess the development and measurement properties of the identified outcome measures. Results were limited by "very low" to "low" quality of evidence for the development process, except for the SAQ [START_REF] Hyland | How patient participation was used to develop a questionnaire that is fit for purpose for assessing quality of life in severe asthma[END_REF][START_REF] Hyland | The construction and validation of the Severe Asthma Questionnaire[END_REF], and incomplete reporting of measurement properties for all outcome measures. Based on the COSMIN guideline, none of the outcome measures met all the quality standards. Only four outcome measures were developed with patient input, even though this is considered as a vital step in ensuring that the instrument is meaningful for patients. Responsiveness to change was rated as "low" to "very low", while definitions of response had "high" quality except for ASSESS [START_REF] Fitzpatrick | Development and initial validation of the Asthma Severity Scoring System (ASSESS)[END_REF].

Evaluation of therapeutic response in asthma has received increased attention with the introduction of biological treatments to improve disease treatment and precision management [START_REF] Bona | Effectiveness of benralizumab in severe eosinophilic asthma: distinct sub-phenotypes of response identified by cluster analysis[END_REF]. More than 70% of patients achieved good or excellent response to omalizumab based on GETE [START_REF] Alhossan | Real-life" effectiveness studies of omalizumab in adult patients with severe allergic asthma: meta-analysis[END_REF]; however, this relies on a single global measure to reflect the heterogeneous response to biological treatment. Thus, GETE does not discriminate the different effects of a treatment on different response areas, such as QoL, exacerbations, maintenance corticosteroid use and lung function. Two asthma symptoms questionnaires (ASUI and ASI [START_REF] Shen | Psychometric properties of the Asthma Symptom Index in patients with severe asthma[END_REF]) were designed to assess cost-effectiveness of treatment, while the ASD [START_REF] Globe | Evaluating minimal important differences and responder definitions for the asthma symptom diary in patients with moderate to severe asthma[END_REF] is a symptom diary and might impose too much burden on participants of biological therapy trials. The SAQ [START_REF] Hyland | The construction and validation of the Severe Asthma Questionnaire[END_REF], which was developed with patient input, showed the best quality of evidence and was selected in the COMSA [START_REF] Khaleva | Development of Core Outcome Measures sets for paediatric and adult Severe Asthma (COMSA)[END_REF][START_REF] Rattu | Identifying and appraising outcome measures for severe asthma: a systematic review[END_REF].

Several composite outcome measures were identified. Neither the CASI [START_REF] Wildfire | Development and validation of the Composite Asthma Severity Indexan outcome measure for use in children and adolescents[END_REF] nor ASSESS [START_REF] Fitzpatrick | Development and initial validation of the Asthma Severity Scoring System (ASSESS)[END_REF] include a QoL domain, and the CASI [START_REF] Wildfire | Development and validation of the Composite Asthma Severity Indexan outcome measure for use in children and adolescents[END_REF] does not assess maintenance oral corticosteroid use, even though reduction in oral corticosteroid use and improvement of QoL have been shown to be the best indicators of response to treatment for patients with severe asthma [START_REF] Clark | What matters to people with severe asthma? Exploring add-on asthma medication and outcomes of importance[END_REF]. The 2-point MID for ASSESS showed good specificity but poor sensitivity and the authors suggested that it should be interpreted with caution until more data are available [START_REF] Fitzpatrick | Development and initial validation of the Asthma Severity Scoring System (ASSESS)[END_REF]. The FEOS tool to quantify response [START_REF] Perez De Llano | Development of a tool to measure the clinical response to biologic therapy in uncontrolled severe asthma: the FEV1, exacerbations, oral corticosteroids, symptoms score[END_REF] was developed for adults with severe asthma using novel methodology, but patients were not involved in the selection of outcome measures and it may not also represent the perspectives of international stakeholders. Unlike the COMSA initiative [START_REF] Khaleva | Development of Core Outcome Measures sets for paediatric and adult Severe Asthma (COMSA)[END_REF], the validity of the included outcome measures for severe asthma was not assessed and exclusion of aspects such as QoL may not represent a patient-centred approach.

This systematic review did not identify any studies which validated definitions of response to biological therapy using clinical outcome measures in patients with severe asthma. Some data are available from the consensus statements, e.g. the MID for FEV1 is 0.20 L [START_REF] Agache | Efficacy and safety of treatment with biologicals (benralizumab, dupilumab and omalizumab) for severe allergic asthma: a systematic review for the EAACI Guidelines -recommendations on the use of biologicals in severe asthma[END_REF] or 10% improvement [START_REF] Stanojevic | ERS/ATS technical standard on interpretive strategies for routine lung function tests[END_REF] and for FENO a reduction of ⩾20% for values over 50 ppb (or ⩾10 ppb for values lower than 50 ppb) should be used to indicate response to anti-inflammatory therapy [START_REF] Dweik | An official ATS clinical practice guideline: interpretation of exhaled nitric oxide levels (FENO) for clinical applications[END_REF]. While a published composite definition of exacerbation has been developed and validated in patients with severe asthma taking benralizumab, no MCID data are available yet [START_REF] Fuhlbrigge | A novel endpoint for exacerbations in asthma to accelerate clinical development: a post-hoc analysis of randomised controlled trials[END_REF].

Most outcome measures identified in the systematic review utilised MCIDs or MIDs to assess response, but we do not regard these definitions as interchangeable, e.g. in one paper the term MID was used when it would seem to be more appropriate to use MCID [START_REF] Krouse | Minimally important differences and risk levels for the Composite Asthma Severity Index[END_REF]. An improvement that patients might recognise as equivalent to the MCID with an inhaled asthma therapy may potentially be rated as less than the MCID in the context of high-cost biologics administered by injection [START_REF] Mcqueen | Cost-effectiveness of biological asthma treatments: a systematic review and recommendations for future economic evaluations[END_REF][START_REF] Anderson Wc 3rd | Cost-effectiveness and comparative effectiveness of biologic therapy for asthma: to biologic or not to biologic?[END_REF]. Also, to be regarded as cost-effective a biological therapy will demand a greater magnitude of response than a less expensive asthma therapy. A further critical variable may be the duration of response, given the case reports of secondary loss of response [START_REF] Cormier | Secondary loss of response to mepolizumab in severe eosinophilic asthma[END_REF], i.e. the loss of response during the treatment over time despite an initial primary response [START_REF] Roda | Loss of response to anti-TNFs: definition, epidemiology, and management[END_REF][START_REF] Clark | The patients' experience of severe asthma add-on pharmacotherapies: a qualitative descriptive study[END_REF].

The concept of "super-responders" to biological treatment has emerged recently [START_REF] Eger | Long-term therapy response to anti-IL-5 biologics in severe asthmaa real-life evaluation[END_REF][START_REF] Kavanagh | Real-world effectiveness and the characteristics of a "super-responder" to mepolizumab in severe eosinophilic asthma[END_REF]. In order to standardise the definition, a modified Delphi exercise among healthcare professionals has been conducted but there is a need to understand patient perspectives [START_REF] Upham | Defining a severe asthma super-responder: findings from a Delphi process[END_REF]. The rate of super-responders in patients prescribed anti-IL-5 depending on criteria ranges from 14% to 28% [START_REF] Eger | Long-term therapy response to anti-IL-5 biologics in severe asthmaa real-life evaluation[END_REF][START_REF] Kavanagh | Real-world effectiveness and the characteristics of a "super-responder" to mepolizumab in severe eosinophilic asthma[END_REF][START_REF] Harvey | Mepolizumab effectiveness and identification of super-responders in severe asthma[END_REF], forming a small but important group. Super-response should be the ultimate goal of treatment. However, patients who fail to achieve such a level of improvement may still benefit from biological therapies. Nevertheless, consideration should be given in such cases as to whether a different biologic may be more beneficial. Evaluation of a complete response, as in haematological disorders [START_REF] Barosi | Response criteria for myelofibrosis with myeloid metaplasia: results of an initiative of the European Myelofibrosis Network (EUMNET)[END_REF][START_REF] Barosi | Response criteria for essential thrombocythemia and polycythemia vera: result of a European LeukemiaNet consensus conference[END_REF], should be explored further in severe asthma even though only a very small percentage of patients experience remission [START_REF] Menzies-Gow | An expert consensus framework for asthma remission as a treatment goal[END_REF].

Unfortunately, some patients with severe asthma do not respond to biological therapy and may even deteriorate. Differences in treatment response may be multifactorial, reflecting medicinal and/or subject variables including mechanisms of action, target, dose and interval of the biological drug or heterogeneity of asthma phenotypes [START_REF] Kroes | Prediction of response to biological treatment with monoclonal antibodies in severe asthma[END_REF]. For example, non-response might reflect differences in the pharmacokinetics of biological drugs; indeed, monitoring plasma monoclonal antibody levels appears useful in various chronic diseases [START_REF] Warman | Therapeutic drug monitoring of infliximab in inflammatory bowel disease patients in a teaching hospital setting: results of a prospective cohort study[END_REF][START_REF] Haraoui | Anti-infliximab antibodies in patients with rheumatoid arthritis who require higher doses of infliximab to achieve or maintain a clinical response[END_REF][START_REF] St Clair | The relationship of serum infliximab concentrations to clinical improvement in rheumatoid arthritis: results from ATTRACT, a multicenter, randomized, double-blind, placebo-controlled trial[END_REF].

Overall, assessing the non-response and response after several months of treatment with biologics facilitates cost control by reducing the duration of ineffective therapy, and should enable better quality of care and patient experience by prescribing alternative treatments including switching to another biological if appropriate [START_REF] Magnan | Treatment response with mepolizumab in severe eosinophilic asthma patients with previous omalizumab treatment[END_REF]. The latter is especially important given the rapidly increasing number of therapeutic options for patients with severe asthma [START_REF] Chung | International ERS/ATS guidelines on definition, evaluation and treatment of severe asthma[END_REF][START_REF] Holguin | Management of severe asthma: a European Respiratory Society/ American Thoracic Society guideline[END_REF].

Strengths and limitations

This systematic review was conducted by a diverse group of academic clinicians, patient representatives, and regulatory and pharmaceutical representatives. This was a strength because it meant that definitions were considered on clinical and patient-centred grounds. A comprehensive search was conducted in four databases and provides a summary of the robust research. Rigorous methods were used including risk of bias assessment and GMPs based on COSMIN followed by the modified GRADE approach to rate the certainty of the evidence. Using transparent and validated COSMIN [START_REF] Prinsen | COSMIN guideline for systematic reviews of patient-reported outcome measures[END_REF][START_REF] Mokkink | COSMIN Risk of Bias checklist for systematic reviews of Patient-Reported Outcome Measures[END_REF][START_REF] Terwee | COSMIN methodology for evaluating the content validity of patient-reported outcome measures: a Delphi study[END_REF] methodology helped to standardise the quality assessment of outcome measures and reduce bias. Many studies were excluded as they used arbitrary definitions of response; only methodologically developed definitions and validated outcome measures were considered for inclusion in the systematic review. Lastly, all studies used data from a large number of paediatric and adult patients with severe asthma who were treated with a variety of biological therapies such as omalizumab, brodalumab, benralizumab, reslizumab and mepolizumab.

Nevertheless, we recognise several limitations. First, only studies published in English were included; however, we screened studies included in the guidelines, previous systematic reviews, references of identified articles and reviews, which made it highly unlikely that relevant studies were missed. Second, the search was conducted in 2021 as part of the development of the COMSA which was published in 2023 [START_REF] Khaleva | Development of Core Outcome Measures sets for paediatric and adult Severe Asthma (COMSA)[END_REF]. Third, we only searched the literature related to biological therapies and did not look at the evidence from response to nonbiological asthma therapies. Biologics have different mechanisms of action, administration approaches, costs and potential adverse effects. Therefore, response criteria could differ with different patient views on what counts as a beneficial response given these considerations. However, it may be possible to also learn from the response to other therapies such as to oral and inhaled corticosteroids in severe asthma. Fourth, definitions of therapeutic response were assessed at different time-points, which might make it difficult to come to definitive conclusions about non-responders and responders. Moreover, COSMIN suggest using the lowest score counts method to assess measurement properties, meaning that having higher quality scores on some items of the checklist was not considered and only the "worst score" was reported. Lastly, it was not possible to run a meta-analysis due to low number of studies per outcome measure and only narrative synthesis was undertaken.

Policy implications and next steps

This systematic review aimed to inform clinicians, regulators and policy makers about the gaps and highlight heterogeneity of the definitions used. Even though the Asthma Control Questionnaire/Test and Asthma Quality of Life Questionnaire are widely used in phase 3 trials of asthma biologics and in clinical practice, definitions of response including MCID or MID have never been specifically assessed in biologics. Further research should aim to explore the identified definitions as primary and secondary outcomes in clinical trials including phase 2 and 3 efficacy studies and assess the MCID/MID of well-validated questionnaires in biological trials. There is also a need to methodologically develop patient-centred definitions of non-response and response to biological therapy for severe asthma for individual PROMs and clinical as well as a composite outcome measures. For example, based on COSMIN methodology for assessing the content validity of PROMs [START_REF] Terwee | COSMIN methodology for evaluating the content validity of patient-reported outcome measures: a Delphi study[END_REF], patients should be asked about their relevance, comprehensiveness and comprehensibility. Engagement of patients is a crucial aspect of the development of outcome measures to meet their needs and preferences as well as to inform health decisions [START_REF] Tapp | Patient perspectives on engagement in shared decision-making for asthma care[END_REF][START_REF] Forsythe | Patient engagement in research: early findings from the patient-centered outcomes research institute[END_REF].

Given the aforementioned, we are planning to develop definitions of non-response and response to biological therapies for paediatric and adult severe asthma trials and clinical practice based on the COMSA selected among key stakeholder groups, including patients with severe asthma [START_REF] Khaleva | Development of Core Outcome Measures sets for paediatric and adult Severe Asthma (COMSA)[END_REF]. We aim to standardise the definitions, which will allow better tailoring of individual treatment and be used in future clinical trials for documenting therapeutic response. Furthermore, looking at multiple dimensions of asthma such as exacerbations, QoL, asthma control and lung function in one single patient-centred composite would help to determine the correct sample size for future clinical trials, assist regulators in determining whether a new biological therapy is effective and identify predictors of treatment response. Use of such definitions will also help in better understanding the applicability of novel biomarkers such as volatile organic compounds [START_REF] Brinkman | Exhaled volatile organic compounds as markers for medication use in asthma[END_REF], peripheral blood gene expression [START_REF] Choy | Peripheral blood gene expression predicts clinical benefit from anti-IL-13 in asthma[END_REF][START_REF] Upchurch | Whole blood transcriptional variations between responders and non-responders in asthma patients receiving omalizumab[END_REF] and serum periostin [START_REF] Caminati | Serum periostin during omalizumab therapy in asthma: a tool for patient selection and treatment evaluation[END_REF] in the prediction and monitoring of response, which have been shown to be promising in biological treatment for severe asthma.

Conclusions

This systematic review is the first to evaluate the quality of evidence for definitions of response to biological therapy for severe asthma and measurement properties of associated outcome measures. There are several high-quality definitions available for use that are mostly based on MIDs or MCIDs, which might not be sufficient to justify continuation of biological therapy on cost-effectiveness criteria. Even though composite outcome measures are available and able to capture the multidimensional nature of severe asthma, none were developed with patient input and all lack a QoL component. Quality of evidence for the development and validation of the outcome measures was rated predominantly "low" and "very low", and none met all the methodological quality standards, highlighting an urgent unmet need. Therefore, the forthcoming 3TR project will aim to develop the definitions of non-response and response based on COMSA [START_REF] Khaleva | Development of Core Outcome Measures sets for paediatric and adult Severe Asthma (COMSA)[END_REF] with involvement of patient representatives and other key stakeholders. Future research will be needed to pilot these definitions in biological trials, and to address practical implications for policy makers, research and clinical practice. Knowing how to evaluate response to biologics using universally acceptable criteria would help in assessing the effectiveness of novel therapies, and improve clinical decision making and the care of patients with severe asthma.

Provenance:

Submitted article, peer reviewed.

Acknowledgements:

We would like to acknowledge the support of 3TR in funding the development of this systematic review. We would like to thank Paula Sands (Research Engagement Librarian, University of Southampton, Southampton, UK) for her assistance in optimising the search strategy. This study is registered at PROSPERO with identifier number CRD42021211249. Novartis, Chiesi, Boehringer Ingelheim, Genentech, Roche, Sanofi, Mologic, 4DPharma and Teva, and support from the 3TR project. A. Bourdin reports being an investigator for clinical trials promoted by AstraZeneca, Chieisi, GlaxoSmithKline, Boehringer Ingelheim, Novartis, Regeneron and Sanofi; having received fees for lectures, attendance of meeting and consultancy from AstraZeneca, Chieisi, GlaxoSmithKline, Boehringer Ingelheim, Novartis, Regeneron and Sanofi; having received research grants from AstraZeneca and Boehringer Ingelheim; and participation on a data safety monitoring or advisory board of AB Science. A. Bossios has received lecture fees from GlaxoSmithKline, AstraZeneca, Teva and Novartis; honoraria for advisory board meetings from GlaxoSmithKline, AstraZeneca, Teva, Novartis and Sanofi; and received support for attending meetings from AstraZeneca and Novartis, all outside the present work; reports being a member of the Steering Committee of SHARP, Secretary of Assembly 5 (Airway Diseases, Asthma, COPD and Chronic Cough) of the European Respiratory Society and Vice-chair of the Nordic Severe Asthma Network (NSAN). K.F. Chung has received honoraria for participating in advisory board meetings of GlaxoSmithKline, AstraZeneca, Roche, Novartis, Merck and Shionogi regarding treatments for asthma, COPD and chronic cough, and has also been renumerated for speaking engagements for Novartis and AstraZeneca. R. Chaudhuri has received lecture fees from GlaxoSmithKline, AstraZeneca, Teva, Chiesi, Sanofi and Novartis; honoraria for advisory board meetings from GlaxoSmithKline, AstraZeneca, Teva, Chiesi and Novartis; sponsorship to attend international scientific meetings from Chiesi, Napp, Sanofi, Boehringer, GlaxoSmithKline and AstraZeneca, and a research grant to her Institute from AstraZeneca for a UK multicentre study. C. Coleman declares funding received to support this work by the European Lung Foundation (ELF) from the European Commission's Innovative Medicines Initiative 2 Joint Undertaking ( JU) under grant agreement number 831434 (3TR), and is an employee of the ELF. R. Djukanovic declares funding from European Respiratory Society, Teva, GlaxoSmithKline, Novartis, Sanofi and Chiesi for the SHARP CRC, consulting fees for Synairgen; honorarium for a lecture from GlaxoSmithKline, participation on a data safety monitoring board or advisory board for Kymab (Cambridge) and shares in Synairgen, outside the submitted work. S-E. Dahlen declares funding from 3TR IMI Grant; consulting fees from AstraZeneca, Cayman Co., GlaxoSmithKline, Novartis, Regeneron, Sanofi and Teva; honoraria for lectures from AstraZeneca and Sanofi. A. Exley declares being a minority shareholder in GlaxoSmithKline PLC. L. Fleming declares participation in advisory boards and honoraria for lectures from Sanofi, Respiri UK, AstraZeneca, Novartis and Teva, outside the scope of this publication. All payments were made to her institution. A. Gupta received speaker and advisory board fees from GlaxoSmithKline, Novartis, AstraZeneca and Boehringer Ingelheim. A. Gupta's institution had received research grants from GlaxoSmithKline, Novartis, AstraZeneca and Boehringer Ingelheim. E. Hamelmann declares support from the German Ministry of Education and Research (BMBF) and German Asthma Net (GAN) e.V.; funding for research in severe asthma in children (CHAMP-01GL1742D) and for Severe Asthma Register. G.H. Koppelman reports receiving research grants from the Lung Foundation of the Netherlands, Ubbo Emmius Foundation, H2020 European Union, Teva, GlaxoSmithKline and Vertex, outside this work (money to institution); he reports memberships of advisory boards to GlaxoSmithKline and PURE-IMS, outside this work (money to institution). E. Melen has received consulting fees from AstraZeneca, Chiesi, Novartis and Sanofi outside the submitted work. V. Mahler has no conflict of interest but declares that the views expressed in this review are the personal views of the author and may not be understood or quoted as being made on behalf of or reflecting the position of the respective national competent authority, the European Medicines Agency, or one of its committees or working parties. F. Singer reports being an investigator for clinical trials promoted by Vertex and having received fees for lectures from Vertex and Novartis, outside the submitted work. C. Porsbjerg declares grants, consulting fees and honoraria from AstraZeneca, GlaxoSmithKline, Novartis, Teva, Sanofi, Chiesi and ALK (paid to institution and personal honoraria); participation in the advisory board for AstraZeneca, Novartis, Teva, Sanofi and ALK, outside the submitted work. V. Ramiconi reports grants paid to EFA from Pfizer, Novartis, AstraZeneca, Sanofi, Chiesi Farmaceutici, Regeneron, DBV Technologies, MSD, GlaxoSmithKline, Aimmune, LeoPharma, AbbVie, Boehringer Ingelheim, OM Pharma and Roche; payment for expert testimony from Novartis Global Respiratory Patient Council 2021 and Novartis EPIS Steering Committee to EFA. G. Roberts declares EU IMI funding and consulting fees from AstraZeneca paid to his institution. No other author has any conflict of interest to declare. Support statement: G. Roberts and E. Khaleva were supported by the NIHR Southampton Biomedical Research Centre. This project has received funding from the Innovative Medicines Initiative 2 Joint Undertaking ( JU) under grant agreement number 831434 (3TR). The JU receives support from the European Union's Horizon 2020 research and innovation programme and the European Federation of Pharmaceutical Industries and Associations (EFPIA). The funder had no role in the development of the protocol, conduct or write up of the review or decision to publish. Funding information for this article has been deposited with the Crossref Funder Registry.

Disclaimer:

The content of this publication reflects only the authors' views and the Joint Undertaking is not responsible for any use that may be made of the information it contains.

Author contributions:

E. Khaleva developed a protocol and a search strategy, and G. Roberts and A. Rattu reviewed this. E. Khaleva and A. Rattu performed abstract screening, data extraction and COSMIN evaluation. E. Khaleva synthesised the evidence and wrote the first draft of the manuscript. All authors critically reviewed the manuscript and approved the final version prior to submission.

Conflict of interests:

E. Khaleva and A. Rattu declare funding for the present manuscript from the 3TR European Union Innovative Medicines Initiative 2 paid to the university. C. Brightling declares grants from GlaxoSmithKline, AstraZeneca, Novartis, Chiesi, Boehringer Ingelheim, Genentech, Roche, Sanofi, Mologic and 4DPharma, consulting fees from GlaxoSmithKline, AstraZeneca,


Vincent Berthier

Hassen Doghmen

Olivier Teytaud

Consistency Modifications for Automatically Tuned Monte-Carlo Tree Search

; these algorithms are now widely accepted as strong candidates for highdimensional control applications. Unfortunately, it is known that for optimal performance on a given problem, MCTS requires some tuning; this tuning is often handcrafted or automated, with in some cases a loss of consistency, i.e. a bad behavior asymptotically in the computational power. This highly undesirable property led to a stupid behavior of our main MCTS program MoGo in a real-world situation described in section 3. This is a big trouble for our several works on automatic parameter tuning [3] and the genetic programming of new features in MoGo. We will see in this paper:

-A theoretical analysis of MCTS consistency;

-Detailed examples of consistent and inconsistent known algorithms; -How to modify a MCTS implementation in order to ensure consistency, independently of the modifications to the "scoring" module (the module which is automatically tuned and genetically programmed in MoGo); -As a by product of this work, we'll see the interesting property that some heavily tuned MCTS implementations are better than UCT in the sense that they do not visit the complete tree (whereas UCT asymptotically does), whilst preserving the consistency at least if "consistency" modifications above have been made.

1 Introduction: tuning vs consistency in MCTS Usually, when working on bandits, theoreticians have various models of problems, for which they propose possibly optimal solutions from the point of view of the rates. Unfortunately, to the best of our knowledge, there's no bandit analysis which can be applied for establishing rates for Monte-Carlo Tree Search (MCTS) algorithms; the rates essentially depend on the quantity of tuning you put in your algorithm for biasing the tree search (this tuning is often performed around the "score" function described below). Unfortunately, manually adding heuristics or automatically tuning MCTS certainly improves rates, but it often destroys the good asymptotic properties (consistency) of MCTS. Our goal, in this work, is to modify MCTS so that we can apply automatic parameter tuning and genetic programming, without any loss of consistency.

Consistency, in UCT-like algorithms [START_REF] Chaslot | Monte-Carlo Strategies for Computer Go[END_REF][START_REF] Coulom | Efficient selectivity and backup operators in monte-carlo tree search[END_REF][START_REF] Kocsis | Bandit-based monte-carlo planning[END_REF], is usually considered as trivial, with arguments like "all the tree is asymptotically explored, and therefore the algorithm is consistent". This is certainly true for the "true" UCT version [START_REF] Kocsis | Bandit-based monte-carlo planning[END_REF], but not necessarily for the many optimized versions of MCTS proposed in the literature [START_REF] Coulom | Efficient selectivity and backup operators in monte-carlo tree search[END_REF][START_REF] Chaslot | Monte-Carlo Strategies for Computer Go[END_REF][START_REF] Gelly | Combining online and offline knowledge in uct[END_REF][START_REF] Lee | The computational intelligence of mogo revealed in taiwan's computer go tournaments[END_REF] which do not visit the whole tree; moreover, as will be shown later, the fact that a MCTS implementation asymptotically builds a complete tree of possible futures does not necessarily make it consistent. These frugal versions (which save up memory and computational power1 ) are the only ones which provide optimal performance. For e.g. the classical testbed of the game of Go, consistency is non trivial. The goal of this research is to provide a as clear as possible frontier between consistent implementations and non-consistent implementations, with the following properties:

-The frontier should provide sufficient conditions that are easy to satisfy by a few corrections in MCTS implementations; also, correcting an optimized implementation for ensuring consistency should give better results on the cases of bad behavior of the algorithm, without reducing its efficiency on low scale experiments (e.g. short time settings). -It should also be compliant with automatic tuning, i.e. the modifications should not forbid free modifications of the scoring function.

Proved (section 2) and efficient solutions will be provided, tested on examples (section 3) and experimented (section 4). Il all the paper, #E denotes the cardinal of a set E.

Model and theory

A game is (here) a finite set of nodes, organized as a tree with a root. Each node n is of one of the following types:

max node (nodes in which the max player chooses the next state among descendants); -min node (nodes in which the min player chooses the next state among descendants); -terminal node; then, the node is equipped with a reward Reward(n) ∈ [0, 1]; -random node; then, the node is equipped with a probability distribution on its descendants.

In all cases, we note D(n) the set of children of node n. We assume, for the sake of simplicity, that the root node is a max node. We will consider algorithms which perform simulations; the first simulation is s 1 , the second simulation is s 2 , etc. Each simulation is a path in the game. Each node n is equipped with:

-Possibly, some side information I(n).

-A father F (n), which is the father node of n; this is not defined for the root.

-A value V (n); this value is the Bellman value; it is known since [START_REF] Bellman | Dynamic Programming[END_REF] that V (n), equal to the expected value if both players play optimaly, is well defined. -For each t ∈ {0, 1, 2, 3, 4, . . . },

• n t (n) ∈ {0, 1, 2, . . . } is the number of simulations with index in 1, 2, . . . , t -1 including node n, possibly plus some constant K 1 (n) [START_REF] Coulom | Efficient selectivity and backup operators in monte-carlo tree search[END_REF][START_REF] Chaslot | Progressive strategies for monte-carlo tree search[END_REF][START_REF] Lee | The computational intelligence of mogo revealed in taiwan's computer go tournaments[END_REF]:

n t (n) = #{i < t; n ∈ s i } + K 1 (n). (1)
• w t (n) ∈ R is the sum of the rewards of the simulations with index in 1, 2, . . . , t-1 including node n, possibly plus some constant K 2 (n) (taking into account expert knowledge of offline values):

w t (n) = i<t;n∈si reward(s i ) + K 2 (n). ( 2
)
• If n is not the root, score t (n) = score(w t (n), n t (n), n t (F (n)), I(n), t).
We consider algorithms as in Algorithm 1.

Remarks:

-The score might be stochastic, without impact on our results.

-The fact that the game is a tree, instead of a directed acyclic graph (DAG), is not central; it's just more convenient for notations, as it gives sense to the score of a node, independently of its fathers; all results could be extended to the case of DAGs. Thanks to this simplification, also, we can identify a move and the situation that is reached after this move -this simplifies the writing of Algo. 1.

Let

Opt(n) = {s ∈ D(n); ∀s ′ ∈ D(n), V (s ′ ) ≤ V (s)} if n is a max node.
(just replace ≤ by ≥ for min nodes) Opt(n) is the set of optimal children of n.

We note Vt (n) = wt(n) nt(n) . The case n t (n) = 0 might be solved with e.g.

n t (n) = 0 ⇒ Vt (n) = F P U (n) (3)
for some First-Play Urgency function F P U . In all the rest of this paper, unless otherwise stated, o(.) and lim refer to t → ∞; also, "infinitely often" means "for infinitely many t". We now state the following Algorithm 1: A framework of Monte-Carlo Tree Search. All usual implementations fall in this schema depending on the score function. The decision step is sometimes different, without impact on our results.

Theorem (Consistency of Monte-Carlo Tree Search.) Consider a MCTS Algorithm as in Algo. 1. Assume that for all sons s, s ′ of a max node:

n t (s) → ∞, n t (s ′ ) → ∞, lim inf Vt (s) > lim sup Vt (s ′ ) ⇒ n t (s ′ ) = o(n t (s)) (4) n t (F (s)) → ∞ and lim inf Vt (F (s)) < V (F (s)) ⇒ n t (s) → ∞ (5)
Assume the same equations 4 and 5, with lim inf replaced by lim sup and < exchanged with >, for min nodes. Then, almost surely, there exists t 0 such that

∀t ≥ t 0 , arg max n∈D(root) N t (n) ⊂ arg max n∈D(root) V (n). (6)
Eq. 6 is the consistency of MCTS, for algorithms using the decision rule in Alg. 1. There are some other decision rules used in MCTS implementations, but Eq. 6 also immediately shows the optimality of these decision rules (other known decision rules are asymptotically equivalent to the classical decision rule we have proposed).

Remark 1: To the best of our knowledge, Eq. 4 holds for all strong implementations. This is certainly not the case for Eq. 5; we'll see counter-examples in section 3.

Remark 2: Eq. 5 becomes particularly interesting when an upper bound on V (F (s)) is known: no use exploring new children when Vt (s) → V (F (s)). In deterministic games with binary rewards, Eq. 5 boils down to:

lim inf Vt (F (s)) < 1 ⇒ n(s) → ∞.
We don't request anything if lim inf Vt (F (s)) = 1. This means that only one move can be simulated at s, as long as we have no refutation for this move. This implies a in-depth analysis of a few moves, which is central in the success of MCTS e.g. in Go and Havannah [START_REF] Teytaud | Creating an Upper-Confidence-Tree program for Havannah[END_REF].

Remark 3: We only claim ⊂ and not = in Eq. 6. We will choose an optimal move, but we will not necessarily find all optimal moves.

Proof of the Theorem:

We will first show that for each node n,

lim t→∞ n t (n) = ∞ ⇒ Vt (n) → V (n). (7)
The proof of Eq. 7 is made by induction; we show it for leaf nodes, and then we show that if it holds for the sons of the node n, then it holds for n.

Proof of Eq. 7: If n is a leaf (a terminal node), then Eq. 7 obviously hold. Let's now assume that Eq. 7 holds for sons of n, and let's show it for n. This is immediate if n is a random node. The case of "min" is symetrical to the case of "max" nodes, and we will therefore only consider the case of max.

First, obviously lim sup Vt (n) ≤ V (n). This is because Vt (n) is a weighted average of the Vt (s), for s sons of n, and Vt (s) → V t (s) for all s simulated infinitely often.

Therefore, we just have to show that

lim inf Vt (n) ≥ V (n). (8)
In order to show Eq. 8, we will assume, in order to get a contradiction, that

lim inf Vt (n) < V (n). (9)
If Eq. 9 holds, then, by Eq. 5, each s (son of n) is simulated infinitely often. Therefore, for each s, son of n, Vt (s) → V t (s). By Eq. 4, this implies that all suboptimal sons s ′ verify n t (s ′ ) = o(n t (s)) for any s ∈ Opt(n). This implies that Vt (n) converges to a weighted average of the Vt (s), for some (possibly one, possibly several) s ∈ Opt(n) simulated infinitely often. Vt (s) → V (s) for all these sons, by the induction hypothesis. This implies

Vt (n) → max s∈Opt(n) V t (s).
This is a contradiction with Eq. 9; Eq. 8 is therefore proved. This concludes the proof of Eq. 7, by induction.

We have shown Eq. 7. We now have to show the conclusion of the theorem, i.e. Eq. 6. If all sons of the root are optimal, there is nothing to prove; let's assume that at least one son of the root is not optimal.

Consider ∆ the minimum difference between V (s) for suboptimal sons, and V (root); i.e.:

∆ = V (root) - sup s∈D(root),V (s)<V (root) V (s).
∆ > 0 by assumption. Eq. 7 in particular holds for the root node. This implies that V (root) → V (root). [START_REF] Lai | Asymptotically efficient adaptive allocation rules[END_REF] Assume, in order to get a contradiction, that for some k > 0 s∈D(root),V (s)<V (root)

n t (s) ≥ k • t (11)
infinitely often.

Then, Vt (root) is infinitely often the weighted average of the Vt (s), for s optimal nodes, with total weight at most 1 -k; -the Vt (s ′ ), for s ′ suboptimal nodes, with total weight at least k.

This implies that lim sup Vt (root) ≤ V (root)-k∆; this contradicts Eq. 10; therefore the assumption (Eq. 11) leads to a contradiction. We have therefore shown, by contradiction, that Eq. 11 does not hold, for any k > 0. This implies Eq. 6.

No consistency with naive ǫ-greedy algorithms. In MCTS algorithms, ǫgreedy strategies do not provide consistency. For example, even with a finite set of nodes, and for any positive value of ǫ, a rule like "in each node, with probability ǫ, select a node randomly and uniformly instead of using the score for choosing the next node" is not consistent. A counter-example is given in Fig. 1.

Saving up memory with MCTS algorithms. One might remark, however, that this counter-example is limited to ǫ-greedy algorithms with ǫ constant; we recover consistency if we have ǫ → 0 sufficiently quickly (but sufficiently slowly as well). However, ǫ-greedy algorithms, in all these cases, have the drawback that all the tree is visited infinitely often. In particular, all the tree is constructed in memory. As pointed out above, this is certainly not the case of MCTS algorithms with other rules: in the binary case with reward in {0, 1}, if a given arm always wins, and at least for good bandit scores, then none of the other arms is never tested. Counter-example for naive ǫ-greedy exploration in MCTS. In this game, the max player should choose the right side and win a reward 1. However, for any ǫ, with an ǫ-greedy heuristic, and for many bandit rules (even UCB, which would be obviously consistent without the ǫ-greedy modification), MCTS would choose the left side (at least if the number of levels is sufficient), and loose if the min player plays well. Please note however that this counter-example would not hold for ǫ decreasing to zero. This is a counter-example for fixed ǫ only. However (see text), the case of ǫ decreasing to zero can be consistent (depending on the score) but it will certainly not have the frugality property: it will visit all the tree infinitely often.

Consistency of UCT. UCT is Algorithm 1 with the particular case score t (s) = Vt (s) + log(n t (F (s)))/n t (s)

(constants or other terms are often placed before log(i); this does not matter for us here) Obviously, the classical UCT (with UCB [START_REF] Lai | Asymptotically efficient adaptive allocation rules[END_REF][START_REF] Auer | Finite time analysis of the multiarmed bandit problem[END_REF] as bandit algorithm) verifies our assumptions; its variants like "progressive widening" [START_REF] Coulom | Efficient selectivity and backup operators in monte-carlo tree search[END_REF], "progressive unpruning" [START_REF] Chaslot | Progressive strategies for monte-carlo tree search[END_REF] (see also [START_REF] Wang | Algorithms for infinitely many-armed bandits[END_REF] for these modifications of UCB), as well. Our purpose is precisely to consider cases different from UCT, and in particular the frugal versions (those which do not visit all the tree).

Unconsistency of some forms of progressive bias. This is not the case for all "progressive bias" methods [START_REF] Gelly | Combining online and offline knowledge in uct[END_REF][START_REF] Chaslot | Progressive strategies for monte-carlo tree search[END_REF]. For example, a classical formula for Rave values is

score t (s) = n t (s) n t (s) + K Vt (s) + K n t (s) + K V ′ t (s) for some constant K > 0, ( 13
)
where V ′ t (s) is the Rave value of s [START_REF] Gelly | Combining online and offline knowledge in uct[END_REF]. It has been pointed out in the computer-Go mailing-list that in some cases, the Rave heuristic [START_REF] Gelly | Combining online and offline knowledge in uct[END_REF] gives value V ′ t (g) = 0 to the only good move (cf the interesting posts by Brian Sheppard on the computer-go mailing list http:// www.mail-archive.com/computer-go@computer-go.org/msg12202.html). If the score is

score t (g) = n t (g) n t (g) + K Vt (g) + K n(g) + K V ′ t (g),
then the score of the good move g is 0 as long as n t (g) = 0; this move is never simulated as long as there is a move with score > 0. The trouble is that a bad move b might have a non-zero Rave value, and therefore its score is

score t (b) ≥ K n t (b) + K V ′ t (b) > 0.
This means that only the bad move has a non-zero score, the good move stays at a score 0 and is never simulated.

Unconsistency of negative heuristics. The program MoGo, known for several successes in the game of Go (including the only ever win against a pro with handicap 6 and the only ever win against a top pro 9p (winner of the LG Cup 2007) with handicap 7. MoGo has a complicated bandit involving a compromise between (i) the empirical reward Vt (s) (ii) the Rave value V ′ t (s) (iii) the pattern-based value (combined with expert rules) H(I(s)). A score can in fact, when patterns and expert rules agree against a move, be negative; this means that the score can be negative. The trouble is that there are particular cases in which even very bad patterns might be the only good move; Go experts know some examples in which the famous "empty triangle" (known as a very bad pattern) can be a good move.

We'll see below that this can happen in MoGo; this is not only theoretical: the situation given below occured in a real game against a professional player. MoGo was convinced that it was going to win, whereas the opponent was convinced that he was in good situation -when black replied E6, the computer (white) simulated this move (which was not yet simulated at all, except during a small transitory regime!) and understood that it had lost the game.

A game lost by MoGo against Fan Hui in 9x9 Go. MoGo had the opportunity to play against Fan Hui, 2nd Dan pro, in Toulouse (August 2009). MoGo won one game, lost two games; if the game presented below had been won by MoGo, this would have been the first ever win of a computer against a pro on a complete match (and not only on a simple game). A very surprising situation happened: whereas usually MoGo, as well as many MCTS implementations, is extremly strong in endgames, it was completly convinced of winning the game whereas the situation was desperate. The game is presented in Fig. 2. Tricks for consistency. We propose below some simple implementation tricks for ensuring that the algorithm is consistent. All "consistency" tricks can easily be checked using Eq. 4 and 5 in the theorem above; and easy counterexamples can be built for all non-consistent rules. The case in which the reward lies in a finite set is particularly interesting, as we'll see that we can then have consistency whilst preserving the suitable property that we do not simulate the whole tree.

-Using score t (s) = Vt (s) with a FPU as in Eq. 3 (and K 1 (.) = K 2 (.) = 0 in Eqs 1 and 2) is not consistent (if there is only one good move and if the first simulation with this move leads to a loss, we can have a score 0 for this move (for the eternity as it will never be simulated again), and a score > 0 (for the eternity) for a bad move which alway looses except for its very first simulation; consistency is recovered with

score t (s) = Vt (s) + log(n t (F (s)))/n t (s)
but this is not satisfactory as it implies that all the tree is visited infinitely often.

-For Rave values for the case of deterministic games with reward ∈ {0, 1} (see Eq. 13 and non-consistency discussed there), a simple solution consists in replacing V ′ (t) by max(V ′ (t), 0.1). -For methods with score closely related to the average reward without any upper confidence bound term, we suggest:

score t (s) = w t (s) + K n t (s) + 2K (14)
with K > 0. This was empirically derived, for the program MoGo, in [START_REF] Lee | The computational intelligence of mogo revealed in taiwan's computer go tournaments[END_REF] as a good tool, instead of wt(s) nt(s) (i.e. the case K = 0) when there's no upper confidence term + log(n t (F (s)))/n t (s) (and removing this upper confidence term makes MoGo much faster!). Interestingly, we clearly see here an advantage of Eq. 14 instead of Eq. 12: we have consistency in both cases (Eqs 4 and 5 are clearly verified with Eq. 14 and Eq. 12), but the difference is that Eq. 12 implies that all the tree is visited infinitely often, whereas Eq. 14 does not: with Eq. 14 some parts of the tree are never visited.

-Consider now the case in which the rule is modified by genetic programming and automatic tuning, and we don't want to modify the score manually.

Consider the case of a deterministic game with discrete reward ∈ {0, 1}. Importantly, this is not only for games; adversarial settings are important in verification, and are usual models for unkown random processes. Consider V threshold some constant such that for n a max node:

∃(s 1 , s 2 ) ∈ D(n), V (s 1 ) < V (s 2 ) ⇒ V threshold ∈] sup s∈D(n),V (s)<V (n) V (s), V (root)[.
(just replace:

• V threshold by V ′ threshold ,
• < by >,

• and sup by inf for min nodes) We see that if Bellman values live in {0, 1} (games with binary reward and no chance), it is sufficient to choose any V threshold ∈]0, 1[, and to use Alg. 2 instead of Alg. 1 (this is a one-line modification of MCTS!). For the game of Go, V threshold = 0.3 is used in our experiments (this was handcrafted, without any tuning). The idea is simply that if Vt (s) < V threshold (for a max node) then we simulate randomly one of the sons of s instead of maximizing the score. This is the choice made for our tests below. Algorithm 2: A modification of Algo. 1 which ensures consistency for binary deterministic games. The difference between this algorithm and Algo. 1 is the application of the consistency modification in the Theorem.

In the game presented in Fig. 2, MoGo poorly estimated the situation. We check now if the new version of MoGo (the version using the consistency modification in Eq. 2) understands more clearly that in this situation, the game is over for white. MoGo provides the following results with 500000 simulations per move:

the expected success rate is above 90% in 13 over 30 runs in the initial version of MoGo; -never over 30 runs with the change suggested in the theoretical analysis above.

We could also test the version with the modification against the version without the modification; no decrease of performance could be found. We also did not find significant improvements. Results are presented in Table 1. These experiments are performed with the "score" function of MoGohttp:

Number of simulations per move Success rate of Algo. 2 vs Algo. 1 500 000 (0.3) 52.9 % ± 0.9 % 5 000 000 (0.3) 52.1 % ± 4.9 % Table 1. Performance of MoGo with the "consistency" modification, vs the current version of MoGo. //www.lri.fr/ ~teytaud/mogo.html. Usual experiments performed in the field consider nearly 10 000 simulations per move -this is because it's much easier to perform automatic parameter tuning with games played in 1s per move (minutes for complete games) than with 5 000 000 simulations per move (which require half an hour per game on a strong machine -this is closer to real-world cases, but much more expensive).

Conclusion

MCTS is a recent and very important class of algorithms. We have shown that consistency becomes an issue in MCTS programs which are highly optimized on experiments with limited scales (typically, limited time settings). This becomes particularly important for genetically evolved programs, as well as automatically tuned programs. We have shown that there are real world cases of failures due to this lack of consistency.

We proposed mathematically proved methods for ensuring consistency independently of the parts of MCTS which are automatically tuned. Interestingly, we also point out that some heavily optimized algorithms are better than UCT in some settings in the sense that they do not simulate the whole tree, whereas UCT does (UCT even simulates the whole tree infinitely often!); and they nonetheless preserve consistency, at least if tricks for consistency as presented in this paper are applied.

Interestingly, the results are particularly visible on a real-world case which would be difficult to reproduce in artificial experiments: big ko-fights as in Fig. 2 are based on cultural knowledge of strong players and are rare in games between computers. It's very difficult to derive a good behavior on such cases just on the basis of artificial experiments, whereas a mathematical consistency analysis has succeeded. This shows the soundness and the generality of the approach.

The main limitation of this work is that it is limited, at least for the detailed mathematical analysis, to the tuning of MCTS algorithms. However, MCTS is a very important class of algorithms, for control and games. Moreover, the principle of exploring infinitely often only children of nodes which have empirical reward below the expected value in case of success control makes sense in all simulationbased control algorithms.

Fig. 1 .

 1 Fig.1. Counter-example for naive ǫ-greedy exploration in MCTS. In this game, the max player should choose the right side and win a reward 1. However, for any ǫ, with an ǫ-greedy heuristic, and for many bandit rules (even UCB, which would be obviously consistent without the ǫ-greedy modification), MCTS would choose the left side (at least if the number of levels is sufficient), and loose if the min player plays well. Please note however that this counter-example would not hold for ǫ decreasing to zero. This is a counter-example for fixed ǫ only. However (see text), the case of ǫ decreasing to zero can be consistent (depending on the score) but it will certainly not have the frugality property: it will visit all the tree infinitely often.

Fig. 2 .

 2 Fig. 2. The game lost by MoGo against the professional player Fan Hui: this is a ko fight. MoGo (white) plays C1 and is very confident; in fact, black replies E6 (which was not simulated by MoGo!) and wins.

  Input: a game. Possibly initialize wt and nt to some arbitrary values. for t = 0, 1, 2, 3, . . . (until no time left) do Output: a decision, i.e. the node in which to move.

	st ← ()	// empty simulation
	s = root(game)
	while s is not terminal do
	st ← st.s // s is added to the simulation if s is a max (resp. min) node and Vt(s) < V threshold (resp. Vt(s) > V ′ threshold )
	then
	randomly draw s ∈ D(s)
	else
	switch s do
	case max node
		s ← arg max n∈D(s) score(n)
	end
	case min node
		s ← arg min n∈D(s) score(n)
	end
	case random node
		s ← randomly drawn node according to distribu-
		tion at s
	end
	end
	end if
	end while
	st ← st.s // s is added to the simulation
	end for
	decision = arg max n∈D(root) nt(n)	// decision rule

We will refer to these algorithms, which do not necessarily visit all the tree, as "frugal" algorithms. We'll see that usual non-frugal algorithms asymptotically visit all the tree infinitely often.

Acknowledgements

We thank the various people who inspired the development of MoGo: Bruno Bouzy, Tristan Cazenave, Albert Cohen, Thomas Hérault, Rémi Coulom, Rémi Munos, the computer-go mailing list, the KGS community, the Cgos server, the IAGO challenge, the Recitsproque company, the National University of Tainan (Taiwan). We thank Grid5000 for providing computational resources for experiments presented in this paper. We thank Jean-Yves Papazoglou and Eric Saves for organizing the games with Fan Hui and for the opportunity of meeting plenty of great Go player. We also thank the many direct contributors of MoGo, who can't all be cited here http://www.lri.fr/ ~teytaud/mogo.html.