Abdul Qayoom Mugheri

Aneela Tahira

Umair Aftab

Ayman Nafady

Brigitte Vigolo

Zafar Hussain

Dr M A Kazi

Prof Hussain Zafar

Ibupoto

Facile Co 3 O 4 nanoparticles deposited on polyvinylpyrrolidine for efficient water oxidation in alkaline media

Keywords: Carbonized Co3O4 nanoparticles, polyvinylpyrrolidone, oxygen evolution reaction

Introduction

The water splitting reaction on the surface of electrocatalyst describes the [H2O → O2 + 2H2] production of hydrogen and oxygen gases and it is considered one of the potential strategies for the generation of green H2 and O2 based fuels renewable energy applications [START_REF] Ghribi | Study of hydrogen production system by using PV solar energy and PEM electrolyser in Algeria[END_REF][START_REF] Yongtao | Structure-property relationship of bifunctional MnO2 nanostructures: highly efficient, ultra-stable electrochemical water oxidation and oxygen reduction reaction catalysts identified in alkaline media[END_REF]. Unfortunately, the big barrier in water electrolysis is oxidation reaction at the anode which consumes large overpotential and it demonstrates a dead kinetics for the oxygen evolution reaction (OER) as OER process is followed by the four-electron transfer [START_REF] Bediako | Mechanistic studies of the oxygen evolution reaction mediated by a nickel-borate thin film electrocatalyst[END_REF][START_REF] Seitz | Enhancement Effect of Noble Metals on Manganese Oxide for the Oxygen Evolution Reaction[END_REF][START_REF] Siracusano | Nanosized IrOx and IrRuOx electrocatalysts for the O2 evolution reaction in PEM water electrolysers[END_REF]. For this reason, numerous research activities have been carried out for the design of highly active electrocatalysts with a low overpotential and enhanced OER kinetics [START_REF] Katsounaros | Oxygen electrochemistry as a cornerstone for sustainable energy conversion[END_REF]. The noble materials like RuO2 and IrO2 have been testified so far as efficient OER electrocatalysts to date [START_REF] Siracusano | Nanosized IrOx and IrRuOx electrocatalysts for the O2 evolution reaction in PEM water electrolysers[END_REF]. Inherently, these precious materials are limited in use at large scale due to their high cost, scarcity, and poor chemical stability in basic media. The development of new nonprecious catalysts for OER application, it is highly needed to consider the cost of fabrication to capitalize them for industrial applications. At the present, we have only Ir/Ru based materials as efficient towards OER, however they are restricted by their high cost. Therefore, it is very important to develop and fabricate a nonprecious, earth abundant and efficient OER electrocatalysts. For this purpose, an intense progress has been made for the synthesis of earth abundant and low cost OER electrocatalysts [START_REF] Bajdich | Theoretical investigation of the activity of cobalt oxides for the electrochemical oxidation of water[END_REF][START_REF] Sheng | MOF-derived PtCo/Co3O4 nanocomposites in carbonaceous matrices as high-performance ORR electrocatalysts synthesized via laser ablation techniques[END_REF]. Despite this, more facile and low-cost techniques are needed for the fabrication of active electrocatalysts based on earth abundant materials. The use of first 3d series transition metal based electrocatalysts has been considered at large extent for the OER application due to their low cost, earth abundance and tunable 3d electronic properties [START_REF] Rosen | Ordered Mesoporous Cobalt Oxide as Highly Efficient Oxygen Evolution Catalyst[END_REF]. These 3d transition metal-based compounds are utilized in OER catalysis in different forms such as metal oxides [START_REF] Wei-Yan | Au-NiCo2O4 supported on three-dimensional hierarchical porous graphene-like material for highly effective oxygen evolution reaction[END_REF][START_REF] Zhuang | Sea urchin-like cobalt-iron phosphide as an active catalyst for oxygen evolution reaction[END_REF], perovskites [START_REF] Chai | High-performance supercapacitors based on conductive graphene combined with Ni (OH) 2 nanoflakes[END_REF], metal oxy/hydroxides [START_REF] Roger | Earth-abundant catalysts for electrochemical and photoelectrochemical water splitting[END_REF]14], metal phosphates [START_REF] Song | Transition metal oxides as electrocatalysts for the oxygen evolution reaction in alkaline solutions: an application-inspired renaissance[END_REF], and organic frameworks [START_REF] Suntivich | A perovskite oxide optimized for oxygen evolution catalysis from molecular orbital principles[END_REF]. Among them, cobalt oxide Co3O4 in spinel structure is of high importance due to its unique properties like high activity [START_REF] Moon G H | Highly Active Cobalt-Based Electrocatalysts with Facile Incorporation of Dopants for the Oxygen Evolution Reaction[END_REF], significant stability [START_REF] Trotochaud | Nickel-iron oxyhydroxide oxygen-evolution electrocatalysts: the role of intentional and incidental iron incorporation[END_REF], and very suitable over wide range of electrolytic pH [START_REF] Menezes | Helical cobalt borophosphates to master durable overall water-splitting[END_REF]. Also, Co3O4 is low-cost material and has high abundance in nature. The use of Co3O4 as OER electrocatalyst with large surface area and high-density active sites is appreciated by the scientific community as it favors Co3O4 for practical applications. Various synthetic methods have been used to produce nanostructured Co3O4 including hydrothermal [START_REF] Xia | Embedding CoS2 nanoparticles in N-doped carbon nanotube hollow frameworks for enhanced lithium storage properties[END_REF], casting [START_REF] Mccrory | Benchmarking hydrogen evolving reaction and oxygen evolving reaction electrocatalysts for solar water splitting devices[END_REF] and plasma engraving [START_REF] Ma | Metal-organic framework derived hybrid Co3O4-carbon porous nanowire arrays as reversible oxygen evolution electrodes[END_REF]. The Co3O4 has been produced into various morphologies like nanowires/nanorods, nanofibers, nanotubes, nanoparticles, nanoflakes and nanosheets through wet chemical and electrochemical methods [START_REF] Gerken | Electrochemical water oxidation with cobaltbased electrocatalysts from pH 0-14: the thermodynamic basis for catalyst structure, stability, and activity[END_REF][START_REF] Xia | Self-supported hydrothermal synthesized hollow Co3O4 nanowire arrays with high supercapacitor capacitance[END_REF][START_REF] Grewe | Development of Highly Active Bifunctional Electrocatalyst Using Co3O4 on Carbon Nanotubes for Oxygen Reduction and Oxygen Evolution[END_REF][START_REF] Xu | Plasma-engraved Co3O4 nanosheets with oxygen vacancies and high surface area for the oxygen evolution reaction[END_REF]. Moreover, the electro-spinning method is promising for the preparation of nanostructured materials like nanofibers which accelerate the ion mobility between electrode-electrolyte interfaces [START_REF] Wang | Morphology-controllable synthesis of cobalt oxalates and their conversion to mesoporous Co3O4 nanostructures for application in supercapacitors[END_REF] due to their large surface area and porosity. The electrospinning technique has been used for the preparation of Co3O4 onto polyvinylpyrrolidone (PVP) fibers to produce composite nanostructure for oxygen reduction reaction [START_REF] Xia | Freestanding Co3O4 nanowire array for high performance supercapacitors[END_REF]. The electro-spun composite nanostructure of RuO2-Co3O4 has been reported with improved catalytic activity for the oxygen reduction reaction [START_REF] Zheng | Preparation of mesoporous Co3O4 nanoparticles via solidliquid route and effects of calcination temperature and textural parameters on their electrochemical capacitive behaviors[END_REF]. The work is novel in terms of functional properties by demonstrating the OER activity in alkaline media for the first time on cobalt oxide nanoparticles deposited on PVP fibers followed by carbonization. Also, the used strategy is facile, low cost, environment friendly and highly suitable from mass production of functional materials for the diverse electrochemical applications. In this study, we present a simple methodology for the synthesis of OER electrocatalyst using PVP carbon nanofibers as a carbonized source for the deposition of Co3O4 nanoparticles by electro-spinning method. The Co3O4 nanoparticles are characterized by XRD, SEM, and EDs techniques. These nanostructured showed an enhanced OER activity in 1.0M KOH with an overpotential of 280 mV for 10 mAcm -2 . Also, the composite catalysts are durable for 20 hrs.

Experimental Section

The Cobalt nitrate hexahydrate, polyvinylpyrrolidine PVP, 20% RuO2/C and ethanol were received from Sigma Aldrich Karachi Pakistan. The chemicals were used as received. The Co3O4 nanoparticles were deposited on PVP nanofibers by electro-spinning method and a detail methodology is as described. The PVP fibers were produced according to the reported work [START_REF] Wang | Preparation and characterization of mesoporous Co3O4 electrode material[END_REF].

First, 0.4g of cobalt nitrate hexahydrate were mixed in 1.6 mL of H2O and 0.4 g of PVP in 2.4 mL of ethanol. Both solutions were kept on stirring for 2 h, then mixed. Then a 19-gauge needle was filled with this solution and electro-spun was performed at the flow rate of 0.3 mL/h at 20 kV with separation distance of 15 cm. The nano fibers product was deposited on the aluminum foil and thermally treated at 500 o C for 5 hrs in air to get carbonized Co3O4 nanoparticles. . The pristine Co3O4 nanostructures were prepared by hydrothermal method using equ-molar concentration (0.1M) of cobalt nitrate hexahydrate and urea at 90 o C for 5 hrs. Afterwards calcination was performed at 500 o C for 5 hours in air. The composition and crystalline structure were studied on Philips's powder diffractogram technique using CuKα radiation (λ = 1.5418 Å), 45 kV and 45 mA.

The SEM measurement was done at at 3 kV and' to ' Scanning electron microscopy was carried out with a ZEISS Gemini SEM 500 equipped with field emission gun.

Electrochemical experiments

The carbonized Co3O4 nanoparticles were used to modify glassy carbon electrode (GCE). The GCE was rubbed with 1µm and 0.05 µm alumina pastes then washed with water. Then sonication was performed on the GCE and rinsed several times with ethanol and left to dry at room temperature. The geometrical surface area of 0.021 cm 2 of GCE was used. The catalyst slurry was as prepared by mixing 5mg/ml of each material such as Co3O4 carbonized Co3O4 nanoparticles and pristine Co3O4 in 0.8 mL of ethanol and mechanically stirred then a 5µl of 5% Nafion were added as binder. Afterwards, 5µL of catalyst was deposited on GCE with a mass of (0.2mg) and dried at room temperature. The use of carbonized Co3O4 nanoparticles and pristine Co3O4 catalyst ink onto the glassy carbon electrode was done separately. The electrochemical measurements were performed on three electrode setup including graphite rod as counter electrode, silver-silver chloride as reference electrode and catalyst modified GCE as working electrode.

The cyclic voltammetry (CV) was used to stabilize the working electrode prior to linear sweep voltammetry (LSV). The LSV was performed at scan rate of 1 mV/s in 1.0M KOH the EIS spectra were recorded from 100 kHz to 1Hz using sinusoidal potential 5 mV and 1.51 V vs RHE in 1.0M KOH. All the potential is converted into RHE using Nernst equation. For the calculation of current density, the measured current was divided by the geometrical surface area of GCE. For better understanding, a brief schematic diagram illustrating the synthesis process and electrochemical analysis is shown in Figure 1.

Results and discussion

To analyze the morphology of different nanostructured materials via scanning electron microscopy (SEM) as enclosed in showed a poor activity, however the carbonized Co3O4 nanoparticles have improved the catalytic performance with an overpotential of 280 mV to produce a 10 mAcm -2 . The use of PVP fibers as a carbonized source not only increased the electrical conductivity of Co3O4 nanoparticles but it also has lifted uniform dispersion of Co3O4 nanoparticles, and exposed of large number of active sites, thus an enhanced OER activity is demonstrated. The bare PVP fibers are found catalytic inactive; therefore, they only provided a high surface area to the Co3O4 nanoparticles. The inactivity of PVP fibers shows that the OER activity is mainly coming from the Co3O4 nanoparticles, and it could be safe to say that the increase in the surface area for a metal oxide nanostructure may have high potential to accelerate the OER kinetics. The meritorious activity of 20%RuO2/C was observed with an overpotential 160 mV to reach a 10 mAcm -2 . As, it is well known that the RuO2 being a noble catalyst is limited by the widespread use due to its high cost and rare abundance. However, the carbonized Co3O4 nanoparticles being a low cost, earth abundant and improved OER activity can be used at large scale for practical applications. The scale up features of proposed material can be capitalized for the overall water splitting. The superior OER performance of carbonized Co3O4 nanoparticles might be assigned to their high surface to volume ratio, excellent conductivity, and fast diffusion of electrolyte. The OER kinetics is well explained by the Tafel plots extracted from the linear region of LSV curves. From Figure 4b, it is obvious that RuO2 exhibited a lower Tafel value of 55 mVdec -1 compared to our Co3O4 hybrid material 96 mVdec -1 . The proposed hybrid material is nonprecious, and the obtained Tafel slope is still low and acceptable which highlight the favorable OER kinetics. The Tafel slope value of Co3O4 hybrid material is comparable to the recently reported results [START_REF] Asano | Capacitance studies of cobalt compound nanowires prepared via electrodeposition[END_REF]. The pristine Co3O4 possess a Tafel slope of 103 mVdec -1 which is bit higher compared to the Co3O4 composite material. This illustrates that the PVP nanofibers improved OER kinetics of Co3O4 by providing large surface area and improved the conductivity of Co3O4 through carbonization. The Tafel plotting is used to understand the limiting mechanism of the electrochemical process. The OER kinetics on the surface of Co3O4 composite, systems were evaluated by Tafel plots under alkaline conditions. The Co3O4 composite material exhibits a low Tafel slope as compared to pristine Co3O4 and PVP. The OER mechanism is not well understood in the literature, however, it involves the 4electron transfer process, thus considered kinetically sluggish. Generally, the Tafel value of 120 mVdec -1 indicates the limiting path involving a first electron transfer step, the slope of 60 mVdec - 1 is revealing that kinetics is controlled by the adsorption, whereas the Tafel slope of 40 mVdec -1 is involving the second electron transfer step [START_REF] Wen | A novel solution combustion synthesis of cobalt oxide nanoparticles as negative-electrode materials for lithium-ion batteries[END_REF][START_REF] Manish | Preparation of electrospun Co3O4 nanofibers as electrode material for high performance asymmetric supercapacitors[END_REF][START_REF] Suryanto | Overall electrochemical splitting of water at the heterogeneous interface of nickel and iron oxide[END_REF]. The Tafel slope of Co3O4 hybrid material is suggesting the OER kinetics followed by the chemical step. The long-term usability of nonprecious catalyst is a big challenge and we investigated it at constant current density.

The durability experiment was performed on the Co3O4 composite material using chronopotentiometry test at a constant current density of 20 mAcm -2 for 20 h as shown in Figure 4c. The durability test has verified the stationary state of the catalyst without any drop of the potential; however, some fluctuation could be seen due to the bubbling of oxygen gas. The negligible loss of potential at constant current density suggests that the Co3O4 composite material has high potential for practical applications especially in water electrolyzer. To support the durability results, we measured the LSV curves before and after the durability test and observed no decay in the onset potential as shown in Figure 4d. The stability experiment indicates that the composite material has maintained the overpotential even after the durability test for 20 h.

It has been believed that the activity of a catalytic material is linearly related to the density of active sites and the intrinsic characteristic of individual active site. For this purpose, the electrochemical double layer capacitance (Cdl) which is inherently associated to the electrochemical active surface area (ECSA) was quantified by the CV study at different scan rates in non-Faradic region as shown in Figure 5 [START_REF] Jang | Electrospun RuO2-Co3O4 hybrid nanotubes for enhanced electrocatalytic activity[END_REF]. It can be seen from Figure 5c the Cdl of pristine Co3O4 is 11.9 µF cm -2 and Co3O4/PVP is 26.4.9 µF cm -2 . It is obvious that the carbonized Co3O4 nanoparticles exhibit a double value of Cdl and it suggests that the more density of catalytic sites are produced in the composite material and it highlights the scientific reason for the improved OER performance.

To access the deeper insight into the improved catalytic performance of Co3O4 composite material, the electrochemical impedance spectra were recorded for the frequency 100 kHz to 1Hz at OER at a potential of 1.51 V versus RHE as shown in Figure 6. Figure 6a, b shows the Bode plots which indicate the low resistance in the film of Co3O4 composite compared to the pristine Co3O4. Figure 6c shows the Nyquist plots for the Co3O4 composite and pristine Co3O4 which again confirms the low charge transfer offered by the Co3O4 hybrid material during the OER process and these observations supports the claims made on the LSV results. The simulated impedance results for the circuit elements are given in Table 1. The Co3O4 hybrid material bears a low charge transfer resistance of 29.23 Ohms which is three times lower than the pristine Co3O4 94.98 Ohms, thus it explains the reason for the outperform performance of Co3O4 hybrid material. The results of impedance are strongly supported by the reported studies [START_REF] Mugheri | An advanced and efficient Co 3 O 4/C nanocomposite for the oxygen evolution reaction in alkaline media[END_REF][START_REF] Anantharaj | Evolution of layered double hydroxides (LDH) as highperformance water oxidation electrocatalysts: A review with insights on structure, activity, and mechanism[END_REF]. The capacitance double layer value of 1.11 mF was also found for the Co3O4 hybrid material which is higher than the pristine Co3O4 0.68 mF. The OER performance of Co3O4 composite was also compared with already reported OER catalysts as given in Table 2. The OER activity of Co3O4 composite material is better in terms of low overpotential compared to the published results.

Conclusions

In summary, we have deposited Co3O4 nanoparticles on PVP fibers via electro-spun method followed by carbonization. The prepared materials of Co3O4 are well characterized by SEM, EDS and XRD. The high surface area and enhanced conductivity of Co3O4 composite due to carbonization have made an advancement in the design of efficient nonprecious electrocatalyst for OER in 1.0M KOH aqueous solution. The Co3O4 composite exhibited a low overpotential of 280 mV at 10 mAcm -2 . Also, they showed a Tafel slope of 96 mVdec -1 . Furthermore, EIS analysis described the low charge transfer resistance of 29.23 Ohms for the Co3O4 composite and it confirms the faster charge transport during electrochemical reaction. These results motivate that the carbonized Co3O4 nanoparticles are the potential materials for several applications particularly in the field of energy and environment sciences.

Acknowledgment

We extend our sincere appreciation to the Researchers Supporting Project number (RSP-2021/79) at King Saud University, Riyadh, Saudi Arabia. We also, would like to thank the platform "Microscopies, Microprobes and Metallography (3M)" (Institut Jean Lamour, IJL, Nancy, France) for access to SEM facilities.

Conflict of interest

Authors declare no conflict of interest in this study

Figure 2 .

 2 It is found that the nanofibers of PVP are successfully formed as shown in Figure 2c. They are decorated with the nanoparticles of Co3O4 and exhibit a dimension of several microns and an average diameter of 300-500 nm as shown in Figure 2b. The pristine Co3O4 possessed a plate like morphology as shown in Figure 2a. The XRD analysis was carried on the pristine Co3O4 and carbonized Co3O4 nanoparticles and the prominent diffraction patterns 111, 022, 113, 222, 004, 133, 224, 115, 044, 135, 244, 026, 335, and 226 were found as enclosed in Figure 2d. The XRD results of both pristine Co3O4 and carbonized Co3O4 nanoparticles are consistent with the reference card no. 96-900-5895. No another suspicious or impurities were identified by XRD measurement. The chemical composition analysis by EDS shows that a quantified amount of 4.3% of carbon is present in the composite samples which revealed the existence of carbon from PVP fibers as shown in Figure 3. An impurity of Sn was found which appeared from the fluorine doped tin oxide substrate on which analysis was carried out. The Co and O are the main elements in the sample as described in Figure 3. The carbonized Co3O4 nanoparticles have enhanced conductivity and surface area which favored an efficient OER kinetics. The OER activity of carbonized Co3O4 nanoparticles, bare PVP fibers, 20%RuO2/C and pristine Co3O4 was tested in 1.0 M KOH electrolyte using three electrodes set up as shown in Figure 4a. The LSV polarization curves were measured for the pristine Co3O4, carbonized Co3O4 nanoparticles, bare PVP and 20%RuO2/C. It is highly needed to design nonprecious OER catalysts with a low overpotential. The pristine Co3O4 nanoparticles are experienced by large over potential due to their poor electrical conductivity and a smaller number of catalytic sites. Pristine Co3O4

Figure 1 .

 1 Figure 1. Schematic diagram showing the synthesis process and electrochemical analysis

Figure 2 .Figure 3 .Figure 4 .

 234 Figure 2. a. pristine Co3O4, b. Co3O4/PVP, c. nanofiberes of PVP, d. XRD for pristine Co3O4 and Co3O4/PVP Figure 3. Elemental mapping and EDS spectra of Co3O4/PVP Figure 4. a. LSV curves at scan rate of 1 mV/s of pristine Co3O4, Co3O4/PVP, PVP fibers, 20%RuO2/C in 1.0M KOH, b. Tafel plots, c. chronopotentiometry experiment at 20 mAcm -2 for 20 h, d. stability before and after the durability test

Figure 5 .Figure 6 .

 56 Figure 5. (a-b) CV curves of pristine Co3O4, and Co3O4/PVP at different scan rates in 1.0M KOH, c. Linear fitting of current density versus scan rate for the calculation of Cdl of pristine Co3O4, and Co3O4/PVP Figure 6.EIS spectra for pristine Co3O4 and Co3O4/PVP, at 100 KHz to 1Hz, amplitude 5 mV and OER onset potential in 1.0M KOH, a.b. Bode plots, c. Corresponding Nyquist plots

Table 1 .

 1 EIS fitting values for the circuit elements

	Catalyst	Rct (Ohm)	Cdl (mF)
	Co3O4	94.98	0.68
	Co3O4 nanofibers	29.23	1.11

Table 2 .

 2 Comparison of presented Co3O4 nanoparticles deposited on PVP fibers with various published results.

	Catalyst	Overpotential	Tafel slope
		mV	mV dec -1


Thomas Gaillat
email: thomas.gaillat@univ-rennes2.fr

Andrew Simpkin
email: andrew.simpkin@insight-centre.org

Nicolas Ballier
email: nicolas.ballier@univ-paris.fr

Bernardo Stearns
email: bernardo.stearns@insight-centre.org

Annanda Sousa
email: annanda.sousa@insight-centre.org

Manon Bouyé
email: manon.bouye@etu.u-paris.fr

Manel Zarrouk
email: zarrouk@lipn.univ-paris13.fr

Predicting CEFR levels in learners of English: the use of microsystem criterial features in a machine learning approach

Keywords: microsystem, criterial features, supervised learning, language functions, Automatic Essay Scoring, linguistic complexity

This paper focuses on automatically assessing language proficiency levels according to linguistic complexity in learner English. We implement a supervised learning approach as part of an Automatic Essay Scoring system. The objective is to uncover Common European Framework of Reference (CEFR) criterial features in writings by learners of English as a foreign language. Our method relies on the concept of microsystems with features related to learner-specific linguistic systems in which several forms operate paradigmatically. Results on internal data show that different microsystems help classify writings from A1 to C2 levels (82% balanced accuracy). Overall results on external data show that a combination of lexical, syntactic, cohesive and accuracy features yields the most efficient classification across several corpora (59.2% balanced accuracy).

Introduction

Proficiency assessments are an essential requirement for language education centres both at individual and institutional levels. For individuals, learning a language requires regular assessments so that learners and teachers can focus on specific areas to train upon. For institutions, there is a growing demand to group learners homogeneously in order to set adequate teaching objectives and methods. The design and organisation of language assessment tests are labour-intensive and thus costly. In this context, automatic essay assessment may appear as a solution.

Automating assessment is conducted with Automatic Essay Scoring systems (AES). Initially grounded in rule-based approaches [START_REF] Page | The Use of the Computer in Analyzing Student Essays[END_REF], more modern systems rely on probabilistic models based on Natural Language Processing (NLP) tools exploiting learner corpora [START_REF] Meurers | Learner Corpora and Natural Language Processing[END_REF]. Some of these models depend on the identification of linguistic features used as predictors of writing quality. In L2 studies, features belong to three dimensions, i.e. Complexity, Accuracy and Fluency (CAF) (Housen et al., 2012;[START_REF] Ortega | Understanding Second Language Acquisition[END_REF][START_REF] Wolfe-Quintero | Second language development in writing: Measures of fluency, accuracy, & complexity[END_REF]. Some of these features operationalise complexity and act as criterial features in L2 language [START_REF] Hawkins | Criterial Features in L2 English: Specifying the Reference Levels of the Common European Framework[END_REF]. They help build computer models for error detection and automated assessment and, by using model explanation procedures, their significance and effect can be measured. Recent work on identifying criterial features has been fruitful, as many studies have addressed many types of features. However, to the best of our knowledge, few studies have tried to test features of several dimensions within a single model [START_REF] Tack | Human and Automated CEFRbased Grading of Short Answers[END_REF][START_REF] Volodina | Classification of Swedish learner essays by CEFR levels[END_REF] to investigate how they compare.

In addition, many of the developed models use features that quantify text items on the syntagmatic axis. For instance, the type-token ratio computes the number of tokens in relation to other elements of the syntagmatic chain. This approach relies on categorising linguistic forms distinctly without relating them to possible substitutes in the same position and with the same language function, thus ignoring the relationships that exist between forms on the paradigmatic axis. The way learners select forms of a specific function is not captured in current feature collection methods. Form variations of a given linguistic function [START_REF] Ellis | The Study of Second Language Acquisition[END_REF] need to be accounted for and a solution may be found in operationalising the notion of microsystem [START_REF] Gentilhomme | Micro-systèmes linguistiques et langagiers: Fonctions heuristiques et didactiques. Introduction méthodologique[END_REF][START_REF] Py | Les données et leur rôle dans l'acquisition d'une langue non maternelle. Les Carnets du Cediscor[END_REF].

Our proposal is to use a machine learning approach to test criterial features of many dimensions within a single model. The purpose is to provide answers on their respective importance. We also test new functional features that capture functional variations within single linguistic microsystems.

Theoretical background 2.1 A multidimensional set of 'criterial features'

Initiated with the Threshold project [START_REF] Ek | Threshold 1990[END_REF] and increasingly active in recent years, research on criterial features has focused on linking linguistic properties to L2 proficiency and to the levels of the Common European Framework of Reference for languages (CEFR). However, since the CEFR descriptors used by examiners are not explicitly linked to any linguistic properties at any of the six levels, the research on criterial features aims at identifying these properties [START_REF] Hawkins | Criterial Features in Learner Corpora: Theory and Illustrations[END_REF].

Among the three components of L2, complexity includes absolute, linguistic complexity which focuses on quantitative features, i.e. "the number of discrete components that a language feature or a language system consists of, and as the number of connections between the different components" (Housen et al., 2012, p. 24). The two authors further divide linguistic complexity into system and structure complexity.

There are two main approaches in the identification of criterial linguistic features for proficiency. The first one falls into the structure category endorsed by projects like the English Profile project [START_REF] O'keeffe | The English Grammar Profile of learner competence: Methodology and key findings[END_REF] or the Global Scale of English project [START_REF] De | Alignment of the Global Scale of English to other scales: The concordance between PTE Academic, IELTS, and TOEFL[END_REF]. Relying on quantitative methods applied to learner corpora (including errors), specific grammatical or lexical forms and syntactic patterns have been mapped to specific CEFR levels, forming the original definition of criterial features. The second approach falls into the systemic category of complexity as it focuses on the learners' L2 system as a whole. It relies on global measurements in texts and provides information on the range, size, and variety of different forms and structures. The literature abounds with such metrics, starting with the ubiquitous Type Token Ratio (TTR). With the advent of computational methods applied to learner corpora [START_REF] Granger | Integrating learner corpora and natural language processing: A crucial step towards reconciling technological sophistication and pedagogical effectiveness1[END_REF], many types of system complexity metrics have been put to the test as criterial features.

The first group of metrics includes lexical complexity metrics. These measures are based on word counts, lexicons and reference corpora. They were tested as predictive features of learner levels in terms of usage and properties [START_REF] Crossley | Predicting lexical proficiency in language learner texts using computational indices[END_REF]Lu 2012).

The second group of measures corresponds to syntactic complexity. By applying pattern extraction, phrases of different types are detected and counted, giving insight in terms of properties and usage [START_REF] Lu | Automatic analysis of syntactic complexity in second language writing[END_REF][START_REF] Chen | Computing and Evaluating Syntactic Complexity Features for Automated Scoring of Spontaneous Non-native Speech[END_REF][START_REF] Khushik | Investigating Syntactic Complexity in EFL Learners' Writing across Common European Framework of Reference Levels A1, A2, and B1[END_REF][START_REF] Lan | Does L2 writing proficiency influence noun phrase complexity? A case analysis of argumentative essays written by Chinese students in a first-year composition course[END_REF]. The results of the research showed that correlations exist between CEFR levels and certain features [START_REF] Lu | Automatic analysis of syntactic complexity in second language writing[END_REF][START_REF] Lu | Computational Methods for Corpus Annotation and Analysis[END_REF].

Semantic and pragmatic features were also tested in studies including cohesion [START_REF] Crossley | The tool for the automatic analysis of text cohesion (TAACO): Automatic assessment of local, global, and text cohesion[END_REF][START_REF] Crossley | Predicting second language writing proficiency: The role of cohesion, readability, and lexical difficulty[END_REF] and semantic measurements based on reference corpora (Kyle & Crossley, 2014). Errors, or negative properties of interlanguage, were also tested. [START_REF] Ballier | A Supervised Learning Model for the Automatic Assessment of Language Levels Based on Learner Errors[END_REF] showed that error-tag frequencies could be used as potential proficiency predictors.

As studies became more elaborate, the question of the relative importance of features of all dimensions was raised. Some tools have been developed for the creation of complexity metrics datasets of various dimensions [START_REF] Chen | CTAP: A Web-Based Tool Supporting Automatic Complexity Analysis[END_REF]. Syntactic and lexical complexity metrics were combined [START_REF] Arnold | Predicting CEFR levels in learner English on the basis of metrics and full texts[END_REF][START_REF] Ballier | Classifying French learners of English with written-based lexical and complexity metrics[END_REF] as well as semantic measures [START_REF] Venant | Towards the Prediction of Semantic Complexity Based on Concept Graphs[END_REF]. Some experimental designs also combined syntactic, lexical, discourse and error features in the form of metrics [START_REF] Vajjala | Automated assessment of non-native learner essays: Investigating the role of linguistic features[END_REF] or properties such as POS and n-grams [START_REF] Garner | N-gram measures and L2 writing proficiency[END_REF][START_REF] Yannakoudakis | A New Dataset and Method for Automatically Grading ESOL Texts[END_REF] or edit distance between erroneous segments and their corresponding target hypothesis [START_REF] Tono | Automatic extraction of L2 criterial lexicogrammatical features across pseudo-longitudinal learner corpora: using edit distance and variability-based neighbour clustering[END_REF]. All these efforts bore their fruits for the research community and learner data challenges (the ACL Building Educational Applications workshop series) helped fostering techniques and modelling beyond the learner corpus research community. For example, a shared task was organised at the CAp18 conference on Artificial Intelligence in France. A dataset including lexical, readability and syntactic complexity metrics was provided to competitors to predict CEFR levels of French L1 writings in English. Competitors added other features such as ngrams and spelling errors to compute their models [START_REF] Ballier | Machine learning for learner English[END_REF].

The results of all these studies show that, in spite of their benefits, other complexity measures are required for the characterisation of proficiency levels. Since the CEFR adopts a functional approach, a line of investigation might reside in identifying system metrics that also inform on specific functional structures as pointed out by [START_REF] Biber | Investigating grammatical complexity in L2 English writing research: Linguistic description versus predictive measurement[END_REF] . One way of approaching the issue could be through the notion of microsystems.

Microsystems in learners

Microsystems are part of the structure complexity construct. They tap into functional complexity because they are composed of several constructions grouped according to functional proximity. Microsystems can be defined as families of competing constructions in a single paradigm. First introduced by [START_REF] Gentilhomme | Micro-systèmes linguistiques et langagiers: Fonctions heuristiques et didactiques. Introduction méthodologique[END_REF] with personal pronouns in native French, the notion was cross-examined with that of Interlanguage [START_REF] Py | Quelques réflexions sur la notion d'interlangue[END_REF]. Py argued that a microsystem makes it possible to view language as an unstable equilibrium. Interlanguage microsystems take several shapes, including that of autonomous sets of elements. Gentilhomme (1980) describes learner microsystems as unexpected uses of forms which are evidence of systemic acquisitional processes. Learners develop microsystems which are unstable and transitory in nature [START_REF] Py | Didactique des langues étrangères et recherche sur l'acquisition. Les conditions d'un dialogue[END_REF]. In terms of syntax, it is possible to illustrate this process with the paradigmatic interactions between forms of the same linguistic function but of different semantic implications.

The article microsystem composed of a, the or Ø ("zero article") can provide a base for illustrating this view. For a description of Ø, see for instance [START_REF] Depraetere | Advanced English grammar: A linguistic approach[END_REF]. Let examples (1), ( 2) and (3) contrast the uses of the in three samples from the EFCAMDAT corpus [START_REF] Geertzen | Automatic Linguistic Annotation of Large Scale L2 Databases: The EF-Cambridge Open Language Database[END_REF].

(1) "Ladies and Gentlemans, My flat was robbed the previous evening. In coming back at my home, I saw that the window was broken." (EFCAMDAT writing ID: 2498) (2) "What do you think about positive discrimination in the companies?" (EFCAMDAT writing ID: 569744) (3) "Why the gender's discrimination is still a problem in our society?" (EFCAMDAT writing ID: 579779) The use of the article might be expected in (1) due to the associative anaphora linking flat and window. However, the is unexpected in (2) and (3) due to misunderstandings of the generic values of companies and gender's discrimination. In examples (2) and (3), Ø is in paradigmatic competition with the (Depraetere & Langford, 2012, pp. 91-93). Learners use articles with variability, which constitutes an unstable microsystem. As learners use forms and constructions to perform certain speech acts linked to specific language functions, microsystems can be seen as an attempt to operationalise systematic form-function variations (Ellis, 1994, p. 135). Evidence of this process has been examined through the use of it, this and that in [START_REF] Gaillat | Reference in Interlanguage: The case of this and that[END_REF].

To capture the variability within microsystems, our proposal is to create metrics that measure the importance of each construction in relation to its counterparts within a given text. Single measures could thus encapsulate the internal variations of multi-variable microsystems. This approach would bridge the gap between structure and system complexity. Microsystem metrics offer an insight into the evolution of linguistic functions at systemic level across categories such as articles, modal auxiliaries, tenses and nouns. We take these grammatical areas to be representative of potential interlanguage grammar rules in construction and analyse written productions through these lenses of microsystems.

To the best of our knowledge, the literature on criterial features does not include heuristics based on microsystems, nor does it report many studies testing many metrics as criterial features of many dimensions. Our approach includes the definition of some microsystems which are used for specific language functions such as determination or the expression of modal possibility. Our experimental design exploits machine learning algorithms to classify learner writings with many types of metrics including specifically-designed microsystem metrics.

Our research aims are (i) to assess many complexity metrics as potential criterial features [START_REF] Hawkins | Criterial Features in L2 English: Specifying the Reference Levels of the Common European Framework[END_REF] and (ii) to investigate the significance of microsystem metrics as criterial features within the broad spectrum of complexity metrics.

Methods

Corpora

The data used for modeling and measuring the correlation between learner levels and microsystems consists of the Spanish and French L1 subsets of the Education First-Cambridge Open Language Database (EFCAMDAT), an 83-million-word corpus collected and made available by Cambridge University and its partner, the organization Education First. This corpus is made up of learner writings in English and rated by humans. It was annotated with metadata such as learner level, nationality but also, for some texts, errors and part-of-speech tagging. The levels which were assigned to learners are based on the levels from EF's online school, Englishtown, with ratings ranging from 1 to 16. Learner levels thus had to be mapped onto CEFR levels. Levels 1-3 correspond to the A1 level and level 16 to the C2 level, as indicated in [START_REF] Geertzen | Automatic Linguistic Annotation of Large Scale L2 Databases: The EF-Cambridge Open Language Database[END_REF]. Data was selected and manipulated independently of the participation of the Cambridge and Education First research teams.

In our study, 49,817 texts written by 8,851 French and Spanish learners were downloaded from the database. This textual data runs across all Englishtown writing topics and CEFR levels. Tables 1 and2 give the breakdown for each L1. To test the validity of our models on external data, we used the CEFR ASAG corpus [START_REF] Tack | Human and Automated CEFRbased Grading of Short Answers[END_REF], a collection of short answers to open-ended questions, written by French L1 learners of English and graded with CEFR levels. It consists of 712 texts written by different learners in response to three questions. We used a balanced sample of 299 texts.

Features

We created new functional metrics based on the notion of microsystems (see Section 2.2). We assume that microsystems are sets of competing constructions (some being more likely for natives, others more prone to be L1-like). Based on intuition, Table 3 provides a list of other potential functional microsystems identified by two expert English teachers and linguists. For instance the nominal microsystem includes three constructions which learners find difficult.

They may use genitive constructions instead of noun+preposition+noun or compound noun constructions. Similarly, other substitutions may be observed among the can, may, might, could modals used to express epistemic and radical possibility. Regarding that, it has been noticed that confusions occur between the relativizer forms. We also specified a type of error linked to the confusion between the relativizer and complementizer functions. Note: Relative pronoun 0 is not included in the operationalisation of the program as the detection of the non-existent tokens remains an obstacle

The microsystems include variability in grammaticality: some of the substitutions among the aforementioned constructions are just semantic differences in the case of modal auxiliaries, others jeopardise grammaticality (which versus who for animate antecedents). The weighting of the parameters of these different constructions is beyond the scope of this paper.

Finding a method to quantify variability in microsystems at text level could help measuring the importance of specific linguistic functions in L2 systems. To operationalise microsystems, we added a set of metrics relying on paradigmatic relations between forms of similar functions, i.e. microsystem variables as defined in Table 3. For each microsystem (e.g., "modals for possibility"), the frequency of occurrence of each variable (e.g., "may") in this microsystem was computed within each text (see Eq. 1a). In addition, a ratio was computed for each variable relative to all variables of the microsystem (see Eq. 1b). The absolute and relative microsystem features were computed as follows:

(1a) (1b) where = the microsystem = the total number of variables in microsystem x = the i-th variable in the set of n variables = the j-th text (learner writing) = the frequency of occurrence of variable i in text j

The microsystem ratios reflect the variations in the proportions of one variable over its paradigmatic competitors. Microsystem features are computed within each writing separately. The L2 Syntactic Complexity Analyzer (L2SCA) tool [START_REF] Lu | Automatic analysis of syntactic complexity in second language writing[END_REF] was modified in order to capture specific linguistic forms belonging to specific microsystems. The program proceeds in two stages. Firstly, it extracts the constructions used in the microsystems and, secondly, it calculates ratios which operationalise the microsystems. The Tregex module of Stanford CoreNLP (Manning 2014) was used to retrieve constructions including nouns, modal auxiliaries, articles, proforms, relativizers and complementizers. For illustration's sake, we focus on the microsystem of proforms. The Penn Treebank tagset used for the program does not have a specific tag for proforms, so that the this proforms were retrieved with the following Tregex patterns:

prf_this1='DT=n1 </[t|T]his/ & >-/NP.*/' (1) prf_this2='/[t|T]his/ > NN=n1' (2)
Pattern (1) identifies all this that are tagged as DT (determiner) and that are the rightmost descendents of Noun Phrase (NP) constituents. Pattern (2) identifies all this immediately dominated by a Noun (NN).

The evaluation of the extractions of all the forms specified in microsystems is outside the scope of this paper. Nevertheless, it must be mentioned that most forms are captured with patterns relying on their POS tags (See Appendix 1). It may be argued that evaluating their extraction relates to evaluating POS tagging in learner corpora (accuracy results above 95%). Several papers have established a high level of accuracy in POS tagging learner English (see [START_REF] Huang | Dependency parsing of learner English[END_REF][START_REF] Van Rooy | An evaluation of three POS taggers for the tagging of the Tswana Learner English Corpus[END_REF]. The analysis of proforms is not based on the identification of the tag and previous works support its reliability (Gaillat, 2016, pp. 183-196). The extraction of this forms was evaluated by applying distinctive patterns on 2,853 occurrences in the Wall Street Journal subset of the Penn Treebank corpus [START_REF] Marcus | Building a Large Annotated Corpus of English: The Penn Treebank[END_REF]. All this proforms were accounted for.

As a result of the extraction process, 51 constructions were incorporated as variables in 29 microsystem metrics (See Appendix 1 for a list of microsystem metrics, their variables and Tregex extraction patterns). The modified version of L2SCA is called L2SCA_microsystem 1 . It also includes the same indices as L2SCA.

In addition to these microsystem features, several other types were extracted and used to compute metrics. The feature types encompass lexical, syntactic, semantic and discourse 1

Available from the project's website: http://www.clillac-arp.univ-paris-diderot.fr/projets/ulysse2019 complexity as well as accuracy. See Appendix 2 for a list of all the implemented metrics and the tools used to compute them. In total, 767 different features were extracted and merged into one dataset to input into the classification models.

Statistical analysis

There were three aims in this statistical analysis.

1. Test the utility of the novel microsystem features over existing features 2. Compare feature importance 3. Build a prediction model for future learners

We implemented this analysis through a Machine Learning (ML) approach. In principle, an ML analysis relies on observations recorded in a computer model. In our experiments, the observations are made up of the features of the texts linked to their CEFR levels, and their statistical relationships are computed by applying a specific mathematical function, i.e. a model. The model is subsequently used to predict CEFR levels in new observations of features. The analysis performed for each of the three aforementioned aims is summarised below. Analysis (see Code in Appendix 3) was performed using R v3.6 through the {glmnet} [START_REF] Friedman | Regularization Paths for Generalized Linear Models via Coordinate Descent[END_REF] and {caret} packages [START_REF] Kuhn | Building Predictive Models in R Using the caret Package[END_REF].

Testing the utility of the microsystem features

In order to test the efficacy of our novel microsystem variables, we built three classification models: (i) using 687 features from previous research as explained in Section 4.2 as a baseline, (ii) adding the 51 microsystem variables introduced in this paper along with 29 microsystem ratios and (iii) adding the 51 microsystem variables introduced in this paper along with 12 interactions (see Appendix 3) involving variables of the same microsystems.

Using dataset (i) we compared multinomial logistic regression, ensemble random forests, linear discriminant analysis, k-nearest neighbours, Gaussian naive Bayes, support vector machine and decision tree classifier. We found the optimal classification model for (i) and applied this model to each set of features (ii) and (iii). We report on the precision, recall, F1score (F1 = harmonic mean of precision and recall, i.e. 2 × precision ×recall precision+recall ), and balanced accuracy (Balanced Accuracy = average of sensitivity and specificity, i.e. sensitivity +specificity 2 ) of each model. Results are presented for each of the six learner classes, and overall by micro-averaging over the classes to take account of different class sizes. Models were run using five-fold cross-validation to allow for testing with multiple random splits of the data. After running these models, results were macro-averaged across cross-validation folds.

Once the model is used to predict learner level in the test set, we perform an error analysis. We define the error group as a three-level categorical variable, i.e. 0 if classification is correct, 1 if classification is one level lower/higher, 2 if classification is two or more levels lower or higher. A one-way analysis of variance is then used to test whether there are mean differences in each feature according to the error group, adjusting for multiple testing across 767 total features, and taking only those p-values of < 0.05/767 to be statistically significant.

Comparing microsystem feature importance

A second analysis used multivariable logistic regression, a classifying method for categorical data, to investigate the relative importance of the 51 new microsystem variables and their 29 ratios across learner levels. We split the data based on learner levels (A, B and C) and ran separate logistic regressions on these data using only the microsystem variables. We report on the strongest positive and negative associated features in terms of their Wald test statistic or zscore for each level, i.e. A2 v A1, B2 v B1 and C2 v C1. A positive association suggests the feature is more common in advanced learners, a negative association suggests the feature is less common in advanced learners. We report on the odds ratios of the features to explore how much the use of a feature increases the odds of being an advanced learner.

Building a classification model for future learners

While the optimal model found using all features in 4.5.1 will allow classification of future learners, using over 700 features will also likely overfit to the EFCAMDAT sample data. Therefore we employed a feature selection algorithm, in particular elastic net regression (Zou & Hastie, 2005), which conducts dimension reduction and prediction simultaneously. Elastic Net Regression is a useful classifying method for modelling the relationship between a binary response variable Y and a large number of potential features X 1 , ⋯, X P . The regression model used is

ln ( π i 1 -π i ) =β 0 + β 1 x 1 +⋯+ β p x p + ϵ
where π=P (Y =1 ), β 0 , β 1 ,⋯ β p are regression coefficients and i=1 , ⋯, n observations are available. In cases where the number of predictors P is bigger than n, some form of model selection or dimension reduction is required. Penalized regression is one such tool which shrinks the coefficients β 0 , β 1 ,⋯ β p with several types of penalty available. The elastic net combines two common penalized regression approaches, i) the least absolute shrinkage and selection operator (LASSO) [START_REF] Tibshirani | Regression Shrinkage and Selection Via the Lasso[END_REF] and ii) ridge regression [START_REF] Hoerl | Ridge Regression: Biased Estimation for Nonorthogonal Problems[END_REF]. This is useful because the LASSO allows for automatic feature selection by shrinking coefficients of some variables to 0, while the ridge regression penalty excels where features are heavily correlated--which is likely the case for linguistic features.

Five fold cross-validation was used to repeatedly test performance across multiple splits of the data. The performance metrics -precision, recall, F1 score and balanced accuracy were calculated in each fold and summarised using their macro-average (i.e. simply taking the average of the five precision, recall, F1 and balanced accuracy metrics) and standard deviation.

Data for evaluation

To evaluate the models we applied a twofold strategy. Firstly, we used a subset of the EFCAMDAT dataset as an internal test set and, secondly, we used the CEFR-ASAG external dataset to test the validity of the model and its resistance to overfitting. We used this corpus as it was made up of small writings and was challenging for the dataset on which we had trained our model. The mean of ASAG texts was 157.62 tokens per writing (SD: 81.66) distributed over the six levels, a value typically associated to A1 in our data. Whereas our corpus is heavily biased towards A1, the ASAG corpus has a majority of B1 writings.

The internal test set was sourced randomly from 25% of the EFCAMDAT dataset, resulting in 12,454 texts. Among the seven model types tested the optimal classification performance in the testing dataset was found using multinomial logistic regression.

The external test set was made up of 299 short texts. It was built with the same feature extraction program described in Section 4.3 ran on the CEFR-ASAG corpus texts. Firstly, the optimal classification model from (i) was used to classify with all the features as in (i) and (ii) (see Section 4.4.1). Secondly, following Ockham's razor principle and to avoid overfitting (capturing non-generalizable features), an elastic net method was applied including feature dimensionality reduction.

Results and feature analysis 4.1 Testing the utility of the microsystem features

Classification of all six CEFR levels

Among the seven model types tested, the optimal classification performance in the testing dataset was found using multinomial logistic regression. The classifier using previously developed features achieved 80% balanced accuracy. Using the additional microsystem variables, along with their ratios increased performance to 82%, which translated to an extra 249/12,454 writings correctly classified. Full results are given in four tables in Appendix 4. It includes classification performance, the confusion matrix and detailed comparisons with and without microsystem features. One comment about confusions is that they mostly occur with adjacent classes. A closer examination shows that many writings tend to be classified in the lower adjacent class. Note that the appendix only includes one of multiple confusion matrices from Cross-Validation (CV).

We performed an error analysis in those 12,454 test essays -10,159 were correctly classified, 1,865 misclassified one level higher/lower error, and 430 misclassified two or more levels higher/lower error. From ANOVA, 469 out of 767 features show mean differences between these two groups, indicating which of the features are associated with errors. The top ten of these are shown in Table 3 in Appendix 4.

Comparing microsystem feature importance

The second analysis of the internal testing protocol relied on the logistic regression model and aimed at investigating the relative importance of microsystem variables across the aggregated A, B and C CEFR levels. We measured the impact of microsystem features in each level. There are two types of features. Figures 1, 2 and 3 (available as supplementary material) show features that indicate occurrences of specific variables and others (with the MS prefix) that show microsystems composed of specific variables. The figures show the strongest features of each level in terms of z-score.

Results regarding the A level (Figure 1) reveal four significant microsystems. Nominal constructions (i.e. prepositional, genitive and compound constructions) relative to each other appear to be significant predictors of the A2 level as opposed to the A1 level. The obligation microsystem composed of modals have to and must also appears as a significant predictor of A2. Likewise, the duration microsystem (based on for and ago) as well as the quantification microsystem (based on quantifiers much, most and many) both show preference for A2 rather than A1 writings. As the microsystems implement forms of a specific language function, these results may indicate that writings are likely to implement the nominal, obligation, duration and quantification functions as a first step in their progress. Even more so as A1 tasks are mostly with the present tense, so that for/since/ago is probably not tested at this stage.

Results (Figure 2) show that the B level is influenced by two microsystems. The determination microsystem tends to be indicative of the B1 level. The quantification microsystem with most and many appears to be indicative of the B1 level too. This trend is to be compared with that of the A level, in which the quantification microsystem is favoured in A2. The level adjacency may indicate that the quantification language function appears and consolidates between A2 and B1 levels. In functional terms, B learners seem to be developing their proficiency by implementing determination and quantification language functions. The B2 level tends to appear as these microsystems stabilise in terms of variable proportions.

For level C writings (Figure 3), the proform microsystem and several specific constructions appear to be significant. The proform microsystem tends to predict C1 as learners overuse this compared with it and that, whereas the microsystem tends to predict C2 as learners increase the relative importance of that. This microsystem suggests the onset of anaphoric and deictic reference processes, which corresponds to more complex discourse. With higher discourse complexity, learners tend to increase their use of referential expressions leading to variability in the proform microsystem. The modals should and will also appear to be significant. This may indicate more elaborate discourse in writing as learners diversify their stance in terms of epistemic or radical modality.

Building a classification model for future learners

Logistic Regression model for classification using all features

In order to test the validity of the logistic regression model trained on the EFCAMDAT dataset, the same model was used to classify a dataset built from the CEFR-ASAG corpus. Classification according to the 6 CEFR levels showed poor results with 51% balanced accuracy in the ASAG data.

There are several reasons for the loss in balanced accuracy between the two datasets. Firstly, performance in test data randomly taken from the training data is always optimistic, because the test and train sets are very similar. Conversely, the CEFR-ASAG corpus corresponds to shorter contexts and different tasks than the EFCAMDAT corpus. Secondly, the ASAG data have few A1 writings (~16%), while the EFCAMDAT has approximately 40%. This lack of calibration between class populations is not reflected in the model, leading to errors.

Elastic net modelling EFCAMDAT data with feature selection

To limit overfitting and improve classification on external data, we used an elastic net regression model on the EFCAMDAT training set. This method is a classifying algorithm which comes with the benefit of including feature dimensionality reduction, i.e. feature selection. The elastic net model fitted in 178 minutes using a Macbook Pro with 8GB of memory. Using just 44 features classification showed 75.0% balanced accuracy (CI [74.3, 75.8], p < 0.001) and 59.2% (CI [53.4,64.8], p < 0.001) on the EFCAMDAT and CEFR-ASAG test sets respectively (see tables in Appendix 4 part B). Compared with the logistic regression model, the elastic net regression model showed lower performance on the EFCAMDAT test set but, most importantly, it improved performance on the CEFR-ASAG test set showing context adaptability.

The elastic net modelling method combines regression with feature selection. In other terms, it employs methods to not only compute best fit for all data points but to also remove non-significant features. In doing so, it combines the smallest set of features for the best classification. In the EFCAMDAT regression model, 44 features are combined. The features encompass several linguistic dimensions. Table 4 shows how features are distributed according to their linguistic dimensions. Among the microsystem features presented in Section 3, the proform microsystem based on that appears to be significant when combined with other lexical, syntactic, accuracy and pragmatic features. The modal ought to, in its raw frequency, is conjointly significant with the other features. This suggests that sophisticated grammatical markers could be used as criterial features for lexical sophistication.

Discussion

The performance in classification of the logistic regression and the elastic net models shows comparable results with those obtained in other studies applying L2 English proficiency classification. To the best of our knowledge, all studies use test sets extracted from the same corpora as their training sets. Likewise, we tested our models internally and best results showed 82% balanced accuracy on the 6-point CEFR scale with a logistic regression model. We even obtained 95% balanced accuracy on a 2 beginner-and-advanced scale, which can be useful for large scale automated groupings of students above and below the B1/B2 border. In comparison, [START_REF] Vajjala | Automated assessment of non-native learner essays: Investigating the role of linguistic features[END_REF] reported 73.2% balanced accuracy on a TOEFL subset categorized according to a 3-point scale. [START_REF] Crossley | Linguistic Microfeatures to Predict L2 Writing Proficiency: A Case Study in Automated Writing Evaluation[END_REF] reported 55% on another TOEFL subset on a 5-point scale, and [START_REF] Tack | Human and Automated CEFRbased Grading of Short Answers[END_REF] reported 53% balanced accuracy on the ASAG corpus with a 5-point scale.

Error analysis in the confusion matrix of the logistic regression model revealed a substantial number of errors between proficiency levels including non-adjacent class errors. Significant differences are mainly due to errors related to word frequencies and syntactic patterns (Complex Nominals and Verb Phrases). Regarding frequencies, some learners may have written an unexpected number of words for their level. Regarding syntactic patterns, the complex nominal (CN1) feature includes nouns plus adjective, possessive, prepositional phrase, relative clause, participle, or appositive. This broad variety of structures may create noise in the model. For instance, learners of different levels may use the relative clause structure leading to ambiguities in classification.

Compared with the logistic regression presented in this paper, all the aforementioned studies showed the advantage of limiting the number of features and increasing their potential for generalization. Our logistic regression model relies on a large array of features, which makes it prone to overfitting. After reducing dimensionality with the elastic net method presented in this paper, the model classified 75% of the data correctly. This result compares well with the aforementioned performance rates.

In order to measure the potential for generalization of our models, we tested the trained models on external data. The logistic regression model showed signs of overfitting since the balanced accuracy on external data dropped from 81% to 51%. Conversely, the elastic net model showed a higher ability for generalisation with a 59.2% balanced accuracy on external data. These results show that external validation of models is a necessary step in order to assess the fit of a model and the significance of its features. This appears as an essential step to include in further studies, and it shows the importance of open access to data sources.

In terms of feature significance, our approach was twofold. The first research question was to assess a large array of complexity metrics as potential criterial features. Based on a dataset of 767 metrics and 49,817 observations, an elastic net method helped identify a limited set of significant features. It is important to stress that it is the combination of features that supports the results. In other terms, it would be incorrect to isolate each of the 44 features and give them independent significance. The feature selection showed that it was mostly lexical and syntactic features that supported best classification. These findings are in line with several studies [START_REF] Crossley | Predicting lexical proficiency in language learner texts using computational indices[END_REF]Kyle & Crossley, 2014;[START_REF] Lu | Computational Methods for Corpus Annotation and Analysis[END_REF][START_REF] Vajjala | Automated assessment of non-native learner essays: Investigating the role of linguistic features[END_REF].

A caveat is in order at this stage. The models were trained mainly on short texts, with a scarcity of data at specific CEFR levels. The models may be sensitive to variations due to differences in instruction tasks implying the use of some microsystems vs. others. Consequently, microsystems and other features may not be captured in sufficient numbers in some classes leading to unclear boundaries between classes.

The second research question was to investigate the significance of new microsystem metrics as criterial features. We tested these features as part of a multinomial logistic regression model. Each microsystem operationalises the paradigmatic relations of competing constructions in learners. The results show that microsystem features contribute to improving CEFR level prediction, albeit to a small extent. The results suggest a series of learning stages. The ratios of nominal constructions relating two nouns, the ratios of modals linked to obligation and the ratios of quantifiers all appear to be indicative of the A level. Concerning the B level, ratios of quantifiers including most, many, little and few, as well as ratios including determiners a, the and Ø, show significance. This suggests that learners introduce quantification between the A2 and B1 levels and that determination starts occurring in significant proportions at B1. The C level shows the proform microsystem as significant as well as specific modals such as should and will. As discourse complexifies, learners introduce language constructions with higher semantic complexity. Learners construct referential processes by including deictic and anaphoric constructions, and they increasingly take stances as they use deontic and epistemic modality devices. Some features may be subject to task effects e.g., the use of modal will in A1 (see Section 4.1.2).

In the context of language teaching, microsystem features might appear very informative. Microsystems contrast forms that compete with each other in the minds of learners. Using them could prove to be fruitful in iCALL systems providing formative feedback based on simple, clear, elaborated manageable units [START_REF] Shute | Focus on formative feedback[END_REF]. Microsystems are operationalised as simple limited sets of items which are clearly organised according to linguistic functions [START_REF] Biber | Investigating grammatical complexity in L2 English writing research: Linguistic description versus predictive measurement[END_REF]. They could be used to build automated feedback on specific language functions as Saricaoglu shows with causal explanations (Saricaoglu, 2019). In addition, the approach could augment the drive towards Data Driven Learning as the system feeds from a corpus to guide learning [START_REF] Boulton | Data-Driven Learning and Language Pedagogy[END_REF].

Conclusion

In this paper, we have reported a supervised learning approach for the classification of learner writings in English according to the six CEFR proficiency levels. Our hypothesis concerned the use of linguistic metrics in the determination of CEFR levels. Firstly, we assessed the significance of many complexity metrics as potential criterial features in proficiency. The models show that a combination of lexical, syntactic, accuracy and pragmatic features helps predict CEFR levels. Among all feature types, lexical and syntactic features appear to be very important. In this respect, frequency information extracted from reference corpora favours prediction. Unlike previous research, our study also provides additional external validation with the ASAG corpus. We tested the portability of the models across corpora with different topics and prompts and showed that some features help with model generalisation.

For the second research question, we investigated the significance of newly designed microsystem metrics as criterial features. These metrics are based on learner-specific paradigms including competing constructions. Specific functional features that function paradigmatically have proved to influence the perception of learner writing proficiency by human annotators. Analysis of the results suggests that some microsystems are connected to acquisitional stages operationalised in terms of levels. The study maps specific constructions to levels in functional terms.

Results are also encouraging as part of the development of an AES protype2 . The project includes an NLP pipeline built upon several state-of-the-art tools measuring lexical, semantic, syntactic, accuracy and pragmatic complexity. The system provides two services: CEFR level prediction and complexity metric extraction. It relies on the Docker technology which makes it deployable as a cloud service [START_REF] Sousa | From linguistic research projects to language technology platforms: A case study[END_REF].

Understanding foreign language acquisition is a long path which involves many dimensions. With experience, language teachers acquire these dimensions intuitively in order to assess and train their students. However, processing students' productions is slow and variable. The research presented here should be seen as a way to invent new tools to assist teachers who would benefit from easy-to-use analytical tools that objectivise the progress of their learners.

Table 1 .

 1 The EFCAMDAT French dataset

		A1	A2	B1	B2	C1	C2
	# of tokens	817,228	888,298	887,987	528,880	138,541	13,689
	# of types	561,688	581,317	571,193	320,973	80,722	8161
	# of writings 17,605	11,584	8,105	3,514	742	76
	Median	36	67	98	134	173	170.5

Table 2 .

 2 The EFCAMDAT Spanish dataset

		A1	A2	B1	B2	C1	C2
	# of tokens	125,500	163,668	228,710	185,094	64,534	5,954
	# of types	84,334	106,553	144,295	108,942	37,150	3,620
	# of writings	2,572	2,066	2,005	1,176	340	32
	Median	38	68	103	143	173	167.5

Table 3 .

 3 Learner microsystems and their variables.

	Microsystems	Function	variables
	Nominal constructs	Denomination	determiner genitive; noun-of/for-
			noun constructions, compound
			nouns
	Modals for	Possibility	may; can; might; could
	possibility
	Modals for	Obligation	must; have to
	obligation
	Proforms	Reference	it; this; that
	Articles	Determination	a; the; Ø
	Relativisers	Reference	that; which; who; 0
	Complementizer vs	Expressing hypotaxis	that
	relativizer
	Duration/start/date	Expressing time	For; since; ago; from; during
	Prepositional	Linking entities	For; to
	constructions

Quantifiers

Quantification (Neutral; large; small) Some vs any; many vs much vs most; few vs little

Table 4 .

 4 Combined features for best model classification with elastic net

	Dimension	Features	Tools
	variation Lexical	Token Ratio Lexical Variation; Noun Variation; Type	LCA
	n sophisticatio Lexical	COCA_lemma_academic_bi_MI ency COCA_Academic_Bigram_Lemma_Frequ Brysbaert_Concreteness_Combined_AW	TAALES
		Component_Brysbaert_Concreteness_Co
		mbined_AW
		Component_Brysbaert_Concreteness_Co
		mbined_CW
		Component_COCA_fiction_tri_2_MI
		Component_COCA_spoken_Frequency_C
		W
		Component_COCA_spoken_tri_2_DP
		Component_COCA_spoken_tri_2_MI
		Component_Freq_N_AW
		Component_MRC_Imageability_AW
		Component_MRC_Imageability_CW
		Component_SUBTLEXus_Freq_CW
		MRC_Concreteness_AW
		MRC_Familiarity_AW
		MRC_Imageability_AW
		eat_tokens_AW
	Syntactic complexity	MD_OUGHTTO MS_PRF_THAT	L2SCA microsyst em
		acad_av_delta_p_verb_cue_type
		acad_av_faith_verb_cue_type
		acad_av_lemma_freq
		acad_av_lemma_freq_type
		all_av_lemma_freq
		all_av_lemma_freq_type
		amod_nsubj_deps_struct
		conj_and_all_nominal_deps_NN_struct
		conj_and_nsubj_deps_struct
		fic_av_lemma_freq
		fic_av_lemma_freq_type	TAASC
		mag_av_lemma_freq
		mag_av_lemma_freq_type

See project's website for demo, data set and other resources: http://www.clillac-arp.univ-parisdiderot.fr/projets/ulysse2019

Acknowledgments

With the financial support of the French Ministry for Europe and Foreign Affairs and the French Ministry of Higher Education, Research and Innovation and the Irish Research Council as part of PHC Hubert Currien Ulysses 2019 (ref 43121RJ).

Supplementary material

Note: all appendices and figures are available for download from the IRIS database under the authors' names. See https://www.iris-database.org

Ethical statement

The authors declare no conflict of interest regarding the publication and their involvement in other roles. All the documented research was conducted according to the EU's GRDP act. This material is the authors' own original work. It has not been previously published elsewhere, nor is currently being considered for publication elsewhere. The authors present truthful and complete results of their work. All sources used are appropriately cited in the articles. All authors have been involved in substantial work for this paper, and will take public responsibility for its content.