Jean-Pierre Marguénaud

Corine Dauchez

Benjamin Dauchez

Undoubtedly, the judgment of the European Court of Human Rights (ECtHR) Ana Ionita v. Romania of March 21 st 2017 marked the beginning of the creation of a European status for notaries. The European judge quite clearly called the notary an "out-of-court magistrate". As a result, notaries are aligned with other judicial professions, yet their special status is recognised. This European recognition influences the notaries' right to freedom of expression which is at issue in this case, but it equally serves to underline the incongruity of the egregious "interprofessionality" swept in by the so-called "Macron's law", which implies that conformity to the European Convention of Human Rights is now called into question.

The close relationship between notaries 1 and judges is traditionally highlighted 2 , but it should be noted that, since the law of 25 Ventôse an XI, which established the organisation of modern notaries, their authority does not come from the judicial authority, as before, but by direct delegation of authority from the State 3 . However, the idea that the notary is an "out-of-court magistrate" is making a comeback, albeit in inverted commas to disguise its provocative audacity, due to an unusual digression by the post-Ceausescu Romania. In fact, the Ana Ionita v. Romania judgment of March 21 st 2017, now definitive, of the European Court of Human Rights (ECtHR), is of interest to notaries of all the member states of the Council of Europe as it has largely disregarded the specific context of that case.

At the beginning of 2006, a notary, Ms Ana Ionita, was subject to disciplinary sanctions for irregularities in her work and failure to pay professional tax. In July 2006, as she still had not paid the tax, she was suspended from her duties by order of the Ministry of Justice, until payment of the amounts due. After almost a month, she paid the sums claimed and she was reinstated. She was overwhelmed by a feeling of deep resentment for this painful professional experience. As a consequence, she put all her energy into making it known at both the judicial (by an unsuccessful legal action to invalidate the order of the Ministry of Justice) and public media level. This was not a good idea. In the heat of a televised debate, as she justified her intention to go on a hunger strike because of her disagreement with the professional tax levied by the UNNPR and the Chamber of Notaries of Bacau, of which she was a member, she made repeated personal attacks against the president of the Chamber of Notaries. She strongly accused him of being a member of the Nomenklatura, and guilty of all conceivable family, administrative and political abuses. After this embarrassing media showdown, at the beginning of 2007, the UNNPR ordered Ms Ionita's suspension from her duties as notary for a period of four months. After unsuccessful litigations before the national courts to challenge the decision, Ms Ana Ionita, whose wrath had not subsided, referred the matter to the ECtHR and complained that there had been a breach of her right to freedom of expression under Article 10 of the Convention. Acting unanimously, the fourth chamber of the ECtHR declared that her complaint was admissible, but after deliberation the ECtHR judged she was not victim of a breach of Article 10.

The ECtHR delivered a judgment in the Ana Ionita case regarding a conflict between the right to freedom of expression and the rights which, like the right to protect one's reputation4 , are guaranteed by Article 8 of the Convention, as are many judgments since that of the famous Grand Chamber of the ECtHR Von Hannover v. Germany (n°2) of 7 February 20125 . We could place this case in a long line of other cases. However, the main interest of the judgment would then be hidden, because the solution established in terms of conflict of fundamental rights in this very specific political case is determined by an unprecedented recognition of a European status for notaries.

The recognition of a European status for notaries (1) will be studied first and then its influence on relations between human rights and notaries (2).

The recognition of a European status for notaries

The notary already alerted the ECtHR to this situation in the Estima Jorge v. Portugal judgment of 21 April 1998 which stated the notarial deed is a writ of execution whose enforcement is, like a judgment, relevant to Article 6 §1 of the Convention. Compared to the other judicial professions, it was like a poor cousin without a real European professional status. The Ana Ionita v. Romania judgment seems to be appropriate to fill this gap by aligning the notary profession with other judicial professions (A) and, at the same time, recognising its specific features (B).

A. The alignment with other judicial professions

The specific status of the bailiff was brought to the attention of the ECtHR through the Pini and Bertani v. Romania judgment of 22 June 2004 6 . In fact, this judgment recognised that, in working to ensure the proper administration of justice, they represent a vital component of the rule of law which is based on supremacy of law and the principle of legal certainty and therefore, the State must take the necessary measures to allow them to carry out their assignment and to prevent the loss of the guaranties offered to the public during the judicial phase7 . Public confidence in the proper administration of justice, without which there is no real rule of law, is the implied criterion of the valuation of the bailiffs' profession. With regard to lawyers, there is no doubt, since the Casado Coca v. Spain case of 24 February 19948 , that they hold a special status which gives them a central position in the administration of justice as intermediaries between the public and the courts and means that they have a key role in ensuring public confidence in the courts. Explicitly taken into account by the judge in order to justify the special European status of the lawyers' profession, this public confidence, without which there is no rule of law or democratic society, relied even more on judges. This is the restatement of the Grand Chamber judgment in the Baka v. Hungary case of 23 June 20169 . Indeed, the Court reiterates10 that it has on many occasions emphasised the special role in society of the judiciary which, as the guarantor of justice, a fundamental value in a state governed by the rule of law, must enjoy public confidence if it is to be successful in carrying out its duties11 . It reiterates that, for this reason, judicial authorities, in their adjudicatory function, are required to exercise maximum discretion with regard to the cases with which they deal with in order to preserve their image as impartial judges. Like previous judgments, the Baka case attempts to reconcile the requirements inherent to this specific professional status with the right to freedom of expression in order to preserve this freedom for the judges.

The Ana Ionita case is also related to the exercise of the right to freedom of expression, so it is not surprising that it gave the Strasbourg Court the opportunity to make this statement about the professional status of notaries. Thus, in paragraph 47 of the judgment of March 21 st 2017, it asserts "… although they are independent professionals, notaries have official powers as a public authority granted by the State, such powers confer to the notarial deeds they draw up a guarantee of its authenticity, (so…) it could be necessary to protect the professional order from damaging attacks to preserve public confidence over them". It's important to note that the Court grants a key role to public confidence to recognise the status of the judicial profession. For the ECtHR, public confidence should be common to all judicial professions including notaries. Nonetheless, the original contribution of the judgment Ana Ionita v. Romania highlights the difference between the profession of public notary and the other judicial professions.

B. The specific nature of the notarial profession

In paragraph 47 and undoubtedly essential, the Court gives notification, closely linked with the requirements of the preservation of the public confidence in notaries, that rules for their conduct arise clearly from their specific role as "out-of-court magistrates", an expression in inverted commas, but explicit and completely reflecting their hybrid nature as independent professionals and public notaries. This reference to notaries as "out-of-court magistrates" marked a significant turning point in the jurisprudence of the ECtHR, compared to the previous decisions which had already ruled on the organization of the profession. These pertinent decisions were O.V.R. v. Russia of 3 April 2001 and National Notaries Chamber v. Albania of 6 May 2008 12 . Recognizing that the object of the Order of Notaries is to regulate and promote the profession, the ECtHR stated that it was outside the scope of Article 11 of the Convention, because, as established by legislation, it remains part of the structures of the State, in remarkable contrast to trade unions. As can be seen, these decisions were related more to the organization of the profession than to the status of the notary. They transferred to the Order of Notaries the solution relevant to all professional orders exercising important public law functions for the protection of the public and they avoided all alignment of the notary's status with judges' status. Now, specifically referred to as "out-of-court magistrate", by the European human rights judge, the notary has a different status from that of the lawyer, the bailiff and the judge. Unlike notaries, lawyers are not public officers, they do not have official powers as a public authority received from the State and no jurists would declare them judge of anything. The bailiffs are indeed public officers, but they only take action to enforce the judgment of the Courts in general and the notarial deeds in particular (both are enforceable titles). As "out-of-court magistrates", the notaries are less likely to be considered judges, compared in France to the administrative or judicial judges, but more likely than in the Court of Justice of the European Union (CJEU) which recently stated in its Leopoldine Gertraud Piringer judgment of 9 March 2017, and this should be likened to the Ana Ionita v. Romania judgment.

The CJEU has just stated, for the first time, that Article 56 of the Treaty on the Functioning of the European Union (TFEU), under which freedom to provide services is enshrined, doesn't preclude legislation of a Member State under which authentication of signatures appended to the instruments necessary for the creation or transfer of rights to property is reserved to notaries, and consequently excluding the possibility of recognition in that Member State of such authentication carried out, in accordance with his or her national law, by a lawyer established in another Member State [START_REF] Nourissat | Quand la Cour de justice reconnaît les activités réservées des notaires…[END_REF] . More specifically, the judgment stated that national law is definitively a restriction on the freedom to provide services 14 , but such a restriction is justified by an overriding reason in the public interest. As such, it first notes that "the land register is of crucial importance especially in certain Member States which operate a system of civil-law notaries, particularly in property transactions", and also "maintaining the land register thus constitutes an essential component of the preventive administration of justice in the sense that it seeks to ensure proper application of the law and legal certainty of documents concluded between individuals, which are matters coming within the scope of the tasks and responsibilities of the State". Second, referring to the terms of its famous judgment of 24 May 2011 15 , the court notes that reservation of authentication of signatures appended to the instruments necessary filed at the land registry serves the public interest, in the same way notarial activities pursue objectives in the public interest. Furthermore, it justifies the reservation of such activity to notaries by notably highlighting that they constitute "a particular category of professionals in which there is public confidence and over which the State exercises particular control". The CJEU does not state notaries are judges [START_REF] Nourissat | Le notaire n'est pas une juridiction au sens des règlements européens de coopération judiciaire civile[END_REF] , nor "out-ofcourt magistrates" [START_REF] Nourissat | Quand la Cour de justice reconnaît les activités réservées des notaires…[END_REF] , because it thinks in terms of activities much more than status. But, by characterizing their activities from the point of view of public confidence, which it linked before to the impartiality of the notary 18 , and the preventive administration of justice which is not conceivable without a guarantee of its authenticity, the CJEU used the main components which permitted the ECtHR to qualify notaries as "out-of-court magistrates". The two judgments rendered in Luxembourg and Strasbourg, within a few days of each other, do not conflict, but exhibit a difference of degree. This is another reason to study the impact of recognising the European status on relations between human rights and public notaries.

The impact of recognising the European status on the relation between human rights and notaries

Human rights which can be affected by the recognition of an "out-of-court magistrate" status are those which can be exercised by a notary (A) and those the public seek to exercise before him (B).

A. The notary's human rights

As public confidence in a good administration of justice is the very heart of the judicial professions' European status, it is mainly the exercise of the right to freedom of expression of its members that is quite specific. In fact, public confidence would be more dangerously affected if the judicial professions could express themselves as freely as journalists, or in fact anybody. show restraint in exercising his freedom of expression in cases where the authority and impartiality of the judicial power are likely to be called into question. The term "out-of-court magistrate" has also been adopted to restrain the freedom of expression for notaries. Above all, the judgment Ana Ionita v. Romania of 31 March 2017 refers to the notary's freedom of speech. Thus, the limits to notaries' freedom of speech have to be carefully considered.

In this respect, it should be particularly noted that the Court, like for lawyers and judges, does not sacrifice the freedom of speech of the notary on the altar of public confidence. In fact, it admits "with satisfaction that national jurisdictions had balanced the interests (Ana Ionita's and Order of notaries' interests), without imposing on the claimant an absolute ban to criticize notarial activities" 24 . As a matter of principle, the freedom of expression of the notary is protected, so it has to be identified in what way it can be limited by specific duties of the "out-of-court magistrate". To succeed, it has to refer to the structure of Ana Ionita's judgment which, very classically, put the disciplinary sanction imposed to the indiscrete member of the notary chamber of Bacau to the three-steps test of legality, legitimacy and necessity in a democratic society.

In recognising the suspension from her duties as notary for a period of four months ruled by the Romanian law (n°36/1995) and the Romanian ethical code of public notaries, the Strasbourg's Court acknowledged undisputably that the interference in the right of freedom of expression of the claimant was legal. This first point is not a matter giving rise to discussions, except to emphasise that, in France, such a disciplinary sanction would satisfy the legacy test. In fact, it is governed by Article 32 et seq. of the writ n°45-1418 of 28 th June 1945 on the discipline of notaries and some ministerial officiers. There will be more to say about the legitimacy test.

The Court considered that the interference pursued, "with no doubt", one of the objectives listed in Article 10 §2 of the Convention, namely "the protection of reputation or rights of others". "Others" were in this case the public notaries' national order, that is to say a legal entity whose reputation is protected on the grounds of freedom of expression as a natural person since the judgment Uj v. Hungary of 19 July 2011 25 . Thus, there is nothing original to emphasize here. 19 Req. n°73797/01. 20 Suffice it to say that the right to freedom of expression of notaries could be of course legitimately limited to protect the reputation or rights of a fellow colleague. However, it has to be underlined that the Court believed it was necessary to add that the disputed interference in Ana Ionita's right of freedom of expression pursued the legitimate objective to protect public confidence towards the professional order's executive bodies. This point is unusual. It adds the legitimate ground of protection of public confidence to the listing of Article 10 §2, in contradiction to the rule, referred to in §37, that this listing must be strictly interpreted. Perhaps, it is an exception in order to provide a better legal framework for freedom of expression for the members of judicial professions by the introduction of an autonomous exception. Perhaps, and more probably, it is a derivation of the guaranty of the "authority and impartiality of the judiciary", which ended the listing of Article 10 §2, that the Court would not be comfortable to use in the case of the "out-ofcourt magistrate".

Lastly, with regard to the test of necessity in a democratic society, the Court distinguishes between the control of proportionality itself and the relevant and sufficient reasons put forward by the domestic authorities26 in support of the interference complained of. Regarding the proportionality and strictly speaking, in order to admit that the Romanian authorities have struck a fair balance between one's right to freedom of expression and the right to reputation of others', the Court observed primarily that the four months' sanction of suspension was not the most severe because national law allows the exclusion of the profession.

In order to make sure that the reasons put forward by the domestic authorities are relevant and sufficient, the Court developed a denser argument which doesn't refer exclusively to paragraph 47, a keystone of the Ana Ionita's judgment already examined from all perspectives. Three others components are used to support the relevant and sufficient reasons put forward to justify the sanction.

First, the Court considers that Ana Ionita's comments are not relevant to any public interest debate regarding general interest questions about the order of notaries. This assertion exercises decisive influence, because in case of public debate interests, it leaves little room for limitations to freedom of expression (cf in particular the Grand Chamber's judgment Verein Gegen Tierfabriken Schweiz v. Switzerland n°2 of 30 June 200927 ). Nevertheless, it is surprising because the question of professional tax that Ms Ana Ionita spoke about on the television show was linked with the organisation of the profession which is connected to the general interest.

Second, the Court gives due attention to the severity and the tone of the unsubstantiated personal attacks aimed at the executives of the notaries' order, to infer that they should have been more factually based than is usually required for a value judgment which, unlike the facts ascribed to any individual, cannot be precisely proven (on the main difference between facts and value judgments, v. judgment De Haes and Gisels v. Belgium of 24 February 1997 28 ). The consequence of this observation is to hide the very specific context in which Ana Ionita's case developed. The claimant argued against the president of the notaries' chamber that he was strongly involved in the functioning of the totalitarian regime which took place in Romania for 45 years. Yet, the question, known as "lustration", of the exclusion from some professions of overzealous collaborators in the old communist regimes, has given rise to a large and paradoxical body of case law in the ECtHR 29 . Undoubtedly, the claimant complained in such an expedient way, but perhaps the question regarding the president of the notaries' chamber was not so absurd in the interest of the profession and was intended to advise the other European member States where the risk of access to the profession of old communist activists is very limited… Third, in order to sustain the relevant and sufficient reasons put forward to justify the sanction, the Court emphasized that the serious charges had been formulated during a televised show whose effects are more immediate and stronger than the written press. Undoubtedly, in comparison with the famous Fuentes Bobo v. Spain judgment of 29 February 2000 30 , it is a particularly unfavourable development for the freedom of expression of the notaries who, in fact, rarely express themselves on air on television and radio shows… In this judgment, the Court admitted that it has to be more lenient towards the comments made during a vigorous live debate with journalists. Unless Ms Ionita had thought through the personal attacks against her adversaries before her participation in the televised show (Comp. judgment De Diego Nafria v. Spain of 14 March 2002 31 ), the right to a more spontaneous freedom of expression intended to take into account the particular context of the speech in the audiovisual media seems to have been refused for the notary.

Thus, the new European rank of "out-of-court magistrate" ascribed to the notary has involved a clear restraint in the exercise of one's right to freedom of expression, however protected this may be. It has to be verified whether a similar negative influence is liable to occur towards other human rights derived from the Convention to the notary (i.e. right to a fair trial provided by Article 6 §1 of the Convention, protection of property provided by Article 1 of the Protocol of the Convention…). But, as they are not relevant in Ana Ionita's case, the examination of these points has been delayed, just to focus on the right to freedom of expression. The conclusion cannot be that these other rights of the notaries are less important or that there is no restraint in their exercise.

B. The human rights exercised before the notary

While not pretending to assimilate the notaries to the judicial or administrative judges, it remains true that characterizing the notaries as "out-of-court magistrates" may give rise, by the transposition of the right to a fair trial under Article 6 of the Convention, to a "right to a fair notary", as well as a "right of access to a notary". The consequence of such an alignment of the preventive "out-of-court magistrates" with the judges has already been examined in a previous article 32 . But, as the ECtHR now recognizes the specific nature of the notary's profession, few clarifications are allowed to be made to add to these last observations. Obviously, the recognition of "out-of-court magistrates" by the ECtHR, by affording a right to a fair notary which involves the respect of the independency and impartiality of the notary, represents a European barrier to the creeping development of the "single law profession" which is emerging as lawyers merge with other law professions. The reference to "out-of-court magistrates" highlights very effectively the incompatibility between the two professions 33 . How is it possible to put lawyers and "out-ofcourt magistrates" together in the same profession? The impartiality is vital for notaries 34 , as it is for judges, while the lawyer has to defend the interests of his client : the lawyer is "logically unfair" [START_REF] Beignier | Droit et déontologie de la profession d'avocat[END_REF] . As has already been illustrated in an adage inspired from the Consorts Richet et le Ber v. France judgment of 18 November 2010: "One cannot be both judge and judged" [START_REF] Marguénaud | Nul ne peut être notaire et partie : émergence d'un nouvel adage européen[END_REF] .

The recognition of "out-of-court magistrates" requires a strong review of the pluriprofessionality introduced by the law n°2015-990 of 6 August 2015, called "Macron's law", and the n°2016-394 writ of 31 March 2016, implemented by three decrees of 5 May 2017. It has already led to the creation of some pluri-professional companies gathering several legal and accounting professions together. So, lawyers and notaries can now practice their professions together in the same company. But, the pluri-professionality has to respect the impartiality of the notary 37 , because the applicable rules of ethics of each professional in the same structure must be kept in place [START_REF][END_REF] . So, after the Ana Ionita v. Romania judgment of 21 March 2017, there are legitimate grounds for questioning the validity of notarial deeds written by a notary for a client of his lawyer partner. The notary, if he accepted to write such a deed, would certainly be placed in a situation "which might suggest that his neutrality has been undermined" [START_REF] Sagaut | Cass. 1 ère civ[END_REF] .

Pursuant to Article 6 §1 of the Convention of the ECtHR, the Court of cassation (French Supreme Court) has recently 40 demonstrated its attachment to the impartiality of the bailiff, public officers in charge of the enforcement of the decisions of the judges and the notarial deeds. The Court of cassation overturned a decision which had rejected a plea for annulment of a writ delivered on behalf of the bailiff's chamber by a bailiff who was also treasurer and member of the board in charge of the asset management and the financial interests of the chamber, because these qualities "cast reasonable doubt upon his impartiality and his independence". The writ delivered on behalf of the chamber by a bailiff, who is "associated" to its activities, is also void. It should be the same for a notarial deed written by a notary on behalf of the client of his associate. Moreover, under "Macron's law", there are serious reasons to doubt the conformity of the pluriprofessional companies, because when several notaries are partners in the same company, the legal person is integrated into the profession : it is the public officer, holder of the office, and practices itself the profession ; the natural persons who are partners are not much more than "under-officers". A notary, legal person, can be at the same time, under the new "Macron's law", a lawyer… This is a serious contradiction. There is now a situation that leads to confusion over the impartiality of the notary and undermines public confidence in the profession. As such, this inter-professionality allows for the creation of "a single profession" by an alternative path through a legal person.

Therefore, banks or insurance companies should never be allowed to take shares in "out-of-law magistrates" companies. No jurist could imagine that banks and insurances could take shares in State courts. It should be the same for public notary companies whose independence and impartiality must be protected. The power of the State must be separate from financial interests. Remember that notarial deeds are enforceable. So, it is essential that automatically enforceable notarial deeds are written by an impartial public officer, like a judgment which is rendered by an impartial judge. Generally speaking, the protection of impartiality is the main stake in order to preserve public confidence in the profession and the State represented by the notary. At the present time, when the restoration of confidence in public action is currently under discussion, it is worth recalling that public confidence in the proper administration of justice, curative and preventive, relies on the impartiality of judges and "out-of-court magistrates" made available to the public by the State.

In addition to the right to a fair notary, which could be invoked by citizens, the Ana Ionita v. Romania judgment suggests a potential alignment of the internal organisation of the profession with the one currently retained for the judges. It may be questioned if the notary profession no longer takes the Superior Council of the Magistracy (CSM) as an example, which decided to integrate some members outside of the profession in the disciplinary bodies, for example, judges to settle the disputes between notaries and their clients or between notaries. The profession could also be inspired by the initiative taken in 2006 by the CSM 41 to tackle the ongoing criticism of corporatism and to preventively inform the judges, to publish a compilation of all the disciplinary decisions while the disciplinary procedure was until this time secret. This initiative could have damaged the profession because the public and the media could have drawn some negative general consequences from particular cases. It did not arise and the initiative has even been welcomed. The corporatism criticism is a current one towards the notary profession. It is to be stepped up with the progress in transparency requirement in civil society. Also, the publication in a collection of the disciplinary decisions rendered against the notaries, as well as the decisions taken by the notaries' chamber, as authority of conciliation, in regard to the relation between the notaries and between the notaries and their clients, could contribute to promote the ethical requirements of the profession. The notary's status as an independent and impartial "out-of-court magistrate" certainly justifies the closure of the profession to other interests, but if the notaries are heard on that point, they should insist on the ethical opening of the profession.

Public confidence towards the profession could only be reinforced in this way, especially as this opening would be spontaneous and would not be made under the pressure of the public authorities. The French notaries would show that the profession is faithful to the trust given to it by the ECtHR in its Ana Ionita v. Romania judgment of 21 March 2017, which recognises them as "out-of-court magistrates".

  Despite the lenient statements expressed by the famous Kyprianou v. Chypre of 15 December 2005 19 and Mor v. France of 15 December 2011 20 judgments or more recently Morice v. France of 23 April 201521 , a lawyer has to remember that, when he expresses himself in the courtroom or through the media, the dignity and majesty of justice are at stake (Cf. Rodriguez Ravelo v. Spain of 12 January 2016 22 ). Regarding judges, it has been established since the Grand Chamber's judgment Wille v. Liechtenstein of 20 October 1999 23 that it can be expected of him that he should

  La légitimation du notariat par le droit européen des droits de l'Homme, JCP éd. N, 2017, n°18, p. 51, Etude 1147, n°14 et s. A translation of this article is available (https://hal.archives-ouvertes.fr/hal-01483911/document, The legitimation of civil law notaries by the law of the European convention on human rights).33 On the incompatibility of the two professions in Europe (except in Germany), J. Pertek, La prestation de services ne permet pas aux avocats d'empiéter sur une activité légitimement réservée aux notaires, JCP éd. G, 2017, n°16, p. 774. 34 J.-Fr. Sagaut et M. Latina, op. cit., p. 50, n°109.

	30 Req. 39293/98.
	31 Req. n°46833/99.

32 J-P. Marguénaud, C. Dauchez and B. Dauchez,

Cf. judgment Petrina v. Romania of 14 October 2008.

Req. n°40660/08.

Req. n°78028/01.

§ §183 and 187.

Req. n°15450/89.

Req. 20261/12.

§164.

Cf. Morice v. France, Req. n°29369/10, Gr. Ch. 23 April 2015 §128.

National Notaries Chamber v. Albania, déc., n°17029/05, 6 May 2008.

Cf. F. Sudre, JCP 2017, éd. G, 289.

Req. n°32772/02.

Req. n°19983/92.

Cf. Sudre and alii, Les grands arrêts de la Cour européenne des Droits de l'Homme, PUF, 8ème éd., 2017, p. 514.


Arnaldo E Pereira
email: arnaldop@neurable.com

Dereck Padden
email: dereckp@neurable.com

Jay J Jantz
email: jay.jantz@queensu.ca

Kate Lin

Ramses E Alcaide-Aguirre

Arnaldo E Pereira Neurable

Padden Dereck

Neurable

Cross-Subject EEG Event-Related Potential Classification for Brain-Computer Interfaces Using Residual Networks

des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

A brain-computer interface (BCI) is a device that allows a user to interact with a computer through neural activity [START_REF] Huggins | Effects of text generation on P300 brain-computer interface performance[END_REF][START_REF] Cecotti | Convolutional neural networks for P300 detection with application to brain-computer interfaces[END_REF]. BCIs were originally conceived as a way to allow paralyzed individuals to communicate [START_REF] Farwell | Talking off the top of your head: A mental prosthesis utilizing event-related brain potentials[END_REF] and move around [START_REF] Barachant | Multiclass braincomputer interface classification by Riemannian geometry[END_REF]. And while that is still a primary motivation behind BCI research, the last decade has seen an increased interest in the potential uses of BCIs-EEG-based BCIs, in particular-for hands-free interaction with consumer devices, industrial applications, and entertainment [START_REF] Scherer | Interaction with virtual and augmented reality environments using non-invasive brain-computer interfacing[END_REF][START_REF] Barachant | Multiclass braincomputer interface classification by Riemannian geometry[END_REF][START_REF] Barachant | A plug&play P300 BCI using information geometry[END_REF]. BCIs may even be a viable interaction method for virtual reality and augmented reality applications [START_REF] Scherer | Interaction with virtual and augmented reality environments using non-invasive brain-computer interfacing[END_REF][START_REF] Jantz | A braincomputer interface for extended reality interfaces[END_REF].

One of the most-commonly studied signals in EEGbased BCIs is the P300 event-related potential (ERP) [START_REF] Lotte | A review of classification algorithms for EEG-based brain-computer interfaces: A 10year update[END_REF]. The P300 is so called because it is characterized by a positive peak approximately 300ms after the subject attends to an "oddball" target stimulus among several similar but nontarget stimuli [START_REF] Farwell | Talking off the top of your head: A mental prosthesis utilizing event-related brain potentials[END_REF]. The stimuli can be visual, such as flashing rows and columns of letters [START_REF] Farwell | Talking off the top of your head: A mental prosthesis utilizing event-related brain potentials[END_REF][START_REF] Cecotti | Convolutional neural networks for P300 detection with application to brain-computer interfaces[END_REF][START_REF] Shan | A simple convolutional neural network for accurate P300 detection and character spelling in brain computer interface[END_REF], objects, faces, or scenery [START_REF] Manor | Convolutional neural network for multi-category rapid serial visual presentation BCI[END_REF][START_REF] Liu | Deep learning based on batch normalization for P300 signal detection[END_REF], or geometric shapes [START_REF] Das | Visually evoked potential for eeg biometrics using convolutional neural network[END_REF]. But they can also be auditory [START_REF] Carabez | Convolutional neural networks with 3D input for P300 identification in auditory brain-computer interfaces[END_REF] or tactile [START_REF] Kodama | Convolutional neural network architecture and input volume matrix design for erp classifications in a tactile p300-based brain-computer interface[END_REF]. The common denominator is the "oddball" task design.

Despite recent progress in the BCI literature, there are a few major obstacles preventing the widespread deployment and adoption of EEG-based BCIs. First, noninvasive methods such as EEG have a low signal-to-noise ratio [START_REF] Lotte | A review of classification algorithms for EEG-based brain-computer interfaces: A 10year update[END_REF][START_REF] Rivet | xDAWN algorithm to enhance evoked potentials: Application to braincomputer interface[END_REF]. Second, the EEG signals that are used to determine user intent in these applications-such as the P300are highly specific to each person and can even vary significantly within an individual over time [START_REF] Zanini | Transfer learning: A riemannian geometry framework with applications to braincomputer interfaces[END_REF][START_REF] Reuderink | A subject-independent brain-computer interface based on smoothed, second-order baselining[END_REF]. While the first problem can be addressed with more or less success through signal filtering [START_REF] Rivet | xDAWN algorithm to enhance evoked potentials: Application to braincomputer interface[END_REF][START_REF] Blankertz | Optimizing spatial filters for robust EEG singletrial analysis[END_REF][START_REF] Cecotti | Single-trial classification of event-related potentials in rapid serial visual presentation tasks using supervised spatial filtering[END_REF], the second problem has been harder to deal with. In practice, that difference in signal profiles across people and data sessions has meant that most classifiers used to detect EEG event-related potentials in real time are trained specifically for each user and often even for a particular data session [START_REF] Lotte | A review of classification algorithms for EEG-based brain-computer interfaces: A 10year update[END_REF].

The solution to this problem is to find a way to train EEG classifiers that can generalize across people and time. Convolutional neural networks have recently gained attention in the EEG literature [START_REF] Cecotti | Convolutional neural networks for P300 detection with application to brain-computer interfaces[END_REF][START_REF] Manor | Convolutional neural network for multi-category rapid serial visual presentation BCI[END_REF][START_REF] Liu | Deep learning based on batch normalization for P300 signal detection[END_REF][START_REF] Shan | A simple convolutional neural network for accurate P300 detection and character spelling in brain computer interface[END_REF][START_REF] Lawhern | Eegnet: a compact convolutional neural network for eeg-based braincomputer interfaces[END_REF] because of their feature-extraction properties [START_REF] Cecotti | Single-trial classification of event-related potentials in rapid serial visual presentation tasks using supervised spatial filtering[END_REF] and success in image classification [START_REF] Rawat | Deep convolutional neural networks for image classification: A comprehensive review[END_REF]. We extend this line of work by applying wide residual networks (WideResNets) [START_REF] Zagoruyko | Wide residual networks[END_REF] with spatial dropout [START_REF] Srivastava | Dropout: A simple way to prevent neural networks from overfitting[END_REF][START_REF] Tompson | Efficient object localization using convolutional networks[END_REF] and cutout [START_REF] Devries | Improved regularization of convolutional neural networks with cutout[END_REF] to 6-electrode EEG data obtained from 66 subjects performing a visual oddball task in virtual reality. This paper presents a minimal preprocessing stream to convert EEG epochs into images. We then examine the performance of ResNets [START_REF] He | Deep residual learning for image recognition[END_REF][START_REF] He | Identity mappings in deep residual networks[END_REF] of different widths and depths on ERP detection. Furthermore, we study the effect of using cutout for regularization of EEG data. Our experiments show that convolutional networks can classify target vs nontarget stimuli accurately on a test set taken from many subjects, and that it may be possible with enough data to train a classifier that can detect ERPs in the general population without any need to calibrate or train specifically for an individual user at runtime.

Motivation and Related Work

Convolutional neural networks (CNNs) are excellent at classifying images of objects belonging to different classes [START_REF] Krizhevsky | Imagenet classification with deep convolutional neural networks[END_REF][START_REF] Simonyan | Very deep convolutional networks for large-scale image recognition[END_REF][START_REF] Szegedy | Going deeper with convolutions[END_REF][START_REF] Szegedy | Inception-v4, inception-resnet and the impact of residual connections on learning[END_REF]. In a convolutional network, lower-level representations of the image are convolved with a learned kernel to extract higher-level features from a larger region of the image [START_REF] Zeiler | Visualizing and understanding convolutional networks[END_REF][START_REF] Rawat | Deep convolutional neural networks for image classification: A comprehensive review[END_REF][START_REF] Krizhevsky | Imagenet classification with deep convolutional neural networks[END_REF]. Multiple convolutional layers can be stacked to extract progressively higher-level feature maps [START_REF] Zeiler | Visualizing and understanding convolutional networks[END_REF][START_REF] Rawat | Deep convolutional neural networks for image classification: A comprehensive review[END_REF]. CNNs have gained attention in recent years as a possible solution for EEG signal classification because of their power to extract nonlinear and holistic features from their input data.

The first study to use convolutional neural networks to classify P300s was done by Cecotti and Gräser [START_REF] Cecotti | Convolutional neural networks for P300 detection with application to brain-computer interfaces[END_REF]. The input to their network for each epoch of data to be classified is a tensor of dimensions N electrodes × N samples . Their network was composed of the input layer just described, feeding into a first hidden spatial convolution layer of N F M feature maps with a convolution kernel of size (N electrodes × 1), then a second convolutional layer producing N F M × 5 feature maps, which performs time convolution and downsampling with a kernel of length 13 = N samples 3

, followed by a fully connected layer and an ouput layer of two neurons representing the probabilities of the two classes (target and nontarget).

Similar studies followed. The authors of [START_REF] Cecotti | Single-trial classification of event-related potentials in rapid serial visual presentation tasks using supervised spatial filtering[END_REF] used a simplified version of the Cecotti-Gräser network to explore the spatial filters learned by the first convolutional layer and compared the results to those obtained by using xDAWN [START_REF] Rivet | xDAWN algorithm to enhance evoked potentials: Application to braincomputer interface[END_REF] and CSP spatial filters [START_REF] Blankertz | Optimizing spatial filters for robust EEG singletrial analysis[END_REF]. Manor and Geva added a Figure 1: Visual stimuli presented during the EEG recording task. Subjects were instructed to make a mental note when the object indicated by a white circle flashed and to ignore the flashing of all other objects. max-pooling layer [START_REF] Lecun | Convolutional networks and applications in vision[END_REF] after the intial spatial convolution and replaced the single temporal convolution and downsampling layer of the Cecotti-Gräser network with temporal convolution → max pooling → temporal convolution. They also incorporated ReLU [START_REF] Nair | Rectified linear units improve restricted boltzmann machines[END_REF] and dropout [START_REF] Srivastava | Dropout: A simple way to prevent neural networks from overfitting[END_REF] into their network. The network in [START_REF] Liu | Deep learning based on batch normalization for P300 signal detection[END_REF] was also based on the Cecotti-Gräser architecture, but it added batch normalization [START_REF] Ioffe | Batch normalization: Accelerating deep network training by reducing internal covariate shift[END_REF] after the input and temporal convolution layers. As in Manor and Geva [START_REF] Manor | Convolutional neural network for multi-category rapid serial visual presentation BCI[END_REF], Liu et al. used ReLU activations in their convolutional layers. Other studies used networks with similar network architectures [START_REF] Shan | A simple convolutional neural network for accurate P300 detection and character spelling in brain computer interface[END_REF][START_REF] Lawhern | Eegnet: a compact convolutional neural network for eeg-based braincomputer interfaces[END_REF].

A few P300 studies using convolutional neural networks have experimented with input tensors different from the ones in Cecotti and Gräser [START_REF] Carabez | Convolutional neural networks with 3D input for P300 identification in auditory brain-computer interfaces[END_REF][START_REF] Maddula | Deep recurrent convolutional neural networks for classifying P300 BCI signals[END_REF][START_REF] Kodama | Convolutional neural network architecture and input volume matrix design for erp classifications in a tactile p300-based brain-computer interface[END_REF]. The paper of Kodama and Makino [START_REF] Kodama | Convolutional neural network architecture and input volume matrix design for erp classifications in a tactile p300-based brain-computer interface[END_REF] is the closest to our approach to network input. The authors of that study created a 20 × 20 square matrix from the time series data for each electrode and then then joined the feature matrices from each of 8 electrodes, plus one feature matrix created from the average of all 8 electrodes, into a single 2D input tensor for each epoch of EEG data [START_REF] Kodama | Convolutional neural network architecture and input volume matrix design for erp classifications in a tactile p300-based brain-computer interface[END_REF]. Our method also represents an entire EEG data epoch as a single 2D tensor, but we arrange our data in a way that we believe better preserves its temporal information in the deeper layers of the network and adds regularization

Materials and Methods

EEG Dataset

Sixty-six subjects participated in a visual oddball task in virtual reality. Each subject wore a Neurable DK1 headset (Neurable, Inc., Boston, Mass.), which comprises an HTC Vive (HTC, New Taipei City, Taiwan) virtual reality headset equipped with DSI dry EEG electrodes (Wearable Sensing, San Diego, Cal.) at the P4, Pz, P3, PO8, PO7, and Oz locations. EEG was sampled at 300Hz. The experiment was programmed in the Unity game engine (Unity Technologies, San Francisco, Cal.) using the Neurable SDK, which allows synchronization of stimuli presented in virtual reality using Unity with the EEG data from the electrodes attached to the headset.

A screenshot from the experiment is presented in Figure 1. The subjects were placed in a room in virtual reality while seated on chair to maintain an approximately fixed distance to a plane presenting five toy objects: a ring stack, a train, a ball, a cube, and an airplane. Each recording session consisted of 10 trials. In each trial, one of the five objects was randomly selected as the target by having a solid white circle appear around the item. This solid circle stayed around the item for the duration of the trial. Subjects were instructed to make a mental note when the target object for the current trial flashed and to ignore the flashing of the remaining four objects.

Each trial comprised 10 sequences. A sequence consisted of each of the five objects flashing bright green once in a random order. That is, in a given trial the first sequence could consist of the flash order airplane, ball, cube, ring stack, train, while the second sequence might be ball, ring stack, train, cube, airplane, and the third might be train, airplane, cube, ring stack, ball, and so on for the given number of sequences per trial. Each flashing object was highlighted in bright green for 60ms. Within a trial, the interstimulus time in between the end of one flash and the beginning of the next one was 100ms. There was a pause of approximately 3s in between trials. During this pause the circle around the target object disappeared and the subject was shown an animation of the object floating towards the subject. The object would then return to its original position along the other four objects and a new circle would appear around another object, indicating it as the target for the next trial. For each stimulus, an EEG data epoch was extracted consisting of the 800ms of recorded data for all 6 electrodes beginning with the stimulus flash onset.

Thus, there were 10 × 10 × 5 = 500 labeled EEG epochs for each session: 100 target epochs and 400 nontarget epochs. Additionally, 3 of the sessions (1 session each from subjects 1, 4, and 25) were mistakenly recorded at 6 sequences per trial rather than 10 sequences per trial. Seeing no downside other than the loss of the extra data which could have been acquired in those sessions, we opted to keep those 3 sessions in the dataset. There were a total of 233 data sessions recorded (across all subjects). In total the dataset consists of 115900 EEG epochs. To create a training and test set, the target and nontarget epochs in each session were separated. The targets and nontargets were then separately shuffled and 80% of each designated for training, Figure 2: Visualization of a random subset of 50 target epochs (red crosses) and 50 nontarget epochs (blue stars) using t-SNE [START_REF] Maaten | Visualizing data using t-SNE[END_REF].

with the other 20% set apart for testing. The training sets from all sessions were then combined and shuffled to form a single large cross-subject and cross-session train set, and the same was done to create a combined test set. The final distribution of the data is shown in Table 1

Data Preprocessing

This section describes our method for turning each raw EEG data epoch into a 2D image-like input tensor. For convenience, we abbreviate N electrodes as N e and N samples as N s . Each raw epoch data matrix E has a shape N e × N s . N s in the case of the data above is 240, since the epochs are each 800ms and the data is sampled at F s = 300Hz. A characteristic P300 signal is expected around 300ms after stimulus onset in the target epochs but not in the nontarget epochs [START_REF] Farwell | Talking off the top of your head: A mental prosthesis utilizing event-related brain potentials[END_REF][START_REF] Cecotti | Convolutional neural networks for P300 detection with application to brain-computer interfaces[END_REF]. Additionally, there might be an attentional [START_REF] García-Larrea | Revisiting the oddball paradigm. non-target vs neutral stimuli and the evaluation of ERP attentional effects[END_REF] or visual component of the event-related potential around 100ms after onset that correlates with the target/nontarget label [START_REF] Bonmassar | Spatiotemporal brain imaging of visual-evoked activity using interleaved eeg and fmri recordings[END_REF][START_REF] Herrmann | Early stages (P100) of face perception in humans as measured with event-related potentials (erps)[END_REF]. Figure 3 shows a P300 epoch and an NP300 epoch in their time series and image forms.

We apply the minimal preprocessing described below to each epoch to turn it from raw temporal data into an input image for the CNN. Minimal data processing is emphasized for two reasons. First, in online applications the data preprocessing must not be too computationally expensive. Second, we are using CNNs precisely for their power as feature extractors.

Each epoch E is bandpass filtered to 2-9Hz. The signal data is then decimated by a factor of 2. We then find an amount of padding P for the decimated matrix D of dimensions N e × N d such that:

For N e , N d , P ∈ Z + , M ≡ N e (N d + 2P ) P = argmin P {M | M ∈ Z + } (1)
Each electrode time series is padded with P zeros at either end. Let D i,t indicate the t th time sample taken from electrode i. There are N d = 120 samples per electrode after decimation and our N e = 6. Therefore, we pad each D i with P = 15 zeros at the beginning and at the end, ending up with a padded D of dimensions M = 6 × 150 = 900, with N p = 150 samples for each electrode after padding. D is then reshaped into a 30 × 30 square matrix D Sq ac-cording to eq. ( 2). We choose this method rather than the block layout of [START_REF] Kodama | Convolutional neural network architecture and input volume matrix design for erp classifications in a tactile p300-based brain-computer interface[END_REF] because in Kodama's method corresponding time samples taken from highly correlated electrodes can be in distant parts of the image, limiting the effectiveness of regularization methods like cutout.

D Sq =         D 1,1 D 2,1 . . . D Ne, √ M Ne D 1, √ M Ne +1 D 2, √ M Ne +1 . . . D Ne, 2 √ M Ne . . . . . . D 1, ( √ M -1) √ M Ne +1 . . . D Ne,Np        
(2) Then, the entries d i,j of D Sq are discretized into 256 bins and normalized to be in [0, 1]. We preprocess the data this way for two reasons. First, CNN architectures have been designed for and tested most extensively on square, discrete image data of this form. Therefore, heuristics and parameters developed for classifying image data will likely transfer most readily to data in this form. Second, by discretizing the data into 256 bins we reduce the entropy of our input layer H in . Therefore, discretizing the input data shrinks the search space for future weight vectors. In particular, let H c1 be the entropy of the output tensor from the convolutional layer connected to the input layer. H c1 = F(H in , p c1 ), where F is a increasing function in H in and p c1 , and p c1 is a constant that depends on the input dimensions and the convolution kernel size, stride, and number of output feature maps. And likewise, for a second convolution following the first H c2 = F(F(H i n, p c1 ), p c2 ) is also increasing in H in , and so on. Intuitively, the space of feature maps that a convolutional layer can produce given all possible discretized inputs is smaller than the space of all feature maps it could produce given all possible nondiscretized inputs from the same distribution. Therefore, this discretization acts as a regularizer.

Neural Network Architectures

A ResNet [START_REF] He | Deep residual learning for image recognition[END_REF][START_REF] He | Identity mappings in deep residual networks[END_REF] is a CNN that uses residual units that add the output of convolutional layers and activations to the input to the residual unit by using skip connections. Several variations of residual convolutional networks (ResNets), introduced in [START_REF] He | Deep residual learning for image recognition[END_REF] and refined in [START_REF] He | Identity mappings in deep residual networks[END_REF], are currently the state of the art [START_REF] Huang | Densely connected convolutional networks[END_REF][START_REF] Xie | Aggregated residual transformations for deep neural networks[END_REF][START_REF] Han | Deep pyramidal residual networks[END_REF][START_REF] Gastaldi | Shake-shake regularization[END_REF][START_REF] Devries | Improved regularization of convolutional neural networks with cutout[END_REF] in image classification on the CIFAR [START_REF] Krizhevsky | Cifar-10[END_REF] and ImageNet [START_REF] Deng | ImageNet: A Large-Scale Hierarchical Image Database[END_REF] datasets. Following [START_REF] He | Identity mappings in deep residual networks[END_REF][START_REF] Zagoruyko | Wide residual networks[END_REF][START_REF] Devries | Improved regularization of convolutional neural networks with cutout[END_REF], our networks use preactivation. That is, batch normalization and dropout precede the convolutional layers in the residual blocks. We use the term residual block to refer to the structure shown in Figure 4b, which in turn is composed of one or more stacked residual units. A residual unit takes the form: 4: Model comparison. We use the best reported results on target/nontarget classification tasks run on fewer than 3 subjects. For tasks with more than 3 subjects where the authors reported mean metrics, we present those. Where a metric was not reported in a paper but it could be calculated from the reported results, we have calculated it. The best results for each metric are shown in bold.

Y = C • R • B • • • • • C • R • B • C(X) + P(X) (3) (a) (b)
tions we use, following the findings in [START_REF] Zagoruyko | Wide residual networks[END_REF]. But it can also be some other mapping of the input X into the same space as the left-hand side of the sum. We use residual units of length 2 for all our networks, again following the results of [START_REF] Zagoruyko | Wide residual networks[END_REF] and [START_REF] Devries | Improved regularization of convolutional neural networks with cutout[END_REF].

Different WideResNet [START_REF] Zagoruyko | Wide residual networks[END_REF] architectures were tested on our data, with and without cutout [START_REF] Devries | Improved regularization of convolutional neural networks with cutout[END_REF]. The name of each network tested describes its architecture. So, for example, WRN-16-2 indicates a network with a depth of 16 weight layers and a width parameter k = 2. A width parameter k = 1 corresponds to the architecture template of [START_REF] He | Identity mappings in deep residual networks[END_REF]. The depth d of the network is an integer d satisfying d = 6n + 4, for n ∈ Z + , n > 0. All networks use in this paper follow the architecture pattern shown in Figure 4.

Training parameters generally followed those for the CI-FAR datasets in [START_REF] Zagoruyko | Wide residual networks[END_REF] and [START_REF] Devries | Improved regularization of convolutional neural networks with cutout[END_REF].The loss function was crossentropy weighted to account for the 4 : 1 class inbalance between target and nontarget stimuli. Dropout of 0.3 was used between convolutional layers in the residual blocks. The models trained with cutout use a 15 × 15 cutout square. See Figure 5 for an example of how cutout was applied to our data. No data augmentation was used.

Results

All models were programmed in Keras with a Tensor-Flow back end and trained on an NVIDIA Tesla V100 GPU. In Table 2 we show the results of runs with networks of depth d = 16 and width parameter k = 1, 2, 4, 8, with and without cutout. The effect of depth on performance was tested by fixing the width at k = 8 and changing the depth to 10, 22, 28, and 40 weight layers. For the experiments with varying depth we used cutout on all models. Those results are shown in Table 3.

We report the following metrics, with T P , T N , F P , and F N standing for true positives, true negatives, false positives, and false negatives, respectively:

F1 = ( Precision -1 + Recall -1 2 ) -1 (10)
It is important to break down performance in this way because of the class imbalance inherent in oddball tasks. For example, for tasks with a 5 : 1 class imbalance, which is common in the literature, a classifier could obtain a raw accuracy value of 83.33% simply by predicting nontarget for every epoch given to it. Table 4 compares our best model and our smallest model to other convolutional neural network models from the literature. Manor and Geva reported performance metrics on two oddball tasks: a 6 × 6 speller and a task where the subjects where presented photographs belonging to different categories. We show their performance on each of those tasks.

Discussion

Our top model (WRN-16-8) outperforms the top CNN models in the P300 BCI literature for which metrics broken down by target/nontarget class are available or could be calculated on accuracy, precision, and false-alarm rate. The same model also comes in a close second in terms of balanced accuracy. And, as shown in Table 4, even our smallest model (WRN-16-1)-which has only 175K parametersperforms competitively with the current state-of-the-art results for CNN EEG classifiers. But whereas-with the exception of Kodama and Makino-the results in the literature have been obtained by training a model or an ensemble of models on data from a single human subject and testing on that same single subject, we demonstrate that is possible to train CNN models that can learn generalizable features from the EEG data of many subjects and still classify accurately between target and nontarget stimuli.

On our EEG data, added regularization from cutout generally improves results, as shown in Table 2. The only exception out of the networks tested was WRN-16-2, and even there the gain in recall and accuracy was obtained at the expense of precision. The model of [START_REF] Kodama | Convolutional neural network architecture and input volume matrix design for erp classifications in a tactile p300-based brain-computer interface[END_REF] has the highest recall and, consequently, also the lowest false omission rate by far. But that is a result of the model predictions being biased towards the target class, which is undesirable in a practical BCI. For practical consumer BCIs precision is more important than recall, since it is always possible to flash another sequence to obtain confirmation of the user's intent. That is no doubt inconvenient-but less so than having the device take unintended actions.

As expected given the empirical findings on ImageNet and CIFAR in [START_REF] Zagoruyko | Wide residual networks[END_REF] and [START_REF] Veit | Residual networks behave like ensembles of relatively shallow networks[END_REF], depth is not the primary driver of accuracy. While the initial proposition for why ResNets work was that the skip connections allow more gradient to flow back through the network during the backpropagation step of training, thus allowing CNNs to grow deeper and deeper while still being able to converge [START_REF] He | Deep residual learning for image recognition[END_REF][START_REF] He | Identity mappings in deep residual networks[END_REF], empirical work in [START_REF] Zagoruyko | Wide residual networks[END_REF] showed that shallower but wider networksi.e., shallower networks with more feature maps in the intermediate layers-could outperform deeper networks with the same computational budget. One interpretation of why this happens is the fact that a ResNets with s skip connections can be conceptualized as ensemble of 2 s networks [START_REF] Veit | Residual networks behave like ensembles of relatively shallow networks[END_REF][START_REF] Huang | Deep networks with stochastic depth[END_REF]. Empirically, most of the gradient of deep ResNets tends to flow through shorter rather than longer paths [START_REF] Veit | Residual networks behave like ensembles of relatively shallow networks[END_REF]. This is one possible explanation for why the deeper networks in Table 3 do not outperform shallower networks. Depth helps up to a point, because depth is created by adding residual units and, therefore, expanding the number of paths through the network. After a certain point, however, additional longer paths may not account for much of the backpropagation gradient [START_REF] Veit | Residual networks behave like ensembles of relatively shallow networks[END_REF], but the added parameters may result in overfitting. This observation is consistent with [START_REF] Zagoruyko | Wide residual networks[END_REF] and our own observations on EEG data.

In terms of EEG data, our models are essentially ensembles of smaller networks looking at the EEG data at different levels of spatiotemporal resolution. Because successive convolutional layers have larger and larger receptive fields for each of their neurons, adding skip connections allows shorter paths to examine the EEG data at a finer resolution, while longer paths are looking at the EEG epoch at a more holistic level. Therefore, a residual network should be expected to outperform a CNN with the same architecture (minus the skip connections). To test this hypothesis we compared a ResNet and a CNN of the same depth in Table 3, and indeed we find that this is true for our data. Even the shallowest ResNet that can be built with the architecture in Figure 4 outperforms an equivalent network with the skip connections taken out.

Conclusion

We show that ResNet models can accurately classify EEG epochs from many subjects into target and nontarget classes based on user intent. The models demonstrated in this paper perform better than the best CNN models in the EEG literature in terms of accuracy, precision, and falsealarm rate-even though those models are trained and tested on an individual subject and the models presented here train and test on 66 different subjects together. Additionally, this study shows the viability of a minimal preprocessing stream to transform EEG data into an input compatible with CNN models traditionally used for image classification, while at the same time adding regularization. Cutout is shown to be effective for regularizing EEG data transformed into images. Finally, our work offers a theoretical explanation for why ResNets can perform so well on EEG epochs. The main takeaway from this study is that ResNets trained on a large and representative cross-section of the population may be viable models for consumer brain-computer interfaces that do not require a calibration or training phase for each new user.

Figure 3 :

 3 Figure 3: Examples of EEG epoch data as electrode timeseries data and as images. (a): A single target epoch after bandpass filtering and decimation. (b): A single nontarget epoch after bandpass filtering and decimation. Note the possible P300 peak around 380ms in the target epoch and a negative peak around 120ms (a possible N100 related to attending to a nontarget stimulus [12]) in the nontarget epoch. (c), (d): The same epochs as images after preprocessing.

Figure 4 :

 4 Figure 4: The networks studied in this paper follow the WideResNet architecture templates from [46]. For convolutional layers, F M , [x, y] indicates convolutions in that layer or block output F M feature maps and use a kernel of size [x, y]. (a): The generic WideResNet architecture. The number of output feature maps for each residual block is given in parentheses. The global average pooling layer is densely connected to a two-way softmax at the top. (b): The architecture of the b th residual block of the network with width parameter k and depth parameter n. All convolutions use a stride of [1, 1], except the first convolutions of residual blocks 2 and 3, which use a stride of [2, 2] for downsampling. A curved arrow indicates a skip connection.

Figure 5 :

 5 Figure 5: (a)-(d): Cutout of 15 × 15 randomly applied four times to the same epoch.

4 )

 4 Acc. =T P + T N T P + F P + T N + F N (

Table 1 :

 1 . As seen in Figure 2, the classes are not separable by a simple surface even in the high-dimensional space of the original features.

		Target	Nontarget
	Train	18480	73920
	Test	4620	18480

Description of dataset. EEG epochs from each session were split 80%/20% between training and test sets and combined into a single cross-subject and cross-session training set and test set.

Table 2 :

 2 Architectures with depth fixed at d = 16 weight layers, with and without cutout. (Mean of 5 bootstrapped runs on random 80% subsets of the test data.) The best results for each metric are shown in bold.In eq. (3), B means batch normalization, R represents ReLU activation, and C means convolution. X and Y are the input and output tensors, respectively. P is a projection such that the right-and left-hand sides of the addition have the same dimensions. P can be the identity or a [1, 1] convolution with stride[START_REF] Barachant | A plug&play P300 BCI using information geometry[END_REF][START_REF] Barachant | A plug&play P300 BCI using information geometry[END_REF] for downsampling-the two op-

	Arch.	Params	Cutout	Acc.	Bal.	Precision Recall	False	False	F1
					Acc.			Alarm	Omis-
									sion
	WRN-	175K	Yes	73.87%	64.03%	39.83%	58.82%	22.35%	11.77%	0.4750
	16-1
			No	72.92%	62.66%	37.78%	55.82%	22.84%	12.45%	0.4506
	WRN-	700K	Yes	77.06%	64.96%	43.35%	47.66%	15.62%	13.43%	0.4540
	16-2
			No	77.75%	65.10%	44.20%	43.75%	13.77%	13.99%	0.4397
	WRN-	2.75M	Yes	79.34%	67.15%	48.15%	43.15%	11.61%	13.85%	0.4551
	16-4
			No	78.58%	65.88%	45.90%	41.93%	12.30%	14.15%	0.4383
	WRN-	11M	Yes	80.51%	68.82%	51.63%	41.23%	9.67%	14.00%	0.4585
	16-8
			No	80.06%	68.03%	50.23%	40.60%	10.06%	14.18%	0.4490

Table 3 :

 3 Architectures with width parameter fixed at k = 8. (Mean of 5 bootstrapped runs on random 80% subsets of the test data.) The best results for each metric are shown in bold.

	Model	Acc.	Bal.	Precision Recall	False	False	F1	Trained	Tested
			Acc.			Alarm	Omis-		on	on
							sion
	WRN-	80.51%	68.82%	51.63%	41.23%	9.67%	14.00%	0.4585	66	66
	16-8
	WRN-	73.87%	64.03%	39.83%	58.82%	22.35%	11.77%	0.4750	66	66
	16-1
	Cecotti	78.19%	66.87%	40.9%	69.2%	20.02%	7.14%	0.5141	1	1
	& Gräser
	Manor &
	Geva
	Speller	79.1%	-	-	70%	19.11%	-	-	1	1
	Objects	75.0%	70.0%	-	64.4%	23.7%	-	-	1	1
	Liu et al. 79.02%	67.56%	42.14%	69.47%	19.07%	7.02%	0.5246	1	1
	Kodama
	&
	Makino
	Corrected	27.52%	57.35%	18.39%	97.40%* 86.46%* 3.70%	0.3094	9	1
	for 5 : 1
	class
	priors

Table


A BRIEF INTRODUCTION TO THE GEOMETRIC ANALYSIS OF DIFFERENTIAL EQUATIONS

LANG XIA

While dierential equations (of ordinary or partial) comprise an essential part in the eld of scientic studies and assist in formulating science mathematically, they also complicate the understanding of physical phenomena with formidable forms and intricate solutions. The formulation of a dierential equation for a physical phenomenon is not easy, and the general solutions to the dierential equation are even hard to obtain. However, the solution set of a given dierential equation may be treated as an embedded manifold of a submanifold in the Euclidean space. From this point of view, the well-developed geometric theory could be employed to study dierential equations. Study dierential equations using modern geometries could be called geometric analysis of dierential equations. In this brief, by sacricing some mathematical rigorousness, the development of geometric analysis of dierential equation is presented gradually. This preliminary idea is then generalized by the employment of the modern jet machinery, which the Lie symmetry methods are then unied and included naturally.

Let's begin with a rst order ordinary dierential equation (ODE) in the form of (1)

d dx u (x) = u (x) + x
which is commonly seen in ODE textbooks. It represents a linear non-homogeneous equation that can be solved by superposing its homogeneous and particular solutions. Actually, we can inspect the behaviors and properties of the solutions without solving the equation by the employment of the following geometrical technique: at each point (x 0 , u 0 ) assign a vector (1, u 0 + x 0 ) associated to the equation (it's nothing but the slope of the solution). The set of the vectors forms a vector eld. We then visualize the vector eld by drawing each vector (1, u + x) at the point (x, u), of which the nal results are shown in Figure 1. The resultant vector eld is also called the direction eld. From the above construction, we may know that the possible solutions u(x) to Eq.( 1) must be the curves (trajectories, solid curves in Figure 1) tangent to the vectors. We can readily nd how the solution curves evolve in the gure. Thus, the dierential equation ( 1) determines a straight line with a slope equal to u (x)+x at each point (x, y). This is how the solution of a dierential equation is visualized without knowing its solution. This very preliminary example demonstrates what the idea of geometric analysis of dierential equation is.

The vector eld can be expressed in the local coordinate as (2) Usually, a dierential equation that can be resolved with respect to its highest order, e.g., written as u (x) = f (x, u), is considered as in the normal form. More frequently, most of dierential equations are in the implicit form, e.g., F (x, u, u) = 0 such that the above method is not convenient in dealing with them. For example, consider the following implicit dierential equation (3)

X = 1 • ∂ ∂x + (u + x) • ∂ ∂u
F (x, u, u) = x 2 + u 2 + u2 -1 = 0
If we want to analyze Eq.(3) by the aforementioned technique, rst, the equation should be converted into the normal form, that is

(4) u = ± 1 -x 2 -u 2
which brings an issue in. Because, at each point (x 0 , u 0 ) (assume

x 2 0 + u 2 0 < 1) the vector 1, ± √ 1 -x 2 -u 2 has
two dierent values, and the integral curve cannot be identied uniquely. Thus it still does not guarantee the eectiveness of the above geometric approach even by converting the implicit equation to its normal form.

However, in analytical geometry we do visualize implicit curves without using an explicit form. It indicates that we may study the above equation in this way: Let p = u be an independent variable, then the dierential equation (3) becomes an algebraic equation x 2 + u 2 + p 2 -1 = 0, which actually plots a sphere E = S 2 (1) being embedded in R 3 . The solution u (x) is the set of injective curves on the sphere

A BRIEF INTRODUCTION TO THE GEOMETRIC ANALYSIS OF DIFFERENTIAL EQUATIONS3

parametrized by the equations of (5)

⎧ ⎪ ⎨ ⎪ ⎩ x = x u = u (x) p = u (x)
with the following restriction (6)

p = d dx u (x)
Take the derivative of the curve dened by Eq.( 5), we will have

(7) ⎧ ⎪ ⎨ ⎪ ⎩ ẋ = 1 u = p ṗ = ü
then the tangent vectors attached to the curve can be written as (8)

X = ∂ ∂x + p ∂ ∂u + ü ∂ ∂ u
At a given point θ = (x 0 , u 0 , p 0 ), the tangent vector is (9)

X(θ) = 1 • ∂ ∂x + p 0 ∂ ∂u + ü(0) ∂ ∂ u
of which the linear combinations span a two dimensional plane C θ such that X ⊂ C θ . Since the third component ü(0) is arbitrary (at the point θ, it corresponds to the curvature of Eq.( 5)), to obtain the equation for the plane C θ , we can choose the projection of X onto the xu plane, that is in the form of (10)

Y = 1 • ∂ ∂x + p 0 ∂ ∂u + 0 ∂ ∂ u
The normal vector n of the plane C θ then can be calculated by ( 11)

n = X × Y
with being written in the vector form of

n = (p 0 , -1, 0)
thus the plane C θ has the following form (12)

p 0 (x -x 0 ) -(u -u 0 ) = 0
which is called the Cartan plane (see the demonstration in Figure 2). Geometrically, the projection of the Cartan plane onto the xu plane is exactly the tangent line of the solution curve (the bottom graph in Figure 2). The set of all Cartan planes is called the Cartan distribution. Note that Eq.( 12) may be changed into the form of (13)

δu -p 0 δx = 0
indicating that the Cartan distribution belongs to the kernel of the following oneform in the local coordinate ( 14)

ω = du -pdx
e.g., C θ ⊂ ker{ω} . This is because ω(X) = 0 gives rise to exactly the Eq.( 13). The set of C ∩T E forms a direction eld on E, which is called the characteristic vector eld. (In this example, a vector in the characteristic eld is the intersection of the Cartan plane C θ and the tangent plane T θ in the top graph of Figure 2).

From the above discussion, we may know that the Cartan distribution is exactly what determines the integral curves. This result can be seen more explicitly by dening the following map

j 1 f : R → R × R 2 x -→ (x, u = f (x) , p = u (x))
and the corresponding pull-back map

j 1 f * x : T * f (x) R × R 2 → T * x R ω -→ j 1 f * x ω = ω (f (x)) then (15) j 1 f * x ω = j 1 f * x (du -pdx) = df (x) - df (x) dx dx = 0
Therefore, the solutions of the equation are integral curves (Figure 2 just shows one integral curve) on the sphere that annihilate the Cartan distribution C. We can easily see that the Cartan distribution is a geometric structure naturally endowed on the sphere. It determines the integral curve on the sphere.

We may abstract the above ideal with considering a more general rst order dierential equation ( 16)

F x 1 , • • • , x n , u, u 1 , • • • u n = 0 where u = f x 1 , • • • , x n and u i = ∂u/∂x i .
We can always assume that

x 1 , • • • , x n , u, u 1 , • • • u n ∈ R 2n+1 = R n × R × R n ,
which extends the rst n + 1 coordinates to 2n + 1. Actually, the equation denes a submanifold E in the product manifold R 2n+1 written as

E = x 1 , • • • , x n , u, u 1 , • • • u n ∈ R 2n+1 | F x 1 , • • • , x n , u, u 1 , • • • u n = 0
The projection of the integral manifolds in E onto the hyperplane R n × R gives rise the solution u x 1 , • • • , x n . e.g. the hyperplane x 1 , • • • , x n , u (or in Figure 2, the integral curve projected onto xu plane).

On the other hand, even more frequently, we can consider the graph of the solution ( 17)

Γ (f ) := σ x 1 , • • • , x n = x 1 , • • • , x n , u x 1 , • • • , x n ∈ R n × R
as a product manifold, which is similar to the starting example of how we plotted the vector eld for the Eq.( 1). Due to the graph containing both domain and codomain, it renders a natural way to deal with geometric ob jects.

The above geometric analysis can be generalized by using the machinery of jet bundles. let x 1 , • • • , x n ∈ M be a smooth manifold, E another smooth manifold (dim(E ) > dim(M )), as well as a smooth map π : E → M . Thus the triple (E, π, M ) forms a bered manifold. If we impose the condition E = M × R here, then (E, π, M ) becomes a ber bundle. Meanwhile, M is called the base manifold, E the total space and π the projection. For example, to the Eq.( 16), we may write that and the section of the ber bundle is dened as a set of graphs (since the functions at each point form a ring, the section is also a C ∞ (M )-module) Γ (M ) := {σ : π • σ = id M } here σ = (x, f (x)) in our example of Eq.( 16). If we also count in the derivatives as additional dimensions, the ber bundle is then extended to the so-called rst jet bundle , of which the coordinate can be expressed as

M = R n E = M × R π : E → M ; with π • σ = id M
x 1 , • • • , x n , u, u 1 , • • • u n
Remember that the Taylor expansion can also be seen as a jet bundle at a point in the base manifold.

An inverse function of the projection can be written as

j 1 x σ : x 1 , • • • , x n -→ x 1 , • • • , x n , u, u 1 , • • • u n
However, there may be many dierent sections in Γ (M ) such that their rst derivatives are equal. To avoid this ambiguity, we require that j 1

x σ ∼ j 1 x τ in T x (Γ (M )) i σ x = τ x and ∂σ ∂x i | x = ∂τ ∂x i | x .
Then the 1-jet ber is denoted as j 1

x M := j 1 x σ| σ ∈ Γ (M ) and the 1-jet bundle is the disjoint union of the 1-jet ber over all points in the base manifold M

J 1 M = x∈M j 1 x M
Thus the rst prolonged jet bundle is the triple J 1 M, π 1 , M , and the 0-jet bundle is J 0 M = E, π, M . They are related by

J 1 M π1,0 ---→ J 0 M π 1 ↓ ↓π M ---→ M
Following the jet bundle construction, the k-th prolonged jet bundle is denoted by the triple

J k M, π k , M . Since (u 1 , • • • u n ) ⊂ T * M , we have J 1 M = T * M × R.
We may better understand the language of the jet bundles by simply expressing the above relations in the local coordinate.

J 0 M = x 1 , • • • , x n , u J 1 M = x 1 , • • • , x n , u, u 1 , • • • u n and the corresponding projections π 1 : J 1 M → M x 1 , • • • , x n , u, u 1 , • • • u n -→ x 1 , • • • , x n π 1,0 : J 1 M → J 0 M x 1 , • • • , x n , u, u 1 , • • • u n -→ x 1 , • • • , x n , u (π 1,0 ) * : T J 1 M → T J 0 M x 1 , • • • , x n , u 1 , • • • u n , u 11 , • • • u 1n , • • • u nn -→ x 1 , • • • , x n , u 1 , • • • u n ∀α = (a, u = f (a)) ∈ J 0 M ,
we can dene a contact element θ = (α, l), where l = T a Γ ⊂ T α J 0 M . The base for each space will be

T α (M ) = span ∂ ∂x 1 , • • • , ∂ ∂x n α T * α (M ) = span dx 1 , • • • , dx n α T α J 0 M = span ∂ ∂x 1 , • • • ∂ ∂x n , ∂ ∂u α T α J 1 M = span ∂ ∂x 1 , • • • ∂ ∂x n , ∂ ∂u , ∂ ∂u 1 , • • • ∂ ∂u n α
So, the Cartan plane can be dened as

C θ = X ∈ T θ J 1 M | (π 1,0 ) * (X) ∈ l θ
in a sense, the Cartan distribution is the eld of the Cartan planes in T θ J 1 M , of which the pro jection on J 0 M is the direction vector eld that tangent to the solution u = f x 1 , • • • , x n . In terms of Eq.( 14), we can also dene the Cartan distribution as

C = ker {ω}

By far, we have known that the Cartan distribution is determined by the oneform on J 1 M . We can also derive its expression in the local coordinate. Let X ∈ T θ J 1 M with the expression in the local coordinate as ω π1,0(θ) X i ∂ ∂x i + u i X i ∂ ∂u = 0

(25)

X i ∂ ∂x i + u i ∂ ∂u ω π1,0(θ) = 0

Figure 1 .

 1 Figure 1. The vector eld (arrows) of Eq.(1) and its 3 dierent trajectories (solid curves)

Figure 2 .

 2 Figure 2. Illustrations of Cartan plane (C θ ), tangent plane (T θ ), and tangent vector (X ) at the point θ on the sphere (blue dot). L is a trajectory (a solution is the projection of L onto xu plane demonstrated by the curve on the bottom graph).

X

  = X i ∂ ∂x i + η ∂ ∂u + η i ∂ ∂u i where η = u i X i (which is a plane tangent to the graph Γ (f )). When X ∈ C, we have (19) (π 1,0 ) * (X) ∈ l θ and(20)ω (X) = 0 thus from the above denition of Cartan plane, we have(π 1,0 ) * (X) = X i ∂ ∂x i + η ∂ ∂uremember we also have the relation(21) π * 1,0 ω θ (X θ ) = ω π1,0(θ) (π 1,0 ) * (X θ )and in terms of Eq.(20) ω θ (X θ ) = ω π1,0(θ) (π 1,0 ) * (X θ ) = 0 substitute the push-forward relation and η = u i X i into the above equation, we nd (24)

(First drafted in August

2014, langxia.org@gmail.com).

thus if X i = 0, we nally get (26) ω = duu i dx i which is also called the contact form. To this end, we have known that: Denition. A general solution to a given dierential equation is a maximal integral manifold of the Cartan distribution on the dierential equation.

Using the language of jet bundles, we can readily dene the point transformation as a dieomorphism φ 0 :

of which the lifting preserves the Cartan distribution C ,e.g. if there is a oneparameter group φ 0 t such that the vector eld

where (28)

and ( 29)

then we call X the Lie point symmetry.

Similarly, the contact transformation is a dieomorphism

which preserves the Cartan distribution C, and if there is a one-parameter group φ 1 t such that the vector eld (30)

then we call X the Lie contact symmetry. Since (33)

the point transformation can be seen as a contact transform lifted from the transformation of J 0 M . The Lie eld is just a vector eld X on J k M corresponding to the one-parameter group φ k t , where k = 0, 1, • • • , n.