Gerald Petz
email: gerald.petz@fh-steyr.at

Werner Wetzlinger
email: werner.wetzlinger@fh-steyr.at

Dietmar Nedbal
email: dietmar.nedbal@fh-steyr.at

Improving Language-Dependent Named Entity Detection

Keywords: Entity Recognition, Entity Detection, Language-dependent, Dataset Development, Gold Standard, NER

Named Entity Recognition (NER) and Named Entity Linking (NEL) are two research areas that have shown big advancements in recent years. The majority of this research is based on the English language. Hence, some of these improvements are language-dependent and do not necessarily lead to better results when applied to other languages. Therefore, this paper discusses TOMO, an approach to language-aware named entity detection and evaluates it for the German language. This also required the development of a German gold standard dataset, which was based on the English dataset used by the OKE 2016 challenge. An evaluation of the named entity detection task using the web-based platform GERBIL was undertaken and results show that our approach produced higher F1 values than the other annotators did. This indicates that language-dependent features do improve the overall quality of the spotter.

Introduction

The recognition of named entities is an important starting point for many tasks in the area of natural language processing. Named Entity Recognition (NER) refers to methods that identify names of entities such as people, locations, organizations and products [START_REF] Petasis | Large-scale Sentiment Analysis for Reputation Management[END_REF][START_REF] Derczynski | Analysis of Named Entity Recognition and Linking for Tweets[END_REF]. It is typically broken down into the two subtasks entity detection (or "spotting") and entity classification. In many application scenarios, however, it is not only of interest which types of entities are contained in a text, but also how the entities can be semantically linked to a knowledge base. The task of correctly disambiguating and linking the recognized named entities in a text into a knowledge base with an external definition and description is referred to as Named Entity Linking (NEL) [START_REF] Rizzo | Benchmarking the extraction and disambiguation of named entities on the semantic web[END_REF]. The overall goal is to make sense of data in the context of an application domain [START_REF] Holzinger | Introduction to Machine Learning and Knowledge Extraction (MAKE)[END_REF]. The whole pipeline (including the aforementioned tasks of NER and NEL) is strongly dependent on the knowledge base used to train the named entity extraction algorithm [START_REF] Rizzo | NERD meets NIF: Lifting NLP extraction results to the linked data cloud[END_REF]. Most approaches for linking entities leverage on the use of Wikipedia (wikipedia.org), Dbpedia (dbpedia.org), Freebase (freebase.com) or YAGO (yagoknowledge.org) as the knowledge base. Although widely used and the largest online encyclopedia with millions of articles, Wikipedia may not be sufficient for more specific domains and contexts. For example, in the German Wikipedia, only some large Austrian organizations are represented, names of persons are rare, etc. Moreover, the English Wikipedia does not hold this specific information either. Among others, Piccinno and Ferragina [START_REF] Piccinno | From TagME to WAT: A New Entity Annotator[END_REF] recognized that recent research tends to focus its attention on the NEL step of the pipeline by trying to improve and implement new disambiguation algorithms. However, ignoring the issues raised by entity recognition leads to the introduction of many false positives, which provoke a significant loss in the overall performance of the system. It would therefore be better to first try to improve the quality of the NER spotter.

Another problem area relates to differences in the language itself. It has been acknowledged that linguistically motivated and thus language aware spotting methods are more accurate than language independent methods [START_REF] Daiber | Improving Efficiency and Accuracy in Multilingual Entity Extraction[END_REF]. The German language has a lot of differences for example in the use of upper and/or lowercase, compound nouns or hyphens to concatenate nouns. However, improvements in a certain language usually come at the expense of ease of adaptation to new languages. In addition, the established NER/NEL challenges and tasks of the scientific community like the OKE challenges [START_REF] Nuzzolese | Open Knowledge Extraction Challenge[END_REF], the NEEL challenge series [START_REF] Rizzo | Lessons Learnt from the Named Entity rEcognition and Linking (NEEL) Challenge Series[END_REF], or the ERD challenges [START_REF] Carmel | ERD'14: Entity Recognition and Disambiguation Challenge[END_REF] are in the English language and therefore language-dependent improvements are often not in the focus of the research.

Moreover, the results from different tools need to be comparable against certain quality measures (cf. Section 4.1) based on the same dataset. Frameworks addressing the continuous evaluation of annotation tools such as GERBIL [START_REF] Usbeck | GERBIL -General Entity Annotator Benchmarking Framework[END_REF][START_REF] Röder | GERBIL's New Stunts: Semantic Annotation Benchmarking Improved[END_REF] can be used for comparison, but evaluation datasets provided by GERBIL as "gold standards" are only available for the English language as well.

The objective of this paper therefore is to (i) develop an approach for language-aware spotting and (ii) to evaluate the proposed spotting approach for the German language.

After an analysis of the state of the art in spotting methods in general (Section 2), the paper focuses on possibilities to optimize the spotter for a certain language in Section 3. In Section 4, evaluation measures are discussed, followed by an analysis of available datasets for evaluation. Additionally, we show how a German dataset was developed and used for evaluation purposes. Section 5 presents results of the experiments and final conclusions are drawn in Section 6.

State of the Art in Entity Detection (Spotting)

As mentioned above the entity detection ("spotting") is an important task in the area of NEL; a couple of authors emphasize the importance of a correct entity spotting in order to avoid errors in later stages of the entity linking task. [START_REF] Piccinno | From TagME to WAT: A New Entity Annotator[END_REF][START_REF] Hachey | Evaluating Entity Linking with Wikipedia[END_REF] Several approaches to the spotting task can be identified in the literature:

 NER tagger. Some tools and approaches rely on existing implementations of NER taggers such as Standford NER tagger or OpenNLP Named Entity Recognition in order to spot surface forms of entities [START_REF] Mendes | DBpedia Spotlight: Shedding Light on the Web of Documents[END_REF][START_REF] Mendes | DBpedia: A Multilingual Cross-domain Knowledge Base[END_REF][START_REF] Rizzo | NERD: evaluating named entity recognition tools in the web of data[END_REF][START_REF] Hoffart | Robust Disambiguation of Named Entities in Text[END_REF][START_REF] Charton | Automatic Semantic Web Annotation of Named Entities[END_REF]. The Standford NER tagger is an implementation of linear chain Conditional Random Field (CRF) sequence models, the OpenNLP NER is based on a Maximum Entropy model.  POS tags and rules. A couple of authors use part of speech (POS) taggers and/or several rules in order to identify named entities [START_REF] Eckhardt | Entity Recognition Based on the Co-occurrence Graph and Entity Probability[END_REF][START_REF] Zhao | Combining POS Tagging, Lucene Search and Similarity Metrics for Entity Linking[END_REF][START_REF] Zhang | Towards Entity Correctness, Completeness and Emergence for Entity Recognition[END_REF][START_REF] Moro | Entity Linking meets Word Sense Disambiguation: a Unified Approach[END_REF][START_REF] Leaman | BANNER: an executable survey of advances in biomedical named entity recognition[END_REF][START_REF] Cucerzan | Large-Scale Named Entity Disambiguation Based on Wikipedia Data[END_REF][START_REF] Dojchinovski | Entityclassifier.eu: Real-Time Classification of Entities in Text with Wikipedia[END_REF][START_REF] Kliegr | Linked hypernyms: Enriching DBpedia with Targeted Hypernym Discovery[END_REF][START_REF] Tonelli | Wikipedia-based WSD for multilingual frame annotation[END_REF][START_REF] Goudas | Argument Extraction from News, Blogs, and Social Media on AI, SETN 2014[END_REF][START_REF] Ritter | Named Entity Recognition in Tweets: An Experimental Study[END_REF][START_REF] Olieman | Entity linking by focusing DBpedia candidate entities[END_REF][START_REF] Chiu | NTUNLP approaches to recognizing and disambiguating entities in long and short text at the ERD challenge 2014[END_REF][START_REF] Barrena | UBC entity recognition and disambiguation at ERD 2014[END_REF][START_REF] Noraset | WebSAIL wikifier at ERD 2014[END_REF][START_REF] Lipczak | Tulip: Lightweight Entity Recognition and Disambiguation Using Wikipedia-Based Topic Centroids[END_REF]. The rules range from simple rules such as "capitalized letter" (if a word contains a capitalized letter the word will be treated as a spot), stop word lists, "At Least One Noun Selector"-rule to complex, combined rules.  Dictionary based techniques. The majority of approaches leverage techniques based on dictionaries [START_REF] Piccinno | From TagME to WAT: A New Entity Annotator[END_REF][START_REF] Eckhardt | Entity Recognition Based on the Co-occurrence Graph and Entity Probability[END_REF][START_REF] Chiu | NTUNLP approaches to recognizing and disambiguating entities in long and short text at the ERD challenge 2014[END_REF][START_REF] Petasis | Sentiment Analysis for Reputation Management: Mining the Greek Web[END_REF][START_REF] Ceccarelli | Dexter: An Open Source Framework for Entity Linking[END_REF][START_REF] Ceccarelli | Dexter 2.0 -an Open Source Tool for Semantically Enriching Data[END_REF][START_REF] Ferragina | TAGME: On-the-fly Annotation of Short Text Fragments (by Wikipedia Entities)[END_REF][START_REF] Mihalcea | Wikify!: Linking Documents to Encyclopedic Knowledge[END_REF][START_REF] Ratinov | Local and Global Algorithms for Disambiguation to Wikipedia[END_REF][START_REF] Agirre | Personalizing PageRank for word sense disambiguation[END_REF][START_REF] Agirre | Random Walks for Knowledge-Based Word Sense Disambiguation[END_REF][START_REF] Milne | An open-source toolkit for mining Wikipedia[END_REF][START_REF] Kemmerer | The neofonie NERD system at the ERD challenge 2014[END_REF][START_REF] Gangemi | Semantic Web machine reading with FRED[END_REF]. The structure of Wikipedia provides useful features for generating dictionaries: ─ Entity pages: Each page in Wikipedia contains a title (e.g. "Barack Obama") that is very likely the most common name for an entity. ─ Redirect pages: Wikipedia contains redirect pages for each alternative name of an entity page. E.g. "Obama" is a redirect page to "Barack Obama". ─ Disambiguation pages: Disambiguation pages in Wikipedia are used to resolve conflicts with ambiguous article titles. E.g. "Enterprise" may refer to a company, to aircrafts, to Star Trek, and many more. Disambiguation pages are very useful for extracting aliases and abbreviations. ─ Bold phrases: Bold phrases in the first paragraph of a Wikipedia entry can contain useful information such as abbreviations, aliases or nicknames. E.g. the bold phrase in the page "Barack Obama" contains the full name ("Barack Hussein Obama II"). ─ Hyperlinks in Wikipedia pages: Pages in Wikipedia usually contain hyperlinks to other pages; the anchor texts of these hyperlinks may provide synonyms and other name variations.  Methods based on search engines. Some authors try to use web search engines such as Google to identify candidate entities [START_REF] Lehmann | LCC Approaches to Knowledge Base Population at TAC 2010[END_REF][START_REF] Han | NLPR_KBP in TAC 2009 KBP Track: A Two-Stage Method to Entity Linking[END_REF][START_REF] Dredze | Entity Disambiguation for Knowledge Base Population[END_REF][START_REF] Monahan | Cross-Lingual Cross-Document Coreference with Entity Linking[END_REF].

 Computational techniques. A couple of authors leverage heuristic based methods or machine learning methods. Some approaches expand the surface forms by searching the textual context based on heuristic pattern matching [START_REF] Lehmann | LCC Approaches to Knowledge Base Population at TAC 2010[END_REF][START_REF] Han | NLPR_KBP in TAC 2009 KBP Track: A Two-Stage Method to Entity Linking[END_REF][START_REF] Jain | Acronym-Expansion Recognition and Ranking on the Web[END_REF]. Other authors use N-Grams [START_REF] Chiu | NTUNLP approaches to recognizing and disambiguating entities in long and short text at the ERD challenge 2014[END_REF][START_REF] Noraset | WebSAIL wikifier at ERD 2014[END_REF][START_REF] Kemmerer | The neofonie NERD system at the ERD challenge 2014[END_REF][START_REF] Hakimov | Named Entity Recognition and Disambiguation Using Linked Data and Graph-based Centrality Scoring[END_REF][START_REF] Milne | Learning to Link with Wikipedia[END_REF], others experiment with CRF [START_REF] Dojchinovski | Entityclassifier.eu: Real-Time Classification of Entities in Text with Wikipedia[END_REF][START_REF] Kliegr | Linked hypernyms: Enriching DBpedia with Targeted Hypernym Discovery[END_REF], Topic Modeling [START_REF] Han | Le Sun: A Generative Entity-mention Model for Linking Entities with Knowledge Base[END_REF][START_REF] Han | Le Sun: An Entity-topic Model for Entity Linking[END_REF], Naïve Bayes and Hidden Markov Models [START_REF] Carpenter | Phrasal Queries with LingPipe and Lucene: Ad Hoc Genomics Text Retrieval[END_REF]. Last but not least one can find approaches based on Finite-state machines [START_REF] Mendes | DBpedia Spotlight: Shedding Light on the Web of Documents[END_REF][START_REF] Mendes | DBpedia: A Multilingual Cross-domain Knowledge Base[END_REF][START_REF] Olieman | Entity linking by focusing DBpedia candidate entities[END_REF][START_REF] Lipczak | Tulip: Lightweight Entity Recognition and Disambiguation Using Wikipedia-Based Topic Centroids[END_REF].

The majority of the papers use dictionary approaches. Nevertheless, the above mentioned approaches are usually combined, e.g. [START_REF] Piccinno | From TagME to WAT: A New Entity Annotator[END_REF] leverages OpenNLP NER, a dictionary approach based on Wikipedia with utilization of several features such as anchor texts, redirect pages, etc. The authors usually provide measures (recall, precision, F1) of the effectiveness of their approaches; unfortunately, these measures cannot be directly compared because usually different datasets are used and the approaches are optimized towards these datasets.

TOMO Approach to Optimize Spotter for the German Language

This section details our approach to optimizing a spotter ("TOMO") for the German language with a focus on the spotting phase within the entity linking pipeline . The base system used is Dexter [START_REF] Ceccarelli | Dexter: An Open Source Framework for Entity Linking[END_REF][START_REF] Ceccarelli | Dexter 2.0 -an Open Source Tool for Semantically Enriching Data[END_REF], an open-source framework (available at https://github.com/dexter/dexter) that implements a dictionary spotter using Wikipedia content. Fig. 2 shows the approach comprising the construction and the annotation process using a dictionary.

Fig. 2: Basic TOMO Architecture

The annotation process involves a spotter and a disambiguator with an annotated text as output. The spotter detects a list of candidate mentions in the input text, and retrieves for each mention a list of candidate entities [START_REF] Ceccarelli | Dexter: An Open Source Framework for Entity Linking[END_REF]. When spotting a text, individual fragments or words from the text ("shingles") are compared with the dictionary of up to six words ("n-grams"). Before being able to use the dictionary for NER, the dictionary needs to be filled with known entities first. Therefore, each Wikipedia article is processed using the title of the article as well as all internal links (anchors within Wikipedia) as spots for the dictionary. In addition, the measures of mention frequency (mf), link frequency (lf) and document frequency (df) are calculated and stored as well. Both in the construction of the dictionary and in the annotation of a text based on this dictionary the text fragments (shingles and known entities) go through a cleaning pipeline with a series of replacements. The cleaning pipeline in pseudocode is as follows: foreach (article in knowledgebase) listOfSpots = preprocess(getTitle(article)) listOfSpots = preprocess(getAnchors(article)) calculateMeasures(listOfSpots) preprocess(textfragment) clean(textfragment) filter(textfragment) map(textfragment)

A "cleaner" performs a simple transformation of a text fragment (e.g. transform a text to lowercase, remove symbols, remove quotes, unescape Javascript, clean parenthesis, etc.). A "filter" allows the removal of a given text fragment if it does not respect a filter constraint (e.g. delete text fragments that are below the threshold for commonness, have less than three characters, consist only of numbers or symbols, etc.). A "mapper" returns several different versions of the spot (e.g. a "quotes mapper" generates from [dave "baby" cortez] the spots [dave "baby" cortez], [baby], [dave cortez]) [START_REF] Ceccarelli | SpotManager[END_REF].

Moreover, for simplification purposes, many tools use lowercase filters. Full text search indices such as Lucene also imply such a lowercase behavior per default, which in many tasks (e.g. search engine querying, microblogging analysis) makes sense. In our setting, lowercase simplification is responsible for introducing several spotting errors (e.g. the sentence "the performance is worse": the word "worse" translates to "schlechter" in German and the spotter identifies this word as a candidate entry for the Wikipedia page "Carl Schlechter"). In German language, only nouns and proper names are written with capitalized initial letters.

Language-aware preprocessing pipeline

In a setting where typing errors are relatively rare (e.g. in press releases, formal documents) the application of a case sensitive setting is therefore a reliable and straightforward approach to increase the precision of the spotter for the German language [START_REF] Neumann | An information extraction core system for real world German text processing[END_REF].

Another important aspect of a language-aware approach is the correct usage of the code page. For the English language, the US-ASCII code page is the preferred setting, as it uses less space than other code pages. In the German language, many named entities contain non US-ASCII characters, like umlaute or the German eszett. Using US-ASCII filters, these characters are replaced by their English representation (umlaut a gets replaced by an "a", etc.). This sometimes changes the whole meaning of the word, as the English replacements are also used in German language and this gets even worse in combination with lowercase filtering. E.g. the sentence "we made this", with its German translation "Wir haben das gemacht": the word "made" translates to "gemacht" in German and this is disambiguated to "Gemächt" (the male genitalia). The UTF-8 code page can be used as a solution to this problem.

Additionally, some minor issues may occur due to differences in the language of the Wikipedia syntax itself. For instance, it is possible to link images within Wikipedia with the common English terms "File:" or "Image:", but the German Wikipedia additionally allows the deprecated terms "Datei:" or "Bild:" as well. Such filters therefore also need to be aware of differences in the German language in order to improve spotting.

Evaluation Measures and Datasets

In this section, we discuss the evaluation of spotting named entities in the German language. This includes which measures, tools and datasets to use.

Measures and Benchmarking

To ensure comparability across different NER and NEL system evaluations the most common measures are precision, recall, F1 and accuracy.

Precision. Precision considers all spots that are generated by the system and determines how correct they are compared to a gold standard. Consequently, the precision of a spotting system is calculated as the fraction of correctly spotted entity mentions compared to all spotted mentions generated by a particular system.

(

) 1
Recall. Recall is a measure that describes how many of the spots of a gold standard are correctly identified by a system. It is the fraction of correctly spotted entity mentions by a particular systems compared to the all entity mentions that should be spotted according to a selected gold standard.

(

) 2
F1. To generate a single measure for a system from recall and precision, the measure F1 was developed. It is defined as the harmonic mean of precision and recall as shown in equation 3.

* *

( 3 )

GERBIL.

As also minor differences between these measures exist, we use the webbased benchmarking system GERBIL (gerbil.aksw.org) to evaluate these measures for our system and compare them with others. GERBIL is an entity annotation system that provides a web-based platform for the comparison of annotators [START_REF] Usbeck | GERBIL -General Entity Annotator Benchmarking Framework[END_REF]. Currently it incorporates 13 annotators and 32 datasets for evaluating the performance of systems. The evaluation is done using uniform measuring approaches and well established measures like the aforementioned recall, precision and F1 [START_REF] Röder | GERBIL's New Stunts: Semantic Annotation Benchmarking Improved[END_REF]. Consequently, GERBIL can be used for benchmarking different annotators. External tools can be added to the GERBIL platform by providing an URL to a REST interface of the tool. Besides the integrated datasets, GERBIL allows for the use of user-specified datasets. As GERBIL is based on Natural Language Programming Interface Format (NIF), user-specified datasets also have to be uploaded using this format. Additionally, GERBIL provides Java classes for implementing APIs for datasets and annotators to NIF. Due to these features, GERBIL is also used by challenges (e.g. OKE challenge) as platform for evaluating the performance of contestants.

Dataset

To test an entity linking system a gold standard dataset must be provided. This dataset has to include all sentences to analyze, spots to be linked and links to a knowledge base for correct disambiguation. Systems are then ranked by comparing the above-mentioned evaluation measures (recall, precision and F1) they score in relation to this dataset. There are already a number of English corpora to test entity recognition and entity linking systems. Some of them emerged from challenges that compare the results of multiple algorithms and systems to assess the performance of different approaches. For example the datasets of the OKE challenge as part of the European Semantic Web Conferences 2016 (2016.eswc-conferences.org), the NEEL challenge of the Microposts Workshop at the World Wide Web Conference 2016 (microposts2016.seas.upenn.edu), or the ERD challenge at the SIGIR 2014 Workshop ERD'14 (sigir.org/sigir2014) are publicly available.

Requirements and Review of Existing Datasets.

To test the performance of our approach to spot entities in the German language we had to select or develop a dataset. For this task, we defined the following requirements:

 The dataset has to be available in German to test the performance of the spotter for German texts.

 The dataset should be testable via the GERBIL web service. Thus, it should be already available in GERBIL or encoded in NIF format.  The dataset should be widely used, specifically by new systems, to be able to compare our results with leading systems and approaches.  The dataset should be independent of a certain domain (e.g. only articles about economics).  The content of the dataset should be comprised of natural language in encyclopedia entries or news. Specific content like tweets or queries were not of interest, since these datasets often have just very few spots with average entries per documents lower than 2.0.  The dataset should include co-references to evaluate the performance improvements of future enhancements of our system.

We examined existing datasets and their suitability for our requirements. Table 1 shows the results of this literature review which showed that nearly all available datasets are for the English language. The three German datasets found were not appropriate for our requirements because they were too domain specific (News-100, LinkedTV), possess only the "classic" named entities (persons, locations, etc.), had no co-references defined, and/or are not publicly accessible (GerNED). Since none of these datasets fitted our requirements, we decided to develop a new dataset to evaluate the spotter against German texts.

Development of a German gold standard dataset. We chose to develop a new German dataset based on the evaluation dataset of the OKE challenge 2016 ("OKE 2016 Task 1 evaluation dataset") for several reasons. Since the content of this dataset originated from Wikipedia articles, it covers a wide range of topics. Therefore, it also consists of natural language and not of tweets or search queries. Furthermore, documents are long enough to contain multiple spots (6.18 average entities per document) and they include co-references as well. Additionally, the English version of the dataset is coded in NIF format and is already integrated in GERBIL. Finally, with 55 documents and 340 entities, we considered this dataset to be of an appropriate length.

To develop the new dataset based on the dataset, we conducted a multi-step approach that consisted of the following tasks:

1. Identify all documents and included spots in the NIF file of the OKE 2016 Task 1 evaluation dataset. 2. Translate all documents in this dataset using Google Translate (translate.google.com) 3. Adjust the initial Google translation by improving German grammar, word order, etc. by native speakers. 4. Identify all English spots of dataset in the German translation. 5. Identify the corresponding entities in the German knowledge base (de.wikipedia.org). 6. Link the spots to the identified knowledge base entities using links in a HTML file. 7. Transform the HTML file to NIF using a converter. This process was not straightforward and a number of problems occurred that were mainly based on ambiguities in steps 5 and 6:  Because the English Wikipedia is more than twice as large as the German Wikipedia, some spots had no representation in the German knowledge base. This was mainly the case with persons (e.g. Andrew McCollum, James Alexander Hendler) and organizations (e.g. Kirkcaldy High School, American Association for Artificial Intelligence).  Literal translation by Google Translate led to surface forms that were wrong or unusual (e.g. "artificial intelligence researcher" was translated to "künstlicher Intelligenzforscher").  Translation by Google Translate led to a sentence structure and grammar that was sometimes unusual or incorrect for German sentences.  In some cases, it was not clear which term was the correct German translation of the English term in the specific context of sentences (e.g. "independent contractor" was translated to "unabhängiger Auftragnehmer" by Google Translate, but "freier Mitarbeiter" was considered to be the correct translation for the context of the sentence).  In a few cases, it was not clear to which entity in the German knowledge base an entity should be linked (e.g. the English term "treasury" can be translated based on traditional British or American interpretations of the word as "Finanzministerium" or "Schatzamt", but is now also used in its English form as a department of corporations.)

In order to cope with these uncertainties, three researchers (German native speakers) independently identified the corresponding German surface form for spot based on the translated text. For every spot that led to different surface forms or links the different solutions from the three authors were discussed and a majority decision was made by voting.

As a result, some spots were not available in the German knowledge base and therefore the resulting dataset has fewer spots than the original. Since not all systems currently support co-references, we developed two versions of the dataset. One with coreferences and one without co-references (15 documents incorporated a total of 24 coreferences). The resulting corpus can be downloaded here: https://github.com/HCSolu-tionsGesmbH/OKE-Challenge-German.

Experiments and Results

Based on the discussion described in Section 4 we built different test cases to evaluate the changes between a language independent (n/a) and an explicit German language (de) setting. In addition, we considered case sensitivity as a test case for our experiments and built a model based on a case sensitive (cs) and case insensitive (cis) setting. These model characteristics led to the four different test cases shown in Fig. 3.

Fig. 3: Test cases

Using the German case sensitive model, we experimented with the commonness threshold. We cut out 0%, 5%, 10%, 15% and 20% of spots with the lowest commonness and evaluated the resulting F1 scores. Results showed that using all spots (i.e. not cutting out any) led to the highest F1 score. Thus, this setting resulted in the highest recall without lowering the precision too much. Consequently, we tested all four test cases using this setting in GERBIL with our developed dataset that is based on the recent German Wikipedia dump from 2017/05/01. Table 2 shows the resulting scores for recall, precision and F1. The annotators TagMe 2, xLisa-NER and xLisa-NGRAM of GERBIL did not produce any results (the GERBIL experiment reported: "The annotator caused too many single errors.") and could not be evaluated. FRED produced the highest recall. As FRED aims at producing formal structure graphs from natural language text and is based on a dictionary comprising different knowledge bases including WordNet aims at a high recall.

On the other side it automatically translates the input text beforehand which may lead to a decrease in precision [START_REF] Gangemi | Semantic Web machine reading with FRED[END_REF]. FOX combines the results of several state-of-the-art NER tools by using a decision-tree-based algorithm which performed best on the precision measure [START_REF] Speck | Named Entity Recognition using FOX[END_REF][START_REF] Speck | Ensemble Learning for Named Entity Recognition[END_REF]. In addition, the tool is capable of automatically detecting German language text input. Results also show that the TOMO approach using the German language setting in combination with the case sensitive model achieved the highest F1 score among all tested annotators.

Conclusions and Future Work

The paper discusses an approach for language-aware spotting and evaluates the proposed spotting approach for the German language. The results indicate that languagedependent features do improve the overall quality of the spotter. This is necessary, because errors introduced in the spotting phase have an effect on the disambiguation step and can hardly be corrected. A limitation of this work is that the performance metrics of TOMO vs. other systems are only partially comparable, because the annotators were either developed only for the English language or do not take into account any language specifics. However, we were able to show that language-dependent features improve spotting quality. With the availability of a dataset in German and English language, it is possible to directly compare the performances of the systems for different languages.

When the authors of this paper developed the German corpus a lot of discussions about which surface forms should be linked to the knowledge base arose. For example this text (taken from OKE 2016 Task 1 evaluation dataset) contains several links (shown as underlined words): "Ray Kurzweil grew up in the New York City borough of Queens. He was born to secular Jewish parents who had emigrated from Austria just before the onset of World War II." It is not quite clear, why "parents" are linked to an entity, but some text fragments that are probably more in need of explanation such as "Jewish" or "World War II" are not spots. Wikipedia contains a separate page that provides guidelines for linking. These guidelines suggest for example, not to link everyday words, but to link to other articles that will help the reader to understand the context more fully [START_REF]Wikipedia:Manual of Style/Linking[END_REF]. However, every gold standard obviously represents a certain way of thinking. Furthermore, performing well on a certain gold standard just means the system replicates a certain way of thinking very well.

Further research work should include a discussion and development of guidelines or rules which terms should be annotated in a gold standard dataset in order to align the different evaluation datasets. Furthermore, a population of a cross-language and crossdomain gold standard in order to evaluate annotation systems for different purposes would be of value for the community.

Fig. 1 :

 1 Fig. 1: Illustration for the entity linking task

Table 1 .

 1 Comparison of gold standard datasets

				Co-		Linked	Avg. En-
	Datasets	Type	Lang	Refs Docs	Entities	tity/ Doc
	ACE2004	news	en			57	257	4.44
	AIDA-Yago2/CoNLL	news	en		231 4485	19.97
	AQUAINT	news	en			50	727	14.54
	Dbpedia Spotlight	news	en			58	330	5.69
	Derczynski -Broad Twitter Corpus	tweets	en		9551 12117	1.27
	ERD2014	queries	en			91	59	0.65
	GERDAQ	queries	en		992 1706	1.72
	IITB	webpages	en		103 11249 109.22
	KORE50	news	en			50	144	2.86
	MSNBC	news	en			20	658	32.50
	Microposts2014 test dataset	tweets	en		1165 1458	1.25
	Microposts2015 test dataset	tweets	en		2027 2382	1.18
	Microposts2016 test dataset	tweets	en		3164	738	2.46
	N3-RSS-500	news	en		500 1000	1.00
	N3-Reuters-128	news	en		128	880	4.85
	OKE 2015 Task 1 evaluation dataset encyclopedia en x	101	664	6.57
	OKE 2015 Task 2 evaluation dataset encyclopedia en x		99	99	1.00
	OKE 2016 Task 1 evaluation dataset encyclopedia en x		55	340	6.18
	OKE 2016 Task 2 evaluation dataset encyclopedia en			50	50	1.00
	N3-News-100	news	de		100 1547	15.47
	Meij	tweets	en		502	812	1.87
	LinkedTV	news	de		150 1346	8.97
	GerNED	news	de		2468 1664	0.67
	Ritter	tweets	en		2394 1672	0.70

Table 2 .

 2 Evaluation of spotting results

	Annotator	Recall	Precision	F1
	TOMO (n/a, cis)	0.8447	0.4575	0.5684
	TOMO (n/a, cs)	0.8599	0.4707	0.5823
	TOMO (de, cis)	0.8447	0.4575	0.5684
	TOMO (de, cs)	0.8569	0.4811	0.5866
	AIDA	0.3109	0.6516	0.3983
	Babelfy	0.4336	0.3599	0.3689
	DBpedia Spotlight	0.4139	0.5077	0.4197
	Dexter	0.3537	0.6176	0.4236
	Entityclassifier.eu NER	0.767	0.4888	0.5689
	FOX	0.4523	0.6777	0.4996
	FRED	0.9519	0.1878	0.303
	FREME NER	0.2353	0.3485	0.2643
	Kea	0.541	0.5953	0.5399
	WAT	0.495	0.6442	0.5163

Acknowledgements

This research was supported by HC Solutions GesmbH, Linz, Austria. We have to express out appreciation to Florian Wurzer, Reinhard Schwab and Manfred Kain for discussing these topics with us.

The TOMO Named Entity Linking is part of TOMO ® (http://www.tomo-base.at), a big data platform for aggregating content, analyzing and visualizing content.


Florian R Ong

Olivier Bourgeois

Topology effects on the heat capacity of mesoscopic superconducting disks

Keywords: numbers:, 74.78.Na (Mesoscopic and nanoscale systems), 74.25.Bt (Thermodynamic properties), 74.25.Dw (Superconductivity phase diagrams)

come

Topology effects on the heat capacity of mesoscopic superconducting disks

Introduction

The effect of the topology of mesoscopic samples on basic physical properties has been demonstrated [1]. On small systems many original effects [START_REF] Imry | Introduction to mesoscopic physics[END_REF] appear linked to the reduced size, such as enhanced surface effects, quantum phase coherence, phonon or electrons confinement or quantization of energy.

However properties of phase transitions likely to occur in nanosystems remain little explored. Heat capacity analysis is one of the most powerful tools to investigate every phase transition whatever its origin. For instance the critical temperature of a superconducting nanograin or the Curie temperature of a magnetic aggregate are hard to define through magnetization measurements since large fluctuations smear out the relevant phase transition.

In the field of mesoscopic superconductivity, physics is dominated by the fluxoid quantization.

For instance in the case of a thin doubly connected superconductor this constraint leads to physical properties that are flux periodic, the periodicity being Φ0 = h/2e, the superconducting flux quantum.

The Little-Parks effect [START_REF] Little | [END_REF]4,5] is one of its more obvious manifestation. Oscillations with periodicity n×Φ0 linked to the metastability of giant vortex states [6] have also been reported in doubly connected mesoscopic superconductors [7,8]. Dealing with simply connected mesoscopic superconductors the physics becomes richer and far less trivial, since fluxoid quantization do not directly leads to flux periodic properties. A pioneering work in the late 1980's by Buisson et al. [9] has shown the effect of edge states of micron-sized disks on the critical temperature and on the magnetization, both exhibiting oscillations as function of magnetic field. Later in the 1990's critical field of mesoscopic superconductors of various geometries and topologies have been studied both experimentally [1] and theoretically [10]. Ginzburg-Landau description is used and precise geometry of the system is taken into account through the boundary conditions of the order parameter on surfaces. In ref. [11] Geim et al. evidence size effects through magnetization measurements carried out on single disks far from Tc. The shape and the features of magnetization curves strongly depend on the value of the ratio R/ξ(T ), where R is the disk radius and ξ(T ) the superconducting coherence length. The order of the superconducting to normal (SN) phase transition is size dependent, and large radii disks exhibit first order phase transitions inside the superconducting region. These transitions are attributed to the entrance or exit of individual vortices.

In this Letter we present the first heat capacity study of phase transitions between vortex states in mesoscopic simply connected superconductors. Such measurements have been reported only once in an unfructuous attempt [12] to investigate the paramagnetic Meissner effect [13,14,15]. Yet calorimetric investigation of such systems can bring an innovative and complementary point of view on these systems [15,8,5], and recent progess in nanocalorimetry [8,16] make it now possible to measure the heat capacity of submicron-sized systems. In this work we show that the heat capacity Cp has a non trivial behaviour under an applied magnetic field H. Heat capacity curves versus T or H are found to depend on the number of vortices threading a single disk, and the entrance or exit of a vortex causes a Cp slip from one curve to another one. This effect is particulary striking when looking at the height ∆C(H) of the Cp jump at the SN transition when sweeping the temperature under fixed H. Indeed ∆C(H) is strongly modulated and encounters sharp non periodic discontinuities. To interpret this behaviour we draw the full phase diagrams of these giant vortex states describing the order parameter in the disks, both in increasing and decreasing fields, and we discuss the stability of these states under different situations of magnetic field history (zero field cooled and field cooled regimes).

System and apparatus

The sample studied in this work is composed of an array of N = 4.02 × 10 5 aluminium disks of radius R = 1.05 µm and of thickness e = 160 nm (see inset of fig. 1), giving a total aluminium mass of 603 ng. Despite their large number all the structures have the same geometric parameters within the measurement accuracy of the scanning electron microscope (≈ 10 nm). Furthermore the separation of 2 µm between two adjacent disks' edges ensures that the disks are non interacting and that the thermal signals are additive. These two points ensure that the measured Cp of the aluminium is N times the heat capacity of a single disk, all the disks being considered identical.

From an independent measure of the mean free path we estimate the Ginzburg Landau coherence length at zero temperature to be ξ(0) ≈ 0.17 µm, and so R/ξ(0) ≈ 6 and e/ξ(0) ≈ 1, a geometry that should prevent the appearance of multivortex states [17,18].

The N mesoscopic disks are patterned by electron beam lithography on a home-made specific heat sensor and aluminum is deposited by thermal evaporation. The sensor [19,8] is composed of a large (4mm×4mm) and thin (10 µm) silicon membrane suspended by twelve silicon arms. On this membrane, a copper heater and a NbN thermometer are deposited through regular photolithography.

Resistances of these thin film transducers are measured by a four point probe technique, enabling the measurement of the total heat capacity by ac-calorimetry [20]. This method has been largely described in numerous publications [8,20,21,19]. The calorimetry setup is cooled down to 0.55 K using a 3 He cryostat and is placed in the center of a large superconducting coil so that a homogenous magnetic field H can be applied perpendicular to the plane of the disks. We reach the quasiadiabatic conditions [20] for a heating current of frequency f ≈ 765 Hz at 1 K. The amplitude δTac of the oscillations of temperature is set in the range 1 mK to 20 mK depending on the working temperature and on the resolution needed for the measurements. The ac-calorimetry enables averaging of the measured signal: typically by averaging over 10 seconds, this apparatus allows measurements of heat capacity within 10 femto-Joule per Kelvin.

Heat capacity versus temperature under fixed magnetic field

In this section we study the influence of a constant perpendicular magnetic field on the shape of the superconducting transition (critical temperature, height, width). The sample is cooled after the field H is applied (FC regime) and we measure its heat capacity versus T . Our aim here is to study the dependence on H of the discontinuity ∆C of heat capacity at the SN transition. In a bulk geometry ∆C is monotonous with H whereas in a doubly connected superconductor Φ0-periodical modulations of ∆C(H) have been reported [5]. To access ∆C from our raw data we have to extract the contribution Cp,H(T ) of the superconducting aluminium to the total heat capacity (normal Al + superconducting Al + addenda) directly measured by our apparatus. This is achieved by substracting the heat capacity recorded under a high magnetic field that destroys superconductivity, as it was explained in ref. [5].

The heat capacity Cp,H=0(T ) of the superconducting aluminium under zero magnetic field is shown on fig. 1. Increasing or decreasing T scans show no difference whatever the applied field. From this curve we can extract the height ∆C(H) of the heat capacity discontinuity at the SN transition, and the critical temperature Tc which is defined at the middle height of the Cp jump.

We perform Cp,H(T ) measurements for several fixed magnetic fields H ranging from -0.5 to 7. have shown a modulation of ∆C(H), but no such discontinuities occured [5]. In ref [15] a modulated ∆C(H) was calculated for disks and no discontinuity was predicted; but the calculation was based on the linearised Ginzburg-Landau equations and the system was considered 2D. In our case, the thickness e is close to the penetration length of the field (λ(0) ≈ 70 nm) so we cannot assume a homogenous magnetic field inside the disks. A full 3D treatment of the Maxwell equations coupled to complete Ginzburg-Landau equations is required [22].

Fine measurements of the critical temperature of the disks reveal a non monotonic dependence of Tc(H) which is superimposed on a bulk-like linear trend T c,b (H). This feature is enhanced on fig. 2.c where this linear background as been removed. Discontinuities of ∆C(H) and of dTc/dH occur at the same magnetic fields. These modulations appear to be non periodical, contrary to the rings' case: the pseudoperiod of the first modulation corresponds to 1.9Φ0, where Φ0 = h/2e is the superconducting flux quantum calculated through the edge of a disk. As H increases the pseudoperiod decreases and tends towards a constant value of 1.5Φ0 at large H, which is in agreement with Tc(H) measurements performed by resistance mesurements in ref. [1]. This non-periodicity is due to the fact that fluxoid is quantized through a non rigid contour as it would be in a ring-like geometry [8,5]. At large magnetic fields the simply connected behaviour (constant periodicity) is recovered since the disks are composed of normal metal in the center, the superconductivity being confined on the edge of the superconductors. Indeed from ref. [1] it appears that parabola-like branches in the SN phase diagram Tc(H) (fig. 2.c) correspond to discrete L states of angular momentum, where the so called vorticity L is an integer. According to the geometry of the disks (large thickness) the authorized states minimizing the Ginzburg-Landau free energy are giant vortex states [6,17], whose order parameter has the form

Ψ(r) = f (z, r)exp(iLθ) = f (r)exp(iLθ) since the thickness e is smaller than ξ(T ) ≈ 1 µm at 1 K.
L is the number of single vortices threading a single disk and corresponds to the number of fluxoid quanta trapped in. A giant vortex state is a state of edge superconductivity. Indeed L vortices share the same core so superconductivity is located near the boundary of the disks. Such a state is not stable in a bulk geometry where vortex-vortex repulsion is not balanced by the vortex-edge repulsion that has to be taken into account in a mesoscopic superconductor. Thus discontinuities of ∆C(H) on fig. 2.b correspond to jumps between successive L states : as the sample is cooled below its critical temperature under an applied magnetic field H, the disks transit from normal state to a supercon-ducting state with vorticity L only depending on H, and remain in this L state as temperature is decreased. This issue will be made clearer in the following. Then as Cp,H(T ) is scanned by increasing T we eventually measure Cp,H,L(T ) at a fixed H and L.

The strong modulations of ∆C(H) on fig. 2.a show that heat capacity can be modulated by an external parameter, here the magnetic field. A consequence is that heat capacity looses its additive property in a mesoscopic superconductor, and more widely in a mesoscopic system. Indeed if one calculates the specific heat of a given mesoscopic superconductor by dividing its heat capacity by its mass, one would obtain a different result depending on the way the system is structured : simply or doubly connected, square or ring etc. Thus the intensive specific heat has no sense at the mesoscale. This original property of small systems has been proposed in ref. [5] but the large error bars due to the small measured mass made this point controversial. In the present work non additivity of the heat capacity in small systems is unambiguoulsy evidenced. Furthermore the influence of the topology on the heat capacity of a mesoscopic superconductor appears clearly when the shapes of ∆C(H) on disks (fig. 2.a) and on rings [5] are compared.

Another mesoscopic effect lies in the temperature broadening ∆T of the SN transitions on the Cp,H(T ) plots. ∆T is plotted versus H on fig. 3. For a small applied field (H < 3 mT) the width ∆T ≈ 30 mK is field independent and has two main origins: the measurement process itself which averages the heat capacity over ≈ 20 mK, and the slight dispersion in the geometrical parameters of the disks. As the field is increased beyond 3 mT, ∆T starts to grow: there is a crossover between a situation where Cp,H(T ) has a discontinuity at Tc and another one where the SN transition is smeared out. From ref. [15] we interpret this as a crossover from a bulk-like behaviour to a mesoscopic behaviour. Indeed at low field surface effects can be neglected since edges are taken into account only through a boundary condition that becomes irrelevant in the absence of field [15]; thus we expect a sharp discontinuity characteristic of second order phase transitions in bulk materials. At higher fields the constraint of vanishing supercurrents normal to surfaces becomes relevant and so a broadening of the transition appears which is characteristic of finite samples. From fig. 3 we state that this crossover occurs when two vortices have entered the disks (state L = 2); in zero field the mesoscopic nature of the disks cannot be evidenced by thermal measurements.

Heat capacity versus magnetic field under fixed temperature

In order to highlight the possibility of modulating the heat capacity of the mesoscopic disks with H, we perform the counterpart of previously described measurements: we fix the temperature before scanning the heat capacity Cp,T(H) versus magnetic field. An example is presented in fig. 4 for T = 0.64 K. We present Cp,T(H) in increasing and decreasing magnetic fields. Starting from zero field, the disks first remain in the Meissner state (L = 0) until H ≈ 3.3 mT where Cp suddenly shifts to a lower value: this Cp jump is the signature of the first order phase transition L = 0 → L = 1 : at this point a vortex penetrates each disk. Then as the field is increased one can observe a cascade of successive phase transitions L → L + 1, until the SN second order transition occuring at Hc3 (here we use Hc3 instead of Hc2 to express the critical field because the giant vortex states describing the order parameter are surface superconductivity states). The change of symmetry related to these phase transitions concerns the order parameter Ψ(r) = f (r)exp(iLθ) whose phase is invariant under a rotation of angle 2π/L. By adding a quantum of angular momentum the order of symmetry shifts from L to L + 1. These phase transitions are not periodic with H for the same reason as described in the previous section (absence of rigid contour to quantize fluxoid). Now looking at the Cp,T(H) plot in decreasing field, it appears that Cp,T(H) is strongly hysteretic, which is consistent with magnetization measurements of ref. [11]. Cp jumps are also visible although less pronounced than in increasing H, and are signatures of phase transitions of type L → L -1: at each jump the disks expell a vortex.

Phase transitions of type L → L±1 are of first order [11] and a non vanishing latent heat proportional to (SL -SL±1) is involved.

From the data of fig. 4 The origin of this hysteresis is related to the occurence of metastability. According to ref. [23] this metastability is caused by the surfacic Bean-Livingston barrier [24]: supercurrents around a vortex circulate near the edges in the opposite direction of the currents which screen the field, leading to a repulsive force between edge and vortices. This interaction prevents the nucleation of a vortex on an edge although the penetration of such a vortex could lower the free energy. However this surfacic barrier is suppressed in increasing magnetic field due to pinning on surface rugosity, which is not the case in decreasing magnetic field [23]. Thus on fig. 4 the heat capacity appears to be the one of the fundamental state when H is swept up, whereas metastability occurs as H is swept down. It is noteworthy that the width of the Cp jumps in increasing field is small (≈ 0.1 mT) compared to the pseudo period (≈ 0.55 mT) of Cp,T(H). This means that all the disks transit at a well defined magnetic field H up L corresponding to the field beyond which the state L + 1 becomes thermodynamically more stable than state L, without considering the individual disorder of a disk.

On the other hand phase transitions L → L -1 in decreasing field occur at magnetic fields that are not so well defined as in increasing field. The reason for this is that the field at which the metastable state L becomes unfavourable compared to state L -1 depends on the surface defects of a given disk.

Thus the distribution of microcsopic disorder among disks leads to a broadening of the L → L -1 transition width.

Giant vortex states phase diagrams

By repeating Cp,H(T ) scans at many fixed temperatures ranging from 0.64 K to Tc = 1.26 K, we observe that as T increases the number of successive transitions lowers since Hc3 decreases. Another effect is that the penetration fields H up L shifts to lower fields as T is increased. To hightlight this effect we plot the dependence of H up L versus temperature T for the 9 first L states accessible in our experiment: the corresponding phase diagram is presented on fig. 5. In the same way we present the temperature dependence of expulsion fields H dwn L on fig. 6. Fig. 6 is much more noisy than fig. 5 since the widths (resp. the heights) of L → L -1 transitions are larger (resp. smaller) than those of L → L + 1 transitions: both effects make it hard to localize precisely expulsion fields. independent. These observations are in good aggreement with ref. [25] and also are complementary to that work. Indeed, Baelus et al. calculated penetration and expulsion fields of mesoscopic disks (with a slightly different geometry) and measured them through a magnetization study at low temperature.

Despite this uncertainty when measuring H dwn

Since magnetization is weak close to Tc they report measurements only from 0.1 to 0.5 K. Within this interval and for the 8 first L → L ± 1 transitions they also found a penetration field that decreases with T and a temperature independent expulsion field in the case of Giant Vortex States. The results we present in this paper extend this property to all the phase diagram including the close-to-Tc area.

Furthermore, according to ref. [25], the monotonous behaviour of the H dwn L (T ) lines confirms our assumption that our disks can only host giant vortex states, since the expulsion fields of multivortex states would increase with T .

The different behaviours of penetration and expulsion fields versus T can be exploited to predict phase transitions of type L → L + 1 driven by temperature (successive jumps in Cp,H(T )) instead of H as in the last section [14]. Indeed up to now we have presented Cp,H(T ) plots that were scanned in the field cooled (FC) regime. In that case as the system is cooled from the normal state to the superconducting state it then remains in the same L state as T is swept up or down, leading to the observation of the only regular SN phase transition. In the zero field cooled (ZFC) regime the situation is quite different. To illustrate this issue we present on fig. driven by temperature at constant applied magnetic field is reported here for the first time.

Conclusion

We have studied the heat capacity dependence versus H and T of an array of independent superconducting mesoscopic disks whose size allows the presence of giant vortex states characterized by their angular momentum L. Looking at the SN transition on Cp,H(T ) curves we observe a crossover from a bulk-like to a mesoscopic behaviour: the mesoscopic character arises for L ≥ 2 since surfaces play no role in the L = 0 state in agreement with predictions of ref. [15]. Another mesoscopic signature in the thermal signal is the strong modulation of ∆C(H) at the SN transition. As a consequence the concept of specific heat fails to describe mesoscopic superconductors since it becomes possible to modulate their heat capacity with an external parameter (here H) in a way that depends on their topology, geometry and size. Playing with the magnetic history of the sample and its hysteretical properties, we are able to draw either the H or the T dependence of the heat capacity CL(T, H) at fixed vorticity L in order to establish a complete phase diagram in temperature and magnetic field.

2 mT

 2 in the FC regime. Such scans are shown on fig. 2.b for H = 0 (right curve) to 3.3 mT (left curve). As H is increased we observe an original behaviour at the SN transition: as the critical temperature Tc(H) regularly decreases, the heat capacity discontinuity ∆C(H) exhibits a non monotonic behaviour. ∆C(H) is measured and presented on fig. 2.a: discontinuities of ∆C(H) are clearly evidenced. Heat capacity measurements on doubly connected mesoscopic superconductors (rings)

L

  it appears clearly that H up L (T ) and H dwn L (T ) do not behave the same way: H up L (T ) decreases as T increases whereas H dwn L (T ) is almost temperature

7

 7 Cp,H(T ) plots scanned under a fixed H = 4.2 mT both in FC and ZFC regimes and we interpret them using phase diagrams shown on figs. 5 and 6. In the FC regime the system evolves along the dashed horizontal line H = 4.2 mT of fig. 6 since the magnetic field is already plugged: once the system has transited into the superconducting state, it remains in the same L = 4 state, whatever the temperature. Thus when increasing T to scan Cp,H(T ), the only expected phase transition occur at the SN boundary. This is what is observed on the lower curve of fig. 7. On the other hand in the ZFC regime we plug the magnetic field once the system is cooled down. Then when increasing T the system has to evolve along the dashed horizontal line H = 4.2 mT of fig. 5: this line crosses several regions of the phase diagram inside the superconducting area leading to several phase transitions (L = 2 → 3 and L = 3 → 4) before the SN transition (L = 4 → N ). Fig. 7 shows the experimental heat capacity signatures of these phase transitions. Such thermal signatures of vortex entrances in mesoscopic superconductors

Figure 1 :

 1 Figure 1: (color online) Main plot : Heat capacity C p,H=0 of superconducting aluminium disks versus temperature under zero magnetic field. Critical temperature and height of the heat capacity jump at the superconducting to normal transition are extracted. Inset : SEM image of a single disk.

Figure 2 :

 2 Figure 2: a) Height ∆C(H) of the heat capacity discontinuity at the SN transition. b) C p,H (T ) plots under fixed magnetic fields ranging from 0 mT (right curve) to 3.3 mT (left curve); the i th curve from the right has been scanned under H = (i -1) * 0.33mT in the field cooled (FC) regime. c) Critical temperature T c (H) after removing a bulk-like linear trend T c,b (H) in order to highlight the oscillating component.

Figure 3 :

 3 Figure 3: Width of the SN transition on C p,H (T ) plots around T c (H). L is the H-dependent vorticity describing the state of the disks. In the region L = 1 we observe a crossover from a bulk-like behaviour to a mesoscopic behaviour.

Figure 4 :

 4 Figure 4: Heat capacity C p,T (H) versus magnetic field at fixed temperature T =0.64 K, in increasing and decreasing fields. The data show strong hysteresis. H up L is the penetration field of the L th vortex and H dwn L its expulsion field. Both are measured at middle height of the C p jump. H c3 is the critical field beyond which superconductivity is suppressed.

Figure 5 :

 5 Figure 5: (color online) Phase diagram of the disks in increasing magnetic field. The stars represents the SN transition line. The squares are the penetration fields H up L versus T . The dashed line corresponds to a magnetic field of 4.2 mT and the dotted lines localize its intersections with the different vorticity lines.

Figure 6 :

 6 Figure 6: (color online) Phase diagram of the disks in decreasing magnetic field. The SN transition line (stars) is the same as in fig. 5. The squares are the expulsion fields H dwn L versus T . Near the critical temperature transitions L → L -1 were hard to localize because of their small height compared to the noise, and so H dwn L (T ) was not measurable near the SN line. The dashed line corresponds to a magnetic field of 4.2 mT and the dotted line localizes its intersection with the normal state line.

Figure 7 :

 7 Figure 7: (color online) C p,H (T ) plots under fixed H = 4.2 mT in FC (black curve) and ZFC regimes (blue curve).

  we can define the penetration (resp. expulsion) field H up L (resp. H dwn th vortex. Both are measured at middle height of the Cp jump. We notice that H up L > H dwn

	L	) of
	the L

L

.

florian.ong@grenoble.cnrs.fr

olivier.bourgeois@grenoble.cnrs.fr

Acknowledgments

We would like to thank E. André, P. Lachkar, J-L. Garden, C. Lemonias, B. Fernandez, T. Crozes for technical support, J. Chaussy, P. Brosse-Maron, T. Fournier, Ph. Gandit and J. Richard for fruitfull discussions and help. We thank the Région Rhône-Alpes for the PhD grant of F.R. Ong and the IPMC (Institut de Physique de la Matière Condensée) of Grenoble for financing a part of this project.