Keywords: TRIZ, IDM, Inventive Design, Machine Learning, Knowledge Extraction, Text Mining, NLP

Previous studies have made it possible to extract information related to IDM (Inventive Design Method) out of patents. IDM is an ontology-defined method derived from TRIZ. As its mother theory, IDM is primarily based on patent's observation and aims at finding inventive solutions on the basis of contradictions. In this paper, we present a new approach for extracting knowledge, this time out of other types of science-related documents: scientific papers and science news articles. This approach is based on a supervised Machine Learning model that classifies automatically the sentences of a text according to IDM's ontology concepts such as problems, partial solutions and parameters. We use two manually annotated corpora (one for each type of document) for the learning part as well as for the evaluation (with the percentage split method). Moreover, a set of semi-automatically selected linguistic features is involved in the model in order to improve the classification.

Introduction

The World Intellectual Property Organization claims that more than 3 million patents were published worldwide in 2016, that is 8.3% more than in 2015 [START_REF]World Intellectual Property Organization: World intellectual property indicators[END_REF]. Next to this massive source of technical knowledge, scientific articles also carry weight in expert scientific content with at least 2.5 million new scientific papers published each year since 2013 [START_REF]Publish or perish? The rise of the fractional author… -Research Trends[END_REF]. While patents are legal sources which provide technical information about the innovations they aim to protect, scientific articles provide theoretical as well as technical answers which help the scientific community understanding and gaining control of their environment through diverse applications. With such a huge number of publications and given that they are not always easily accessible, science news websites have developed on the Internet and advertise some important researches and innovative breakthroughs in a condensed and accessible form, making it possible for the scientific community to stay update and learn about advances in other science-related fields.

With innovation going very fast, engineers are facing a challenge to find creative ideas that may lead to innovation. To help them in their innovation process, researches have been made since the 1990s, developing and adapting Altshuller's theory of inventive problem solving (TRIZ) [START_REF] Altshuller | And Suddenly the Inventor Appeared: TRIZ, the Theory of Inventive Problem Solving[END_REF]. Yet, though patents seem the best way for engineers to find solutions [START_REF] Bonino | Review of the state-of-the-art in patent information and forthcoming evolutions in intelligent patent informatics[END_REF], it doesn't mean that other science-related documents cannot also lead to innovation. In order to determine the significance of this kind of documents on innovation, we need to find out whether these documents include enough content useful for TRIZ theory application. During the last few years, our laboratory has been developing a tool for automated extraction of knowledge related to the ontology of the Inventive Design Method (IDM) from English-language patents [START_REF] Cavallucci | The theory of inventive problem solving: current research and trends in French academic institutions[END_REF].

Following these researches, we want to adapt the information extraction from patents to the case of scientific papers and science news articles, i.e. automating the extraction of topics, problems, partial solutions, evaluation and action parameters, as well as values, inside these two alternative types of science-related documents. This is a challenging project in that both scientific research articles and science news articles are unstructured data, with much more liberty regarding their structure and writing style. Therefore, we need to evaluate the structural, syntactic as well as semantic features implemented in the patent-knowledge extraction tool [START_REF] Cavallucci | The theory of inventive problem solving: current research and trends in French academic institutions[END_REF], and then to find new features that are specific to the new document types. This implies the use of machine learning and other advanced methods for Natural Language Processing.

In this article, we will give you an overview of the literature on the Inventive Design Method (IDM) and its tool for patent-knowledge extraction, followed by the literature on scientific research papers and science news articles (Section 2), and then we will present our methodology along with the current advances on the project and an emphasis on our evaluation method (Section 3).

2

State of the art

The Inventive Design Method

The Inventive Design Method (IDM) is an application method derived from TRIZ, the theory of inventive problem solving. The mother theory was developed in the ex-USSR by Genrich Altshuller and has started to spread worldwide in the 1990s, with multiple applications and tools in the engineering field [START_REF] Cavallucci | The theory of inventive problem solving: current research and trends in French academic institutions[END_REF]. The theory is based upon four central notions which are key to an inventive solution without compromise: a few evolution laws that apply to all technical systems and their environment, the principle of increasing ideality, the principle of contradictions -at administrative, technical and physical levels -, and finally the principle of available resources. The derivate Inventive Design Method describes four steps for innovation [START_REF] Cavallucci | From TRIZ to Inventive Design Method (IDM): towards a formalization of Inventive Practices in R&D Departments[END_REF] : during the first phase, the users must extract knowledge and organize it into a graph. With this graph, they must then formulate a set of contradictions, which will be solved individually in phase 3, and finally, they must choose the most innovative Solution Concepts before investing in it and setting it up.

For the contradiction formulation, the Inventive Design Method offers a formal and practical definition of broad TRIZ contradiction notion, which is very useful for industrial innovation and introduces other notions linked to IDM-ontology [START_REF] Rousselot | Towards a formal definition of contradiction in inventive design[END_REF]. This contradiction "is characterized by a set of three parameters […] where one of the parameters can take two possible opposite values Va and 𝑉𝑎 ̅̅̅̅ " (Fig. 1). The first parameter is called action parameter (AP) and is defined with its characteristic to be able to "tend towards two opposite values" and to "have an impact on one or more other parameters". Moreover, "the designers have the possibility of modifying them". The two other parameters in a contradiction are called evaluation parameters (EP) and are defined with their capacity to "evolve under the influence of one or more action parameters", thus making it possible to "evaluate the positive aspect of a choice made by the designer". Knowledge of how the contradiction must be formulated helps for the knowledge extraction part as well as the graph creation, because the elements to be extracted from the documentation are in priority : problems, partial solutions, evaluation and action parameters as well as their possible values Va and 𝑉𝑎 ̅̅̅̅ . As of problems and partials solutions, research on IDM contains clear definitions, both in the form (syntax and graphical representation) and in its content [START_REF] Cavallucci | Initial situation analysis through problem graph[END_REF]. A problem (Fig. 2) "describes a situation where an obstacle prevents a progress, an advance or the achievement of what has to be done". A partial solution (Fig. 3) "expresses a result that is known in the domain and verified by experience.". Cavallucci et al. give more precision about the concept of partial solution [START_REF] Cavallucci | Initial situation analysis through problem graph[END_REF] :

"It may materialize a tacit or explicit knowledge of one or more members of the design team upon their past experience, a patent filled by the company or a competitor or any partial solution known in the field of competence of the members of the design team. We wish also to remind that a partial solution is supposed to bring the least possible uncertainty about the assertions of its effects on the associated problem.

Confusion can appear between a ''solution concept'' (which is the result of an assumption made by a member) and a ''partial solution'', which has been validated by experience, tests, calculations or results known and verified."

Patent-knowledge extraction

An important principle both for TRIZ and for IDM is that inventive solutions are generally found with analogy, and therefore thanks to solutions or tools which belong to another domain. Patents are documents with very rich technical content, which is generally not to be found elsewhere, like for instance in scientific papers [START_REF] Bonino | Review of the state-of-the-art in patent information and forthcoming evolutions in intelligent patent informatics[END_REF]. In order to save engineers a lot of time and help them getting to these analogies, a tool has been developed by Souili et al., which automatically creates a graph of problems, partial solutions and parameters out of English-language patents (Fig. 4), selected by a user in a large database [START_REF] Cavallucci | The theory of inventive problem solving: current research and trends in French academic institutions[END_REF]. This corresponds to the first phase of the Inventive Design Method, which has been automated thanks to Natural Language Processing tools. the same concepts, are not formulated the same way. Moreover, the number and topics of the sections vary a lot, rendering text-mining techniques more difficult to apply. However, many researches have been conducted on the structure and content of research papers. Pontille, based on a previous research by Bazerman [START_REF] Bazerman | Shaping Written Knowledge : the genre and activity of the experimental article in science[END_REF], describes a generic structure format named IMRAD (Introduction, Material and methods, Results And Discussion) [START_REF] Pontille | Matérialité des écrits scientifiques et travail de frontières: le cas du format IMRAD[END_REF] as a stabile tool for professional practice harmonization; "a particular form of proof expression" 1 and "a standardized argumentative matrix"2 which has become an official norm with the Standard for the preparation of scientific papers for written or oral presentation in 1979 in the USA.

According to Pontille, authors generally define a problem and propose hypotheses in the introduction. Therefore, it will be interesting for our research to study how the main problem is introduced in this section, in order ameliorate its extraction.

From the section about "Material and methods", which explains "the way the study has been conducted" 3 and has become central for the argumentation in the last decades [START_REF] Pontille | Matérialité des écrits scientifiques et travail de frontières: le cas du format IMRAD[END_REF], we won't focus on intermediate problems and solutions for our research, since they are often criticized during the article. However, Pontille mentions the occasional presence of a subsection for the description of "variables", which we can link to the IDM-ontology concept of parameter. Therefore, it could be interesting to extract the main parameters and to find a way to know, with the help of their limited context, whether they are evaluation or action parameters, i.e. whether they have an impact on the final output and can be modified, or whether they are central to the evaluation of the output.

The discussion section, as described by Pontille, is also interesting for our extraction tool. By "identifying the practical and theoretical implications of their analysis" 4 and "opening perspectives for futures researches"5 [START_REF] Pontille | Matérialité des écrits scientifiques et travail de frontières: le cas du format IMRAD[END_REF], we could find a repeated version of the main problem, hopefully the main partial solution of the article, and potential unresolved new problems. Moreover, since this section is the interpretation of the results evaluation, it will be interesting to look particularly for evaluation parameters here.

Considering the content, Gosden study on rhetorical themes and more specifically on contextualizing frames throughout research articles [START_REF] Gosden | Discourse functions of marked theme in scientific research articles[END_REF], with an analysis of their functions and even some links to problems and solutions, is very interesting for the extraction of features which will help us in our information extraction task.

The abstract is also a key part of a research article. According to a study conducted on articles in the field of linguistics and educational technology, three moves seem to be unavoidable in research articles : "Presenting the research", "Describing the methodology" and "Summarizing the findings" [START_REF] Dzung | Research article abstracts in applied linguistics and educational technology: a study of linguistic realizations of rhetorical structure and authorial stance[END_REF]. These move are very close to actual sections of the IMRAD structure, and since it is a summary of the research, the presence of the major problem, partial solution and parameters seem very probable.

Science news articles. Science news articles are not a famous research topic, but media coverage of science is [START_REF] Brossard | New media landscapes and the science information consumer[END_REF][START_REF] Brumfiel | Supplanting the old media[END_REF][START_REF] Puschmann | Micro)Blogging Science? Notes on Potentials and Constraints of New Forms of Scholarly Communication[END_REF][START_REF] Allgaier | Journalism and Social Media as Means of Observing the Contexts of Science[END_REF][START_REF] Minol | Portals, blogs and co.: the role of the Internet as a medium of science communication[END_REF][START_REF] Mahrt | Science blogging: an exploratory study of motives, styles, and audience reactions[END_REF][START_REF] Brossard | Science, New Media, and the Public[END_REF]. An increasing amount of science-related content is published regularly on the internet, in very diverse forms: online versions of traditional print newspapers and magazines, science section of online newspapers, science blog articles from actual researchers, videos, specialized science news websites, etc. While traditional online sources are declining in terms of audience rates, nontraditional online sources such as blogs and "other online-only media sources for information about specific scientific issues" now get much more attraction for the public [START_REF] Brossard | New media landscapes and the science information consumer[END_REF]. Science news websites therefore seem to be interesting sources for condensed information, with regularly new publications in multiple research fields.

This kind of source is nevertheless contested in the literature because of the trustworthiness of its contents [START_REF] Brumfiel | Supplanting the old media[END_REF]. Since the targeted audience is interested in scientific innovation and breakthrough, but does not have a large scientific background, information is often summarized and the risk is that is diverts from the intention of the original author. In order to prevent engineers from using erroneous extracted data during the IDM process, we will have to compare the results of the extraction from the science news article on the one hand, and from the full original document on the other hand.

Methodology

The information extraction tool we aim to create is mainly an adaptation of the existing tool used for the extraction related to IDM ontology. This adaptation requires a certain number of steps, from building a corpus to the evaluation of the final program. As mentioned above, the major challenge for this project is to adapt the features needed to select the candidate sentences for problem or partial solution classification.

The corpus creation

Annotated corpus. The first step for adapting the extraction tool is to build an annotated corpus that will be used for classification, in order to extract the main features (words or phrases) for each category -problem, partial solution, evaluation parameters, action parameters, values. Since research articles and science news articles have a very different structure, they should not be treated the same way, but preferably with two separated corpora. Moreover, there are multiple sources publishing articles in diverse fields, with differences in the global structure as well as the length and style of the articles. Therefore, each corpus must contain articles form different sources.

We chose to give the annotation task to 44 students enrolled in a course on Inventive Design Method. With this condition, we decided to build both corpora that corresponds to the level and number of students who will annotate it.

For the annotation task, all articles have been cleaned and transformed into PDF when necessary for reading comfort and homogeneity of annotation procedure, at least for the science news articles that were in html format in the beginning

The corpus of science news articles contains 44 articles from the 7 following sources:

─ Machine Design [START_REF]Machine Design[END_REF] The annotation procedure is done on the PDF-formatted files with the highlighting tool of Adobe Acrobat, following a given color legend for each element: problem, solution, evaluation parameter, action parameter, value. After collecting all annotations, evaluating and modifying them when necessary, we plan to use Sumnotes, a web application [START_REF]FiftForce: Sumnotes -Summarize PDF Annotations[END_REF], to extract a list for all sequences corresponding to each element.

Clean corpus for the extraction.

For the extraction of the different elements from the IDM ontology, we cleaned and transformed the articles into JSON format. Since most of the articles were available in HTML format, both for science news articles and scientific research papers, we used the BeautifulSoup library for Python 3 [START_REF]Beautiful Soup Documentation -Beautiful Soup 4.4.0 documentation[END_REF], which is a very efficient library for cleaning data from HTML and XML documents. However, we are still looking for a way to efficiently clean PDF articles from PNAS, because the GROBID service [START_REF] Lopez | grobid: A machine learning software for extracting information from scholarly documents[END_REF] we used for transforming them into TEI-XML documents mixed up all the sections, therefore making our information extraction task impossible. In the meantime, we had to drop this source for the features extraction, implementation and evaluation steps.

The extraction and selection of features

The adaptation of the features needed for the sentences, keywords or phrases selection and classification tasks involve three steps, and its goal is to create lists and dictionaries of words, phrases or n-grams that makes the extraction efficient and specific to both scientific research articles and science news articles. The three steps are:

─ Patent features assessment ─ Specific features extraction ─ Features selection Evaluation method. The whole features adaptation process involves multiple evaluations which have to be undertaken with the same parameters. The main purpose for the extraction program is to get sufficient results for a further use in the inventive design process following IDM-TRIZ. All the evaluations have to be undertaken with the basis of a single Gold Standard for each element category (e.g. parameters or problems), built from the annotated corpus. The program has a two-step classification process for problems and partial solutions. The first step extracts candidate sentences and the second step confirms or rejects the first choice. The rejected sentences are then classified as "neutral". For a complete evaluation, we will also consider those sentences and add them to the "concepts" category (thus containing all extracted problems, solutions and neutrals). The evaluation criteria are:  The of concepts, parameters, partial solutions, neutrals and problems extracted in average per article  The precision for each category, which is the ratio between the number of relevant elements found automatically in the category and the total number of elements automatically retrieved in the category.  The recall for each category, which is the ratio between the number of relevant elements retrieved automatically in the category and those found manually in the Gold Standard for this category.  The rate of misclassification, i.e. the proportion of sentences that should have been classified in another category but is considered useful. We consider as "useful" neutrals, sentences that are really interesting for the research or its future developments, but does not belong to the category of problems or partial solutions.

Patent features assessment. The first step is the assessment of the features from the original program that is made for patents. With a first minimal adaptation of the original program, keeping all the features intact, we assess the features globally, and then individually. The global assessment will be made by evaluating the whole program following the procedure described above. This assessment will serve as a comparison point for the individual assessment that follows, for which the program will run on a loop and evaluate its results, each time ignoring a different feature. This makes it possible to remove the features which don't have any positive impact on the final result. The removing task, however, considering the relatively small number of articles in our corpus, should be done manually in order to maintain the possibility of keeping some features for which we estimate a potential impact with a bigger corpus.

Specific features extraction.

The second step is the extraction of new features. This task will involve several tools for advanced Natural Language Processing (NLP) which are still to be defined with testing, as well as a Machine Learning classification task on the basis of our annotated corpus, using Weka [START_REF] Frank | The WEKA Workbench[END_REF], which will also make a ranking of features that helped him guess the results correctly.

Features selection. The third task will be to select the final features from the remaining patent features and the previously extracted new features, then to implement them into the main classification program, and again make an assessment of these features with the same methodology as for the patent features assessment. Depending on the results, it could be necessary to make slight changes, and run a new evaluation again, in order to get the best evaluation results as possible.

Conclusion

The automatic extraction of concepts related to IDM ontology in scientific articles and science news articles is mostly an adaptation of the existing tool working with patents. However, keeping the same features as in the original program does not make sense, since the structure, length and style of the three kinds of articles are very different. Therefore, we hope for good results while using the methodology presented in this article, i.e. creating and annotating two corpora and a golden standard, evaluating the features used in the original program, extracting useful new features and implementing the adjusted list of selected features in the program, with potential fine-tuning.

Fig. 1 .

 1 Fig. 1. Possible representation model of contradictions[START_REF] Rousselot | Towards a formal definition of contradiction in inventive design[END_REF]

Fig. 2 .Fig. 3 .

 23 Fig. 2. Graphical representation of a problem

Fig. 4 .

 4 Fig. 4. An example of a problem-solution graph

  Accounts of chemical research [35] -15 articles, about 8-10 pages long ─ Annual Review of Condensed Matter Physics [36] -8 articles, between 21-28 pages long ─ Chemistry of Materials [37] -11 articles, between 6-12 pages long ─ Proceedings of the National Academy of Sciences of the USA (PNAS) [38] -10 articles, about 6 pages long

	-6 articles, about 1-2 pages long
	─ New Atlas [29] -7 articles, about 1 full page long
	─ Phys.org [30] -7 articles, about 1-2 pages long
	─ Research & Development [31] -6 articles, about 1 page long
	─ Science Daily [32] -6 articles, about 1 full page long
	─ Science News [33] -6 articles, about 1-2 pages long
	─ Science News for Students [34] -6 articles, about 2 pages long
	─ The corpus of scientific research articles contains 44 articles from the 4 following
	sources:
	─

Scientific research papers and science news articlesScientific research papers. Many studies have been conducted on the automated information extraction from scientific research papers. While some of them are strongly related to their domain or a specific application (like biology)[START_REF] Andrade | Automatic extraction of keywords from scientific text: application to the knowledge domain of protein families[END_REF][START_REF] Krallinger | Linking genes to literature: text mining, information extraction, and retrieval applications for biology[END_REF][START_REF] Müller | Textpresso: An Ontology-Based Information Retrieval and Extraction System for Biological Literature[END_REF][START_REF] Yakushiji | Event extraction from biomedical papers using a full parser[END_REF][START_REF] Gelbukh | LINK (Online service) éd: Computational linguistics and intelligent text processing: 6th international conference[END_REF], other present, for general purposes, the extraction of general features of articles such as keywords or key phrases[START_REF] Gelbukh | LINK (Online service) éd: Computational linguistics and intelligent text processing: 6th international conference[END_REF][START_REF] Lopez | HUMB: Automatic key term extraction from scientific articles in GROBID[END_REF][START_REF] Krapivin | Large Dataset for Keyphrases Extraction[END_REF][START_REF] Kim | SemEval-2010 task 5: Automatic keyphrase extraction from scientific articles[END_REF]. However, scientific research papers have not yet been the topic of research for TRIZ or IDM applications.One of the main challenges for the adaptation of the patent-knowledge extraction tool is to understand and characterize the differences between the different types of documents: patents, scientific research papers and science news articles.While all patents have a very similar structure with four sections, and despite the higher degree of liberty in the description section, research papers have much more differences amongst themselves. For instance, title names, even if they often indicate

Our translation. The original sentence is: « Le format IMRAD constitue ainsi une forme particulière d'expression de la preuve » (p.2)

[START_REF]Publish or perish? The rise of the fractional author… -Research Trends[END_REF] Our translation. The original phrase is: « une matrice argumentative standardisée » (p.

6)[START_REF] Altshuller | And Suddenly the Inventor Appeared: TRIZ, the Theory of Inventive Problem Solving[END_REF] Our translation. The original phrase is : « explicitation de la manière dont l'étude a été conduite »

(p.6)[START_REF] Bonino | Review of the state-of-the-art in patent information and forthcoming evolutions in intelligent patent informatics[END_REF] Our translation. The original quote is : « ils identifient les implications pratiques et théoriques de leur analyse

» (p.7)[START_REF] Cavallucci | The theory of inventive problem solving: current research and trends in French academic institutions[END_REF] Our translation. The original quote is : « ouvrir des perspectives de futures recherches » (p.7)


Julian Vincent

Denis Cavallucci

Development of an Ontology of Biomimetics Based on Altshuller's Matrix

de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.


Jeongwoo Kang
email: jeongwoo.kang@insa-strasbourg.fr

Achille Souili
email: achille.souili@insa-strasbourg.fr

Denis Cavallucci
email: denis.cavallucci@insa-strasbourg.fr

Text simplification of patent documents

Keywords: Inventive Design Method, TRIZ, Information extraction, Text simplification, Syntactic analysis, Text mining

This paper represents an automatic text simplification system for patent documents. The simplification system is embedded in the broader context of an information retrieval system which extracts IDM related knowledge from patent documents. Extracting elements of IDM ontology from patents involves training machine-learning model. However, an accuracy of the model is compromised when the given text is too long, hence the need of simplifying the texts to improve machine learning. There have been precedent studies on automatic text simplification based on hand-written rules or statistical approach. However, few researches addressed simplifying patent documents. Patent document has its particularity in its lengthy sentences and multiword expression terminology, which often hinder accurate parsing. Therefore, in this research, we present our method to automatically simplify texts of patent documents and scientific papers by analyzing their syntactic and lexical patterns.

Introduction

This paper represents an automatic text simplification system for patent documents.

The simplification system is embedded in the broader context of an information retrieval system which extracts IDM (Inventive Design Method) related knowledge from patent documents. Patent documents are important source of information which contains a history of the evolution of the artifacts [START_REF] Souili | Contribution à la méthode de conception inventive par l'extraction automatique de connaissances des textes de brevets d'invention[END_REF]. By looking into patents, one can learn progress of technologies over the course of history, and more importantly, technological challenges and their solutions that were invented by specialists and engineers in the field. In the light of the importance of patents as information resource, there have been a number of academic researches and activities in patent mining in recent years. The research of [START_REF] Souili | Contribution à la méthode de conception inventive par l'extraction automatique de connaissances des textes de brevets d'invention[END_REF], which inspired our work of research, applied text mining approach for information retrieval from patents. He analyzed patent documents and automated extraction of IDM related knowledge from patents. The Inventive Design Method presents a framework for inventive problem solving process. The IDM was created by [START_REF] Cavallucci | From TRIZ to OTSM-TRIZ: addressing complexity challenges in inventive design[END_REF] and it was inspired by the Theory of Solving Inventive Problem (TRIZ), a theory invented by [START_REF] Altshuller | And suddenly the inventor appeared: TRIZ, the theory of inventive problem solving[END_REF]. Therefore, it is worth looking into TRIZ before we discuss the IDM in further detail.

TRIZ is "a problem-solving, analysis and forecasting tool derived from the study of patterns of invention in the global patent literature" [START_REF] Hua | Integration TRIZ with problem-solving tools: a literature review from 1995 to 2006[END_REF]. In the research of [START_REF] Altshuller | And suddenly the inventor appeared: TRIZ, the theory of inventive problem solving[END_REF], he suggested a systematic methodology to find inventive solutions to technological problems. According to [START_REF] Altshuller | And suddenly the inventor appeared: TRIZ, the theory of inventive problem solving[END_REF], there are generalizable patterns in the nature of inventive solutions and problems. These patterns can be applied across industries and sciences, therefore, by defining principles of technological challenges and inventive solutions, one can reuse this strategy to solve a similar problem in different context. [START_REF] Altshuller | And suddenly the inventor appeared: TRIZ, the theory of inventive problem solving[END_REF] also argued that a number of technical problems which necessitate inventive solutions stem from technical contradiction. Accordingly, defining these contradictory situations and the way to solve the contradiction are dealt as a key factor in TRIZ methodology.

IDM was created to complement the limits of TRIZ. Classical TRIZ methods are not easy to comprehend due to a lack of formalized ontology, nor simple to make a computation design model upon its concepts [START_REF] Souili | Toward an automatic extraction of IDM concepts from patents[END_REF]. Therefore, IDM presented its formalized ontology to formulate relevant ideas and concepts for inventive design. IDM ontology mainly consists of three concepts: problem, partial solution, and parameter. Parameters consist of two sub-categories, which are evaluation parameters and action parameters. Problems often refer to a contradictory situation where two features conflict with one another, and therefore, ameliorating one feature deteriorates the other. Solution is a way to solve this contradiction without compromising any of the features. On the other hand, a partial solution is an incomplete solution and while it provides a solution to one problem, it may bring about another problem. Action parameter is what a designer can make a design choice on, and evaluation parameter represents a value with which one can evaluate the result of the choice of a designer [START_REF] Cavallucci | Structuring knowledge use in Inventive Design[END_REF].

PatExtractor is an information extraction system for patents, into which the simplification system presented in this paper is integrated. PatExtractor automatically retrieves IDM knowledge from patents with its machine learning algorithm. However, the accuracy of the system is compromised when the system analyses lengthy sentences, which are fairly common in patent documents. Overlong sentences in patents make it difficult for the machine learning algorithm to classify the texts accurately and extract relevant information from them. This is because the algorithm analyzes a sentence based on the words composing the sentence, and the algorithm can be distracted when there are too many words to analyze. It is even more so when the words in a sentence contain conflicting information. Moreover, a syntactically-complex sentence, which consists of more than one predicate, often has more than one piece of information to extract. For example, a sentence composed of a few predicates that are joined by concessive or contrastive conjunctions such as yet, but, although, though or an adverb however sometimes includes a problem and a partial solution at the same time in different predicates. Therefore, it is necessary to split this sentence into smaller pieces so that the classifier identifies each part of information and label them properly.

To handle the above-mentioned sentences and to improve accuracy of the classifier by doing so, this research contributes to making an automatic sentence simplification system. This simplification system detects complex sentences and simplify those sentences by splitting, dropping and modifying. This research addresses simplification on two levels: coarse simplification and fine simplification. The coarse simplification system is designed to improve the performance of the machine learning algorithm as mentioned above. The fine simplification system is, on the other hand, designed to improve the readability of the extracted data. PatExtractor not only extracts IDM knowledge from patents but also visualizes the results using an ontology-based graphic. As its name suggests, the fine simplification system simplifies given texts in more elaborate way after the coarse simplification. Further details on the two simplification systems will be discussed in methodology section. In the next section, we review precedent researches on automatic text simplification.

Literature review

There have been a number of precedent researches on text simplification. Text simplification is a task that "aims to rewrite a sentence so as to reduce its complexity, while preserving its meaning and grammaticality" [START_REF] Lee | Splitting Complex English Sentences[END_REF]. Complexity of sentences stem from either syntactic complexity or semantic complexity [START_REF] Carroll | Simplifying text for language-impaired readers[END_REF]. As its name suggests, syntactic complexity is derived from a syntactic structure of a sentence. Syntactically complex sentences consist of more than one predicate. These predicates are joined by conjunctions such as but, and, for, until or relative pronouns such as who, which, that, whose or relative adverbs such as when and where. Therefore, this syntactic complexity can be resolved by splitting a complex sentence into shorter phrases and then restructuring each phrase to complete it. Sentence 1 is an example of a syntactically complex sentence and Sentence 3 shows how its complexity is reduced by splitting. Semantic complexity, on the other hand, is derived from difficult vocabulary. Thus, this complexity can be reduced by paraphrasing, more particularly, replacing difficult words to understand with easier and more understandable ones. Sentence 4 is an example of semantically complex sentence and Sentence 5 shows how its complexity can be diminished by paraphrasing.

Sentence 1: I have a friend whose cat is annoying. Sentence 2: I have a friend. Whose cat is annoying. Sentence 3: I have a friend. His cat is annoying. Sentence 4: The workers acquiesced to their boss' request. Sentence 5: The workers accepted their boss' request.

Text simplification is mainly used in two contexts. First, it is used to help people with low-literacy or reading disability such as foreigners, kids or people suffering from dyslexia [START_REF] Carroll | Simplifying text for language-impaired readers[END_REF][START_REF] Inui | Text simplification for reading assistance: a project note[END_REF]. In other context, it serves as a preprocessing tool for natural language processing (NLP) tasks [START_REF] Chandrasekar | Motivations and methods for text simplification[END_REF]. Simplified texts are easier to parse or translate, thus text simplifications are often integrated into automatic text summarization [START_REF] Knight | Statistics-based summarization-step one: Sentence compression[END_REF] or machine translation system [START_REF] Poornima | Rule based sentence simplification for english to tamil machine translation system[END_REF] as a preprocessing tool. Moreover, the desired results could vary depending on the context in which the simplification is used. For example, when text simplification is used as a supporting tool to aid people with low-literacy, both syntactic and semantic complexity should be reduced so that the texts are more readable and understandable for target readers. However, when text simplification is used to facilitate natural language processing, simple syntactic simplification could be enough to fulfil the objectives, depending on the context of its application.

There have been different approaches to achieve text simplification. At the early stage, most of researches were based on hand-crafted rules [START_REF] Chandrasekar | Motivations and methods for text simplification[END_REF][START_REF] Siddharthan | An architecture for a text simplification system[END_REF][START_REF] Siddharthan | Text simplification using typed dependencies: a comparison of the robustness of different generation strategies[END_REF]. In most rulebased approaches, they use POS tagger or syntax parser to detect complex sentences, and then split these complex sentences into smaller pieces using predesigned rules [START_REF] Carroll | Simplifying text for language-impaired readers[END_REF][START_REF] Poornima | Rule based sentence simplification for english to tamil machine translation system[END_REF]. This step is often followed by phrase-reordering or restructuring. As shown above in Sentence 1 to 3, a syntactically complex sentence (Sentence 1) is firstly identified as a sentence to simplify by syntactic parsing, and then split into two or more phrases (Sentence 2) and finally, restructured (Sentence 3). To restructure the phrases, co-reference relations and grammar are taken into consideration and dependency parser is often used to achieve this. Dependency parser represents the words of a given sentence in its dependency relation with other words, so it enables to capture co-reference relations in a sentence, such as a relation between a relative noun (whose in Sentence 1) and its antecedent (friend in Sentence 1). However, this rule-based approach requires linguistic knowledge to craft the rules, and it is difficult to cover all the complex sentences of different structures with these rules.

To complement the limits of rule-based approaches, academic society shifted their focus to data driven approaches [START_REF] Lee | Splitting Complex English Sentences[END_REF]. Some of these approaches use statistical machine translation system to achieve simplification [START_REF] Zhu | A monolingual tree-based translation model for sentence simplification[END_REF][START_REF] Narayan | Hybrid simplification using deep semantics and machine translation[END_REF] and they are known to produce accurate results. This approach considers text simplification as a monolingual translation task. More particularly, they consider complex sentences as source texts and simplified sentences as target texts. These approaches use parallel corpus, which consists of original texts aligned with their simplified texts, to train their simplification model [START_REF] Mille | Making Text Resources Accessible to the Reader: the Case of Patent Claims[END_REF][START_REF] Sheremetyeva | Automatic text simplification for handling intellectual property (the case of multiple patent claims)[END_REF]. This type of data is easily accessible on Simple English Wikipedia 1 . By aligning texts of Wikipedia and Simple English Wikipedia, one can obtain a parallel corpus needed to train their statistical text simplification model [START_REF] Zhu | A monolingual tree-based translation model for sentence simplification[END_REF][START_REF] Narayan | Hybrid simplification using deep semantics and machine translation[END_REF]. To train this statistical simplification system, texts, syntactic parse trees or semantic parse trees [START_REF] Narayan | Hybrid simplification using deep semantics and machine translation[END_REF] are often used as input data. This data driven method yields good results for lexical substitution and deletion, but is less effective for sentence splitting and sentence reordering [START_REF] Bott | A hybrid system for spanish text simplification[END_REF][START_REF] Siddharthan | A survey of research on text simplification[END_REF]. Moreover, parallel corpora to train this model are available for few languages.

Patents simplification has been studied in a few researches as a sub-category of general text simplification. Patent documents are known for its notoriously poor readability, and it is especially more so in the claims part. Poor legibility of patents stem from their abstract vocabulary, overlong sentences [START_REF] Mille | Making Text Resources Accessible to the Reader: the Case of Patent Claims[END_REF] and terminology. In patents, it is not uncommon to see a sentence with 200 words and there are even sentences with more than 500 words [START_REF] Souili | Contribution à la méthode de conception inventive par l'extraction automatique de connaissances des textes de brevets d'invention[END_REF]. To make those documents more readable and comprehensible for readers, there have been a few, yet not many, researches on patent simplification [START_REF] Mille | Making Text Resources Accessible to the Reader: the Case of Patent Claims[END_REF][START_REF] Sheremetyeva | Automatic text simplification for handling intellectual property (the case of multiple patent claims)[END_REF][START_REF] Shinmori | Patent claim processing for readability: structure analysis and term explanation[END_REF]. In most researches on patents simplification, rule-based approaches are preferred to statistical approaches. This is not only because there are few parallel corpus on patent documents, but also because texts in patents have specific linguistic features such as long distance anaphoric references and repetitiveness [START_REF] Mille | Making Text Resources Accessible to the Reader: the Case of Patent Claims[END_REF]. Accordingly, handcrafted rules which take these linguistic features of patents into consideration are often used for patent simplification tasks.

3

Our methodology

PatExtractor and IDM ontology

Our simplification system is integrated into an information extraction system, PatExtractor. PatExtractor takes patent documents (unstructured data) as input data, and extracts concepts of IDM ontology from the documents with NLP approach. As output, it creates a graphic (structured data), on which each node contains problem, partial solution or parameter.

In general contexts, ontology is an ensemble of relevant concepts in a specialized domain where each concept is interrelated to another. On the contrary, IDM ontology is a generic one and is not limited to a specialized domain, which makes it applicable across domains. IDM ontology is composed of concepts that are useful to formulate problematic situation and its solution during inventive design. Our simplification system is integrated to PatExtractor in order to aid the system with extracting main concepts of IDM ontology -problem, partial solution, action parameter and evaluation parameter -by simplifying the texts and thereby facilitating the information extraction process.

Text simplification

Overall, our methodology for text simplification follows the precedent rule-based approaches. However, our method is distinguished from the previous methods in two aspects. First, we don't apply lexical simplification since our texts to simplify are patents and most of terminology in patent documents cannot be replaced with another lexicon. Moreover, our target readers are specialists who seek specialized knowledge from patents, thus we have to keep the precise information that is contained in the original terminology. Secondly, our simplification system consists of two different levels of simplification. Our system is integrated into PatExtractor, an information extraction system for patents, therefore, it should fit the objectives of PatExtractor. Integration of the simplification systems to PatExtractor is as shown in Fig. 1.

PatExtractor necessitates two levels of simplification for different goals. First simplification is to facilitate machine learning algorithm's identifying target conceptsconcepts of IDM ontology -from texts. Thus, the objective of the simplification is to split sentences so that each phrase contains one idea, not several. Second simplification is designed to improve readability of the graph drawn with extracted information. Pa-tExtractor identifies problems, partial solutions, and parameters, from a given text and then visualizes them as a graph 2 . Each node of the graph contains extracted knowledge. However, when the contents of each node are too wordy, it decreases readability of the graph, hence the necessity of simplifying texts. Desired outputs for each level of simplification will further be discussed in the following sections. Before we look into different methodologies for two levels of simplifications, we address a general approach and tools that were used throughout our experiment as well as a development environment for our system. First, we adopted a rule-based approach to simplify texts. Based on our analysis on patent documents, we determined that not every complex sentence is our target for simplification. Texts need to be split only when the splitting helps the classifier to label them correctly. Otherwise, it can hurt the performance of the classifier. Accordingly, a rule-based approach with which we could specify our own rules for simplification was adopted.

Furthermore, we used a syntactic parser to identify a subordinate phrase and split the subordinate off from a matrix phrase. Among several syntactic parsers, we adopted the BLLIP parser [START_REF] Charniak | Coarse-to-fine n-best parsing and MaxEnt discriminative reranking[END_REF] given its high accuracy and fast speed 3 . The BLLIP parser is a statistical natural language parser which creates several candidate syntax parses of a given sentence and ranks n-best among them. Another advantage of using the BLLIP parser is that it supports NLTK module [START_REF] Bird | NLTK: the natural language toolkit[END_REF], so one can easily manipulate syntactic trees created by the BLLIP parser. Thus, in our research, the BLLIP parser and NLTK modules are mainly used tools for simplification task. Lastly, our system is written in Python 3.5 and operates on Linux.

Coarse simplification system. Our goal at the coarse simplification level is basically to split subordinate phrases from a matrix phrase so that information extraction algorithm treats shorter phrases and yields more accurate results. More specifically, our coarse system splits subordinate phrases that begin with a subordinate conjunction such as if, until, so that, because or relative adverbs such as, when, where, why, but it doesn't split relative clauses, which begin with a relative pronoun such as who, that, which. Relative clauses are handled at the fine simplification level, which is placed after classification task. Let's take an example of Sentence 1 to be more clear. When Sentence 1 is given as input, we expect to have two split phrases Sentence 2 and Sentence 3 as output after the coarse simplification. Sentence 1: If there is any unmatched state, the signal A turns to H. Sentence 2: If there is any unmatched state, Sentence 3: the signal A turns to H. Sentence 4: It is advantageous if all the permanent magnets are surrounded. Sentence 5: A known problem with DC currents is that when opening a switch by separating the switch contacts a spark builds between the contacts.

To achieve this, our simplification system functions as shown in Table 1. First, for a given sentence, the system checks if the sentence contains a subordinate conjunction or a relative adverb with part of speech (POS) tagger. If the sentence contains these elements, it is analyzed by syntactic parser to create a syntax tree. The syntax tree is used not only to identify a scope of subordinate phrase, but also to double-check if it truly is our target structure. In this step, we filter out unwanted structures.

Table 1. Operation of coarse simplification system

Coarse simplification system Identify complex sentences

If given text is a complex sentence  Parse a complex sentence to obtain a syntactic tree If given syntactic tree is our target structure  Split off subordinates from a matrix phrase  Clean the result sentence Let us recall that the goal of our coarse simplification is to aid the classifier in identifying problems, partial solutions and parameters from a given text. In case of Sentence matrix phrase. As a result, a sentence is divided into two or more phrases. As a last step, we clean the simplified sentence. During the syntax parsing, a sentence is tokenized, which means that words are segmented and then some punctuation and symbol characters, such as quotation marks and parenthesis, are replaced with different signs. Moreover, there can be left-out phrases or words during the simplification. Therefore, the system normalizes the modification made during the syntax parsing and simplification, and joins the left-out part to an appropriate location of the split sentence.

Fine Simplification. The objective of fine simplification system is improving a readability of a graph drawn by a web application Finder, which enables a visual representation of extracted information from patents. This can be accomplished by dropping supplementary information such as conjunctions (such as if, when, although), connective adverbs (such as then, therefore, however, by contrast, that is to say) and supplementary subordinates. Let us take an example of Sentence 5 to clarify the desired result of the fine simplification. When a sentence 5 is given, it is firstly split into two phrases as Sentence 6 and 7 by the coarse simplification system. Then conjunction 'if' in Sentence 6 and 'then' in Sentence 7 are deleted to make each of them a complete sentence. Afterwards, relative pronoun phrase 'which could interrupt the system' of Sentence 7 is deleted. The last two simplification steps -dropping a conjunction and connective adverb; deleting a relative subordinate -are done by the fine simplification system. As a final output of two simplification systems, we obtain two sentences, Sentence 8 and 9.

Sentence 5: If this offset varies from test strip to test trip, then noise which could interrupt the system is added to the measurement. Sentence 6: If this offset varies from test strip to test trip, Sentence 7: then noise which could interrupt the system is added to the measurement. Sentence 8: This offset varies from test strip to test trip. Sentence 9: Noise is added to the measurement.

The general method of the fine simplification is similar to that of the coarse simplification in the way that it exploits syntactic tree to find subordinate phrases. In the fine simplification system, however, deletion is applied instead of simple split. Moreover, only certain types of relative subordinate clauses that are followed by who, that, which, whose are subject to be treated. Generally, relative pronoun subordinate carries supplementary information of its antecedent. This supplementary information does not affect a principle message of a sentence even when deleted. In Sentence 5, the relative clause 'which could interrupt the system' carries additional information of its antecedent 'noise.' However, even when it is deleted, it does not change the main idea, that is, a problematic situation to the stated system caused by added noise. Let us recall the final output of the entire system is a visual representation of extracted information, which, thus, should be concise and clear. Therefore, dropping additional information and conveying only principal ideas is the objective of the fine simplification system. Furthermore, original sentence before simplification is also provided to users on Finder, so that users can have access to the fuller information. This is to prevent information loss or distortion which could occur during the simplification process. Table 2 shows how the fine simplification system functions. Overall, it roughly follows the same process as the coarse simplification. However, one difference is dropping connective adverbs and conjunctions at the beginning of the system. Let us recall the fine simplification system takes place after the coarse simplification. That is, some sentences are already split into more than two phrases such as in Sentence 6 and 7. In this case, we need to edit these sentences to make them complete. For example, in case of Sentence 6, it is grammatically not complete because of conjunction 'if.' Thus, we drop this conjunction to make it a complete sentence. Moreover, sentence 7 does not need its connective adverb 'then' anymore since it does not have the preceding phrase. Therefore, we drop this adverb. Dropping connective adverbs (not conjunctions) are applied not only to complex sentences which were split by the coarse simplification system, but also to the simple sentences which were not treated by the coarse simplification system. Connective adverbs (or phrases) such as however, by contrast, that is to say, in addition, moreover present the semantic relation with its preceding sentence. However, our information extraction functions on a sentence unit, which means that each sentence extracted by the system is represented without its context -its preceding and following sentence. Thus, connective adverbs which show the links between a current phrase and its preceding or following sentence are dropped to keep the extracted information simple.

Discussion

The evaluation on our simplification systems is twofold given that each level of simplification has different objectives. We integrate the coarse simplification system to the information extraction system, PatExtractor. Afterwards, we evaluate the simplification system by evaluating the performance of classification algorithm. We compare the performance of the algorithm before and after the integration of the coarse simplification system. On the other hand, the fine simplification is evaluated by readability score of the result sentences since the fine simplification system is designed to improve legibility of a graph, on which extracted information is displayed.

Evaluation on the coarse simplification system. One of the most commonly used measures to evaluate a classification algorithm are precision and recall. Recall and precision score of PatExtractor V 2.0 for partial solution and problem is as shown in the Table 3. To evaluate the system, we randomly chose 250 patent documents with different topics and extracted information from them. During our experiment, however, we did not observe a significant improvement on precision score after the integration of the coarse simplification system. The precision remained roughly the same (its improvement rate was less than 0.1%) and recall was not measured because given the structure of the entire system, our simplification does not affect the recall of the algorithm. Nevertheless, more interesting observation could have been possible if more test data, especially annotated data were available. Given the fact that only a small number of phrases are selected to classify and simplify from one patent text, the phrases that were finally used for evaluation (from 250 corpora) were not large in number. Furthermore, for lack of annotated data, the evaluation was largely based on the author's judgement. Evaluation on the fine simplification system. To evaluate the fine simplification system, we used Flesch Reading Ease score [START_REF] Kincaid | Derivation of new readability formulas (automated readability index, fog count and flesch reading ease formula) for navy enlisted personnel[END_REF]. It is one of the most commonly used scoring formula to evaluate readability of a text. The scale of the score is from 1 to 100 and the higher the score is, the more readable the text is. To evaluate the fine simplification system, we chose three sets of corpus: copurs1 consisting of 10 patent documents treating a topic in automatic transfer switch; corpus2 consisting of 10 documents covering a topic in automobile; corpus3 consisting of 10 documents addressing a topic of batteries. The score reflects not only the result of the fine simplification system but also that of the coarse simplification system. This is because the fine simplification takes place consecutively after the coarse simplification step where complex sentences are firstly split into shorter phrases. As shown in the Table 4, after applying the fine simplification system, we obtained higher readability score by 0.8 point with Corpus1, which is equal to 2.1 % of improvement rate. With corpus 2, we obtained 1.6 higher readability score after the simplification, that is, 4.83 % of the improvement rate. With corpus3, we obtained higher readability score by 1.3 point, which is equal to 3.5 % of improvement rate. The average improvement rate is therefore 3.37 %. water-absorbing ability exceeds over 800 g/g' is a cause of a problem and the latter 'the paste may be gelled and become difficult to be uniformly coated on the conductive substrate' is a resulted problem. Among these two ideas, only the latter is what our focus goes on since it concerns a problem, which is one of the IDM ontology elements we would like to extract. The previous version of PatExtractor extracts the full sentence and labeled the entire sentence as a problem. On the contrary, PatExtractor v.3.0 with the text simplification systems integrated extracts only the latter part and label it as a problem separately from its former phrase (Sentence 11). This is different from the previous version of PatExtractor which treats the entire sentence as a whole. Sentence 12 is another example of extracted information by the previous version of PatExtracotr. Sentence 13 and 14 are extracted by the current PatExtractor v.3.0. Sentence 12 delivers two main ideas: magnetic flux control means are controlled individually ; the flow of the magnetic flux can be controlled more flexibly. Both information contains important concepts which correspond to partial solution of an IDM ontology. The updated version of PatExtractor treats these two concepts separately (classifiy separately) and returns two results. This makes the final graph of extracted information more concise. However, as we discussed above, we observed that the integration of simplification systems does not improve the accuracy of the classification significantly. Sentence 12 corresponds to a partial solution but is labeled as a problem by PatExtractor v.2.0. This incorrect labeling remains unchanged in PatExtractort v.3.0 as seen in Sentence 13 and 14.

Sentence 10: If the water-absorbing ability exceeds over 800 g/g, the paste may be gelled and become difficult to be uniformly coated on the conductive substrate.

[problem] Sentence 11: The paste may be gelled and become difficult to be uniformly coated on the conductive substrate.

Conclusion and Future work

Even though integration of the simplification systems did not lead to a significant improvement of the classification algorithm, it improved a readability of a graph created from extracted information to a certain degree. Furthermore, deleting supplementary information made it possible to extract information with focusing only on important part of information, which corresponds to elements of the IDM ontology. Moreover, simplification system enabled PatExtractor to treat different concepts and ideas in one sentence separately, contrary to the previous version which treated them as a whole.

For the future work, phrase restructuring function could be added to the simplification system since, while splitting off sentences into several phrases, anaphoric relations between a pronoun and its antecedent could be lost. Moreover, it would be interesting to evaluate the precision of the upgraded version of PatExtractor with more corpus that are annotated by experts.

Fig. 1 .

 1 Fig. 1. Integration of the simplification systems to PatExtractor

[problem] Sentence 12 :

 12 When the magnetic flux control means are controlled individually, the flow of the magnetic flux can be controlled more flexibly. [problem] Sentence 13: The magnetic flux control means are controlled individually. [problem] Sentence 14: The flow of the magnetic flux can be controlled more flexibly. [problem]

Table 2 .

 2 Fine simplification system Fine simplification system Drop connective adverbs and conjunctions Identify sentences containing a relative subordinate If given text has a relative subordinate clause  Parse the sentence to obtain a syntactic tree If given syntactic tree is our target structure  Delete subordinates from a matrix phrase  Clean the result sentence

Table 3 .

 3 Recall and Precision of PatExtractor V 2.0

		Recall	Precision
	Partial solution	47.87 %	74.99 %
	Problem	43.83 %	88.14 %

Table 4 .

 4 Readability score before and after simplification Let us take examples of sentences produced by PatExtractor v.3.0 to see the result in detail. Sentence 10 is produced by the previous version of PatExtractor, that is, version 2.0. which does not have the text simplification systems. The former part of the phrase 'if the

		Before simplification After simplification	Improvement rate
	Corpus1	37.1	37.9	2.1 %
	Corpus2	33.1	34.7	4.83 %
	Corpus3	36.7	38.0	3.5 %
	Average	35.6	36.8	3.37 %

It is an edition of Wikipedia geared towards students, children or adults with learning difficulties. It is therefore written in basic and easy English.

The graph is displayed on Finder (https://finder.inventivedesign.unistra.fr), a web application which visualizes extracted information with a graph.

According to the authors' experiment, compared to Standford Parser, which is one of the most commonly used parsers for syntactic analysis, BLLIPparser functions more accurately when analyzing sentences with multi-lexical terms, which are common in patents. As for parsing speed, BLLIPparser is slightly faster than Stanford Parser. NLTK syntax parser was not considered for our task since it requires users to write their own rules.

and

5, a phrase located before if and when can be an indicator telling the class of the following phrase. For example, the phrase 'It is advantageous'[START_REF] Hua | Integration TRIZ with problem-solving tools: a literature review from 1995 to 2006[END_REF] in Sentence 4 indicates that the following phrase has a positive connotation, and therefore, can be a partial solution. Likewise, the phrase 'a known problem with DC currents is that' in Sentence 5 gives information that the following phrase may contain a problem. Therefore, splitting these phrases from its subordinates can hurt the performance of the classifier. To avoid this, we rule out sentences containing some keywords such as problem, solution, appreciated, advantageous off the list of sentences to simplify. We use the same, but filtered, keywords list that is used to extract problems from patent documents.Once the given structure is identified as our target structure after the filtering, we split subordinate phrases, which is represented as SBAR on a syntactic tree, from a 4 This is a common sentence structure in patents and they indicate that the current sentence includes a solution (or a problem). This type of sentences has a structure of 'It is [adjective] if /when/ that [subordinate phrase].' As for adjectives, there are certain adjectives or past participles that are commonly used, such as beneficial, advantageous, recommended, appreciated, accepted.


Pei Zhang

Denis Cavallucci

Zhonghang Bai

Cécilia Zanni-Merk

Facilitating Engineers abilities to solve inventive problems using CBR and Semantic similarity

Keywords: TRIZ, Case-based reasoning (CBR), semantic similarity

Our industry currently undergoes a period of important changes. The era of computerization implies to companies to change not only through their organization, but also in automating as much as possible their internal processes. Our research focuses on the computerization of the problem-solution couple when facing inventive situations in R&D. The method used is based on Case-Based Reasoning (CBR) that has already been proven to be useful in routine design. On the other hand, CBR is hardly used in inventive situations because the latter require reasoning outside the circle of knowledge recorded in a database. Our proposal consists in coupling CBR with semantic similarity algorithms. The aim is to resolve a new problem based on its semantic similarity with the old problems. Then the old solution can be adapted to solve the new problem. We postulate that a multidisciplinary case base sufficiently populated of multidisciplinary problem-solution couples is likely to considerably improve the performance of R&D engineers to solve inventive problems. This being possible by bringing them alternative solutions based on the semantically similar problems, which are distant from their field of origin. In this way, we provide the possibility to enhance the inventiveness of solution. This type of reasoning, largely inspired by the TRIZ theory, is the subject of this paper. The methodology, the experiments and the conclusions that we develop here validate that this type of approach produces the claimed effects on designers although limited to the context where it has been conducted.

Introduction

Our industry is moving through a period of important changes. The era of computerization pushes companies to change not only through their organization, but also through their internal physical functioning. Among the research performed around the notion of industry 4.0, a large part is dedicated to the computerization of workshops, machines, controls and computer flow that optimizes and responds to customer demand by its physical functioning. However, there is a sector of the company, which is only little researched in industry 4.0, its R&D department. As the most important stage that leads to an inventive product, a methodology adopted by R&D departments is the theory of inventive problem solving (TRIZ) [START_REF] Altshuller | The Innovation Algorithm: TRIZ, Systematic Innovation and Technical Creativity[END_REF].

TRIZ offers various components for solving different types of inventive problems, such as the 40 Inventive Principles, the 76 Inventive Standards. According to the researches respectively carried by [START_REF] Cavallucci | World Wide status of TRIZ perceptions and uses a survey of results[END_REF][START_REF] Ilevbare | A review of TRIZ, and its benefits and challenges in practice[END_REF][START_REF] Spreafico | TRIZ industrial case studies: a critical survey[END_REF], the Contradiction Matrix with its 40 Inventive Principles and the Su-Field analysis with its 76 Inventive Standards are among the most popular tools used by the TRIZ community.

However, the "intellectual" cost for an enterprise to use a tool such as TRIZ, which is based on the users' experience, is exceptionally high. In addition, in the era of Industry 4.0, where the world is engaged in sharing data and automating the manufacturing technologies, R&D departments seem to be behind schedule in the way they automate problem solving.

Therefore, a legitimate axis of research is to find ways to facilitate the inventive problem solving process. Researches can be mainly categorized into two groups. One group addresses the problem solving model, and finding ways to ease the use of the models, while another group addresses on the knowledge sources used by TRIZ and attempts to facilitate the use of TRIZ by knowledge modelling.

The first group of researchers regard problem solving as an analogical process with the objective of finding inventive solutions to their problems [START_REF]Analogies Are the Way of Breakthrough Innovation[END_REF]. This process is composed of two essential activities, with the help of TRIZ tools. One is the analogy reasoning between the problem and the problem model, the other is the analogy reasoning between the problem model and the solution model. To cope with the former problem, the work of [START_REF] Duflou | On the complementarity of TRIZ and axiomatic design: from decoupling objective to contradiction identification[END_REF] provided a way to systematically map design parameters with the Generic Engineering Parameters in axiomatic design. Moreover, to ease the use of the Contradiction Matrix, the authors in [START_REF] Coelho | Matching TRIZ engineering parameters to human factors issues in manufacturing[END_REF] matching considered the aspects regarding to the human factors issues of the Generic Engineering Parameters. To cope with the latter problem, ASIT [START_REF] Toshio | How people interact with objects using TRIZ and ASIT[END_REF] grouped 32 Inventive Principles into 5 thinking tools; the analysis conducted in [START_REF] Cong | Grouping of TRIZ Inventive Principles to facilitate automatic patent classification[END_REF] classified the Inventive Principles into clustered principles assuming the fact that there might be losses as compared to the use of TRIZ in a classical way.

The other direction of research addresses the aspect of taking advantage of the knowledge sources of TRIZ and knowledge modeling. The work of [START_REF] Prickett | The development of a modified TRIZ Technical System ontology[END_REF] proposed the TRIZ Technical System class ontology with its four sub-ontologies provided a framework that will enable the storage of knowledge found by other problem solving applications. The work of [START_REF] Yan | An ontology-based approach for inventive problem solving[END_REF] connect the TRIZ knowledge sources with different abstract levels and designed rules to facilitate the problem solving process using ontology and its related rules.

Compared with the researches above, we propose an approach to collect and represent the problem-solution pairs from the problem solving know-how of experts (their professional life experience). In addition, we adopt Case-based reasoning to reuse available solution at hand in order to support the problem solving process. This paper is organized as follows: section 2 discusses the importance of experience, section 3 details the proposed approach, section 4 validates the proposed approach by a case study and finally section 5 gives the conclusion and the future perspectives.

Is experience important?

With the aim of finding out how experience influences the problem solving, we did an experiment about how it can influence the time used for problem solving. We asked 28 students of the same grade in an engineering department of our university, and separate them into two groups with respectively 14 students: the experimental group and the control group.

For the experimental group, we designed a set of online forms with 10 simple cases and assigned them to 14 students. Each form contains the problem description and multiplechoice questions to guide students to find a solution model for each given problem. Once a student solves a problem and submits an online form, his/her problem solving time is recorded. For the control group, we ask the students to use the rule-based problem solving prototype [START_REF] Yan | IngeniousTRIZ: An automatic ontology-based system for solving inventive problems[END_REF] to solve the same 10 problems. The time used by each student for finding the solution model is automatically recorded by the prototype. The data we gathered concerning the time spent for problem solving is presented in Fig. 1. The x-axis represents the average time used by the two groups using the assigned approach and the y-axis represents the ID of the problems.

As it can be observed in Fig. 1, the average time used by the experimental group (the green bars) is decreasing as the students are solving more problems while the control group (the blue bars) remains stable. This result indicates that experience is a crucial factor influencing the time used of inventive problem solving, the more problems the students solve; the faster they tend to be. Furthermore, the students of the control group who uses the rule based approach; the time they use for solving the first problem until the last problem stays stable. This is because when using the rule-based approach, the problem solving is facilitated by if-then rules and the needed knowledge for each step, which makes the problem finding process easier. Therefore, the students in the control group use less time than the students in the experimental group for solving problem No.1. However, the problem with the rule-based approach is that every time the student solves a new problem, they have to go through the problem solving process from the beginning. In addition, this is why the problem solving time used by the control group stays balanced.

We can conclude that the collection and reuse of experience have the potential to improve problem solving, at least in terms of time. In addition, it is worth mentioning that in this experiment, we only ask the students to find the solution model in order to make the time used for problem solving comparable. In fact, the rule-based approach is not able to provide the knowledge about the solution in its knowledge base. Therefore, the user with no experience would have trouble to interpret the solution model into a solution. Consequently, our research is focusing on finding a proper way to collect and reuse experience in the previous problem solving with the aim of providing users with the knowledge needed for solution finding. Case-based reasoning (CBR) is a methodology that has been widely adopted for problem solving and learning. It can benefit from the specific knowledge of the previous experience rather than making associations between general problems and its general solutions. In this way, when a problem needs to be solved, it is able to retrieve similar cases in the past and solves the new problem by reusing it in the new problem [START_REF] Aamodt | Case-based reasoning: Foundational issues, methodological variations, and system approaches[END_REF].

Applying case-based reasoning consists in retrieving a similar past case, reusing the past case and retaining the new case.

Semantic similarity

If we want to find a similar past problem to a new inventive problem, the problem is that they are expressed in terms of natural language. Depending on the way the problem is described, the words used can vary but the meanings are similar. For example, something that is big and small is similar to something that is large and tiny. This type of problems are semantically similar but do not share a single word. In order to solve this problem, we need some external knowledge that enables us to find the semantically similar problems.

Facilitate inventive problem solving using Case-based reasoning and semantic similarity

With the aim of using the specific knowledge from experience to facilitate the inventive problem solving, we propose an approach using case-based reasoning and semantic similarity.

Restricted by the length of this paper, we only introduce here the resolution of a contradiction. The CBR approach comprises four activities: Case representation; Case retrieval; Case reuse and Case retain.

Case representation

First, we need to represent the case in order to retrieve similar cases. The case representation is introduced in the previous work of [START_REF] Zhang | Case-based Reasoning for Knowledge Capitalization in Inventive Design Using Latent Semantic Analysis[END_REF].

Case retrieval

In order to retrieve similar cases, we need to find semantically similar cases. Therefore, we adopted WordNet1 as the knowledge base for calculating the semantic similarity between the new problem and the old problem. We apply the semantic similarity algorithm based on the short-text similarity proposed in [START_REF] Yan | An ontology-based approach for inventive problem solving[END_REF].

To calculate the similarity consists in two steps: calculating the similarity between the new problem and the old problems and then calculating the weighted similarity between the new problem and the old problems. The former is composed of five sub-steps, which are:

1. Pre-processing: It includes a sequence of actions. Firstly, split the shot text into single words (for example ''smoothing the surface'' becomes ''smoothing'', ''the'' ''surface''). Next, remove the stems of the words (for example, the words obtained in the previous step becomes ''smooth'', ''the'' and ''surface''). Finally, eliminate stop words (words like the, of, or etc., are eliminated). The output of this step is a set of terms. 2. Sense search: For each term, we look for its corresponding senses using WordNet ( e.g. the word ''smooth'' has eight different senses). 3. Sense similarity: We adopt Lin's method [START_REF] Lin | Extracting collocations from text corpora[END_REF] to calculate the semantic similarity between two terms. Using this method, the higher rate of sharing information, the more the two terms are similar. 4. Term similarity: The maximum value of the sense similarity value between two terms is the term similarity. In general, we select the most similar terms in P2 for each term in P1 and then we calculate the average value as defined in Equation 1:

∑ max 1≤𝑗≤𝑚 (𝑠(𝑃 1𝑖 ,𝑃 2𝑗 )) 𝑛 𝑖=1 𝑛 (1)
Then, we apply the tf*idf method [START_REF] Salton | Computer evaluation of indexing and text processing[END_REF], [START_REF] Jones | A statistical interpretation of term specificity and its application in retrieval[END_REF] to assign weight to the obtained similarity.

Applying the tf*idf method, we can obtain the word weight ww1i, that is the ith word in 𝑃 1 and the word weight ww2i, that is the ith word in 𝑃 2 . In addition, to calculate the weighted similarity, we should apply the word weight to all possible situations. One is the most similar word in 𝑃 2 for 𝑃 1 . The other is the most similar words in 𝑃 1 for 𝑃 2 .

In both situations, 𝑠(𝑃 1𝑖 , 𝑃 2𝑗 ) represents the word similarity between new problem 𝑃 1𝑖 and the old problem

𝑃 2𝑗 (1 ≤ 𝑖 ≤ 𝑛, 1 ≤ 𝑗 ≤ 𝑚). 𝑠 𝑤𝑒𝑖𝑔ℎ𝑡𝑒𝑑 (𝑃 1 , 𝑃 2 ) = ∑ 𝑤𝑤 1𝑖 max 1≤𝑗≤𝑚 (𝑠(𝑃 1𝑖 ,𝑃 2𝑗 )) 𝑛 𝑖=1 𝑛 + ∑ 𝑤𝑤 2𝑖 max 1≤𝑗≤𝑚 (𝑠(𝑃 2𝑖 ,𝑃 1𝑗 )) 𝑚 𝑖=1 𝑚 (2)
Case reuse and retention When the similar cases are retrieved, the next step is to reuse the retrieved cases. If the user finds an identical case and reuse its solution without adaptation, the solved case will not be retained. However, if the user selects a similar case and reuse its solution with certain adaptations, the new case will be retained in the case base.

Case study

Clothes hangers are used in our daily life to hang clothes vertically. Their design should make them versatile enough to fit for hanging clothes with different sizes. Therefore, length of the arm should be long enough to support the cloth, but short enough to ease their entering into the clothes without efforts. The traditional design of the cloth hanger is often in the shape of a triangle that has this type of fixed structure design has disadvantages. In this section, we apply the proposed approach to solve this problem.

First, we describe the problem in terms of a conflicting situation as the follows: The length of the clothes hanger has to be both short to satisfy the ease of hanging and long to satisfy the clothes stability. Then, we apply the semantic similarity calculating to find the similar problems in meaning. The retrieving results are illustrated in Table .1. We present in Table .1 the first 10 old cases similar to the new one. The similarity values are given in the first column in Table .1. We can observe that the retrieved similar cases are similar in meaning rather than using the same words for describing the problem. For example, large is similar to big; freedom of movement is similar to ease of hanging and length is similar to distance, and so on. Among the similar cases we retrieved, suppose we want to reuse the solution of case 102, which is making the arrangement of the shopping cart s-shaped. Inspired by the solution, we can design the solution for the hanger problem as it can be seen from Fig. 2. The design of the s-shaped hanger, inspired by the S-shape of shopping carts case, is an interesting way of solving our new problem of cloth hanger. On the one hand, if the cloth has a tight neck, we can first stick the hanger vertically by the long edge, and then rotate it to the horizontal position so that the short edge can be used to hang the cloth easily without damaging it. On the other hand, the long edge supports the cloth sufficiently so that it will not produce wrinkles. In this way, the application of the proposed approach facilitate problem solving by simply adapt the solution of the similar case that is retrieved.

Table 2. Retrieval results

Case

Conclusion and future perspectives

In this paper, we explored an approach to combine case-based reasoning and semantic similarity to facilitate inventive problem solving. Compared with the rule-based approach [START_REF] Yan | A Method for Facilitating Inventive Design Based on Semantic Similarity and Case-Based Reasoning[END_REF], the proposed approach provides the users with the specific knowledge from past successful cases. In this way, the user can solve the new problem by only adapting the old solution of the old problem. Therefore, the proposed approach can ease the problem solving process by reusing experience. Moreover, since the case base stores cases from different domains, the proposed approach can benefit from finding similar cases from different domains and reuse its solution.

Response to reviewers

Reviewer#1 Overall evaluation: 3: (strong accept) Very interesting approach and a good demonstration.

Our answer:

Thanks for the comments.

Reviewer#2

Overall evaluation: 3: (strong accept) Good paper. Addresses an important issue, proposes a solution and demonstrates it.

Well structured.

There are some minor language mistakes. A review is suggested. Some mistakes were marked in the pdf file.

Our answer:

1. Page 2, 4th paragraph, line 4: While another group addresses on the knowledge sources used by TRIZ and attempts to facilitate the use of TRIZ by knowledge modelling.

It has been changed to:

One group addresses the problem solving model, and finding ways to ease the use of the models, while another group addresses on the knowledge sources used by TRIZ and attempts to facilitate the use of TRIZ by knowledge modelling.

Page 2, 5th paragraph, line 12:

To cope with the latter problem, ASIT [START_REF] Toshio | How people interact with objects using TRIZ and ASIT[END_REF] grouped 32 Inventive Principles into 5 thinking tools; the analysis conducted in [START_REF] Cong | Grouping of TRIZ Inventive Principles to facilitate automatic patent classification[END_REF] classified the Inventive Principles into clustered principles assuming the fact that there might be losses as compared to classical TRIZ use.

It has been changed to:

To cope with the latter problem, ASIT [START_REF] Toshio | How people interact with objects using TRIZ and ASIT[END_REF] grouped 32 Inventive Principles into 5 thinking tools; the analysis conducted in [START_REF] Cong | Grouping of TRIZ Inventive Principles to facilitate automatic patent classification[END_REF] classified the Inventive Principles into clustered principles assuming the fact that there might be losses as compared to the use of TRIZ in a classical way.

3. Page 2, 5th paragraph, first sentence: Another direction of research addresses the aspect of taking advantage of the knowledge sources of TRIZ and knowledge modeling.

It has been changed to:

The other direction of research addresses the aspect of taking advantage of the knowledge sources of TRIZ and knowledge modeling. Once the students solve a problem and submits an online form, his problem solving time is recorded.

It has been changed to: Once a student solves a problem and submits an online form, his/her problem solving time is recorded.

5. Page 3, 3 rd paragraph, line 7: Furthermore, the students of the control group who uses the rule based approach; the time they use for solving the first problem until the last problem stays table.

It has been changed to: Furthermore, the students of the control group who uses the rule based approach; the time they use for solving the first problem until the last problem stays stable.

6. Page 4, section 3.1, line 4: In this way, when a problem need to be solved, it is able to retrieve similar cases in the past and solves the new problem by reusing it in the new problem [START_REF] Aamodt | Case-based reasoning: Foundational issues, methodological variations, and system approaches[END_REF].

It has been changed to:

In this way, when a problem needs to be solved, it is able to retrieve similar cases in the past and solves the new problem by reusing it in the new problem [START_REF] Aamodt | Case-based reasoning: Foundational issues, methodological variations, and system approaches[END_REF].

Reviewer#3

Overall evaluation: 2: (accept) The topic is relevant, the context of the study is well posed as well as the problematic however it could be interesting to detail a little more the experimentation during the oral presentation.

Our answer:

Thanks for the comments, we will address the experimentation details during the oral presentation.

Fig. 1 . 3 Facilitating

 13 Fig. 1. Average time respectively used by the two groups

Fig. 2 .Fig. 3 .

 23 Fig. 2. The shopping cart problem

4 .

 4 Page 3, 2 nd paragraph, line 4:

7 .

 7 Page 9, reference 1: [1] G. S. Alʹtshuller, L. Shulyak, and S. Rodman, The Innovation Algorithm: TRIZ, Systematic Innovation and Technical Creativity. Technical Innovation Center, Inc., 1999. It has been changed to: [1] G. S. Altshuller, L. Shulyak, and S. Rodman, The Innovation Algorithm: TRIZ, Sys-tematic Innovation and Technical Creativity. Technical Innovation Center, Inc., 1999.

Table 1 .

 1 Features to describe a case.

		Input Features	Output Features
	Problem features	Value
		Negative value
		Action Parameter
		Evaluation Parameter to improve
		Evaluation Parameter to degrade
	Solution features		Inventive Principle Concept Solution

  [START_REF]Analogies Are the Way of Breakthrough Innovation[END_REF]. Semantic similarity: Based on the obtained term similarity, we can calculate the semantic similarity. Let's assume that a new problem, P1, includes a sequence of words P11 , P12 ... P1n and an old case P2 , includes a sequence of terms P21, P22... P2m.

		𝑠(𝑃 11 , 𝑃 21 ) 𝑠(𝑃 11 , 𝑃 22 ) ⋯ 𝑠(𝑃 11 , 𝑃 2𝑚 )
	[	⋮	⋱	⋮	]
	𝑠(𝑃 1𝑛 , 𝑃 21 ) 𝑠(𝑃 1𝑛 , 𝑃 22 ) ⋯ 𝑠(𝑃 1𝑛 , 𝑃 2𝑚 )

s(P1i, P2j) represents word similarity between P1i and P2j , 1 ≤ i ≤ n , 1 ≤ j ≤ m . We can build the matrix of similarity M (P1, P2) :

https://wordnet.princeton.edu/

As it has been discussed in Section 2, a large case base has the potential to increase the efficiency of the problem solving process. However, we are limited to a small case base and the future work will be dedicated to enlarge the case base with more cases.


Didier Casner

Achille Souili

Rémy Houssin

Jean Renaud

Agile'TRIZ Framework: Towards the Integration of TRIZ Within the Agile Innovation Methodology

Keywords:

des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.