Hong-Bae Jun
email: hongbae.jun@hongik.ac.kr

Dimitris Kiritsis
email: dimitris.kiritsis@epfl.ch

Several aspects of information flows in PLM

Keywords: BOL, MOL, EOL, closed-loop PLM, information flows

There are lots of activities in the product lifecycle, which are grouped into three main phases: beginning of life (BOL), middle of life (MOL), and end of life (EOL). Thanks to recent emerging technologies, information flows of whole product lifecycle can be visible and controllable. The PLM under this environment allows all actors of the whole product lifecycle to access, manage, and control information flows. These flows can be used to streamline several operations of BOL, MOL and EOL. In order to recognize the benefit of these information flows, first of all, it is necessary to comprehend them in detail. For this purpose, this study will deal with several aspects of information flows in PLM. It will clarify the concept of information flows and identify which product lifecycle information are required for streamlining lifecycle operations, and classify them into several types depending on their characteristics.

Introduction

Recently, the product lifecycle management (PLM) has been in the spotlight. The PLM is a new strategic approach to manage product-related information efficiently over the whole product lifecycle [START_REF] Ameri | Product Lifecycle Management: Needs, Concepts and Components[END_REF]. In general, the product lifecycle is defined as "a series of stages that a product goes through from its concept generation to its disposal." It can be decomposed into three main phases: Beginning of Life (BOL), including Design and Production, Middle-of-Life (MOL), including Use, Service and Maintenance, and End-of-Life (EOL) where products are recollected, disassembled, remanufactured, recycled, reused, or disposed of.

The PLM is ideally an information processing system [START_REF] Saaksvuori | Product Lifecycle Management[END_REF]. It supports the management of all information such as CAD drawings, technical documents, and structured and unstructured information created, changed, transferred, stored, managed, and converted by along the lifecycle of a product, from its design to end of life. It becomes the backbone for managing product related information in an enterprise. Physically, product data and information are dispersed along a variety of information systems, generated and used in the diverse phases of the lifecycle by many different actors (Terzi et al. 2005). Thus, PLM should enable sev-eral internal and external actors (i.e. stakeholders) to do collaborative creation, modification, dissemination, and search of information throughout product lifecycle. It entails modeling, searching, manipulating, exchanging, and using of lifecycle information over the whole lifecycle. As [START_REF] Rachuri | Information sharing and exchange in the context of product lifecycle management: Role of standards[END_REF] and [START_REF] Xu | Modelling for product information tracking and feedback via wireless technology in closed-loop supply chains[END_REF] mentioned, the scope of information to be managed increases as the product lifecycle evolves. As a result, a great deal of lifecycle information is generated during the whole product lifecycle. Since the information is usually created and consumed by various stakeholders in a certain sequence, lots of information flows are generated. Because of emerging information and communication technologies, it is no doubt that there are more complex information flows over the whole lifecycle. As [START_REF] Ouertani | Traceability and management of dispersed product knowledge during design and manufacturing[END_REF] mentioned, querying and sharing product knowledge is becoming a key issue in enterprise information systems. Hence, the success of PLM lies in identifying what kind of information are available in the other phase, and how we can use them in order to streamline business processes.

Currently, there is the lack of study on information flows to be identified across product lifecycle operations. Although some studies dealt with information flows in PLM, many of them addressed the framework or protocol for communicating information flows, not information flows themselves. As a result, very little attention has been paid to clarifying the product lifecycle information flow. The unavailability of explicit flows of product lifecycle information leads to a certain degree of inefficiency in performing lifecycle operations. Thus, the methods to efficiently represent, control, and search information flows are critical. It requires the identification of information flows and their efficient management, which can play an important role in analyzing and making decisions of several operational issues in the product lifecycle.

To this end, first and foremost, this study focuses on clarifying the concept of product lifecycle information flows from several viewpoints. This study is organized as follows. Section 2 addresses the previous studies related to product lifecycle information. Section 3 deals with several aspects of lifecycle information flow. Finally, some discussion points are addressed with conclusion.

Previous study

This section summarizes the part of previous studies related to product lifecycle information and also introduces the PROMISE project with its main concept, the closed-loop PLM, where closed information flows had been addressed.

Previous studies on product lifecycle information

Until so far some literatures dealt with lifecycle information model itself or how to capture lifecycle information or data, rather than on information flows themselves. For example, [START_REF] Goncharenko | An agent-based approach for collecting and utilizing design information throughout the product life cycle[END_REF] proposed an approach to collect and utilize product feedback information throughout product lifecycle. [START_REF] Harrison | Information management in the product lifecycle-the role of networked RFID[END_REF] addressed the information required to support various decisions to be taken along a product lifecycle. They mentioned the characteristics of lifecycle information in terms of information quality as follows: uniqueness, completeness, timeliness, and accuracy. Furthermore, [START_REF] Terzi | A holonic metamodel for product traceability in PLM[END_REF] dealt with the product traceability problem concerning with the management of the identification of a product along its lifecycle. They presented a reference meta model with the concept, 'product holon' that merges product physical part with product information needed for tracing its life. In addition, [START_REF] Främling | Requirements on unique identifiers for managing product lifecycle information: comparison of alternative approaches[END_REF] analyzed and compared three approaches (EPC Network, DIALOG and WWAI) for providing a linking mechanism to product information that may be stored in backend systems of different organizations. Recently, [START_REF] Liu | Product Lifecycle management: A review[END_REF] addressed various information related to product based on previous PLM literature survey. [START_REF] Xiao | Modeling framework for product lifecycle information[END_REF] proposed a 4D (prototype system, geometry, task, and lifecycle dimension) view model for sharing and managing product lifecycle information, especially, simulation data. They mentioned that product lifecycle information could be described in aspects of time and space.

On the other hand, some literatures concentrated on the modelling of product lifecycle information. For example, [START_REF] Jun | Product lifecycle information modeling with RDF[END_REF][START_REF] Jun | Product life-cycle metadata modeling ad its application with RDF[END_REF] applied the resource description framework (RDF) to represent and search the meta-data of product lifecycle with a modelling procedure. [START_REF] Jun | A framework for RFID applications in product lifecycle management[END_REF] also introduced key information required for capturing closed feedback flows among three main lifecycle phases with the RFID application framework in PLM. Furthermore, [START_REF] Sudarsan | A product information modeling framework for product lifecycle management[END_REF] proposed a product information modeling framework to support the full range of PLM information needs, based on the NIST Core Product Model (CPM) and its extensions, the Open Assembly Model (OAM), the Design-Analysis Integration model (DAIM) and the Product Family Evolution Model (PFEM). [START_REF] Rachuri | Information sharing and exchange in the context of product lifecycle management: Role of standards[END_REF] also proposed a model of information flows in PLM. They dealt with the protocol for communicating information among PLM stakeholders. They classified content of product information used by various stakeholders in a product lifecycle into three main categories: form information, functional information, and lifecycle information. In addition, [START_REF] Xu | Modelling for product information tracking and feedback via wireless technology in closed-loop supply chains[END_REF] proposed a framework and methodology to model three principle information loops in wireless technology-enabled closed-loop supply chains (CLSC). They addressed the models of three information flows (MOL to MOL, MOL to EOL, All to BOL product design), focusing on dynamic information related flows and tracking activities. In addition to the traditional forward flow of products to customers, they considered the acquisition and return flows of products for functional rehabilitation or value recovery.

PROMISE project and closed-loop PLM

For tracking and utilizing the lifecycle information throughout the whole product lifecycle, the concept of closed-loop PLM had been proposed in the PROduct lifecycle Management and Information tracking using Smart Embedded systems (PROMISE) project [START_REF] Kiritsis | Closing product information loops with product embedded information devices: RFID Technology and Applications, Models and Metrics, RFID technology and Applications[END_REF]). The main objective of PROMISE project is to develop a new generation of product information tracking and flow management system. In general, it is acknowledged that the information flow breaks down after the delivery of the product to the customer. The fact that the information flow is in most cases interrupted shortly after product sale restricts the feedback of data, infor-mation and knowledge, from service and maintenance and recycling experts back to designers and producers. However, with the recent advances of emerging technologies, the whole product lifecycle can be visible and controllable. In PROMISE, Product Embedded Information Devices (PEID) such as RFID tag and on-board computer with associated firmware, software components, and tools, had been used for this purpose. This resulted in closed-loop information flows during the whole product lifecycle, which appeared in 10 industrial applications of PROMISE.

Product lifecycle information flows

In general, information is typically processed (or generated) in a certain sequence. This is referred to as the information flow [START_REF] Schneider | Information technology support in the chemical process design life cycle[END_REF]. Product lifecycle information has the following characteristics. ; it provides no judgment or interpretation and no sustainable basis of action. However, PPRT relations create information since they can give some useful meanings. During lifecycle operation, all lifecycle content data are processed throughout the PPRT relations. Hence, lifecycle information includes relevant product (input/output of process), process to deal with the product, and resource to support the process. The resource includes the following: human, hardware (facility), software (information system), and PLM agent. Here, the PLM agent has the role in processing lifecycle information and transmitting them into some information systems, that is, creating information flow. For example, maintenance engineer with some smart device like PDA or Product embedded information device is a PLM agent. Product lifecycle information flow occurs when the lifecycle information generated at the past period are changed or referred to by several lifecycle actors.

(2) It has closed-loops, and is horizontally and vertically closed.

It has closed-loops since generated information in a certain process can be used in any other lifecycle processes, and updated information can go back to the process for reuse. There are two kinds of closed information flows in the PLM: forward and backward. For the example of forward flows, BOL information can be used to streamline processes of MOL and EOL. Service and maintenance and recycling experts will be assisted in their work by having an up-to-date report about the status of the product during MOL and product design specification made from BOL phase. Also, recyclers/reusers will be able to obtain accurate information about 'value materials' and 'disassembly instruction' of EOL products from product design information. On the other hand, for the example of backward flows, MOL and EOL information can also go back to the designer and producer for improvement of BOL decisions. For example, designers will be provided with product usage data about the modes of use and conditions of retirement and disposal of their product, and thus improve product designs towards product lifecycle quality goals. It indicates that the information flow is horizontally closed over the whole product lifecycle. In addition, information flow is vertically closed. This means that in a certain process, based on gathered data related to resource, process, and product, we can analyze product related information and take some decisions on optimization of processes with support of some information systems, which will affect data gathering again.

(3) It is accumulated and transformed.

As the lifecycle proceeds, lifecycle information and its flows are more and more accumulated. With the accumulated lifecycle information and its flows, we can see how content data are updated and reused for other processes. In addition, it is possible to find some clues for solving problems or defects encountered in product content data. Although each information or information flow itself may be incomplete for capturing any meaning, the accumulated lifecycle information can be transformed into knowledge and usefully applied for streamlining lifecycle operations. Throughout analyzing, synthesizing, comparing, tracking, and tracing accumulated information flows, we can derive or deduce certain knowledge, e.g. on product status or the source of (bad) quality.

(4) It is initiated by business drivers and used in appropriate information system.

Information flows can be initiated by several business drivers of lifecycle operations such as incorporating voice of customer into design process, improving internal engineering collaboration/concurrent engineering, improving supply chain collaboration, establishing systematic knowledge capture and reuse, product design improvement using product usage data, fast reconfiguration of a production system, efficient warehouse management, efficient production logistics, predictive maintenance, MOL maintenance/service process optimization, EOL product logistics/waste stream management, and EOL product recovery decision making, and so on. Some business drivers are already considered in the PLM implementation. Others can be newly provided after the implementation of PLM by the requests of stakeholders. In any case, business drivers make information flows generated. The generated information flows are used in appropriate information system for streamlining lifecycle operations

Classification of information flows

In this study, to gain a clear understanding of information flows in PLM, we classify them as follows. First, according to the direction of closed-loops, they can be classified into vertically closed and horizontally closed information flows.: (1) Vertically closed, and (2) Horizontally closed. At first, as the lifecycle is evolving, vertical information flows are generated. Then, by the request of business applications, horizontal information flows are generated. Regarding horizontal information flows, according to the direction of lifecycle phases, they can be classified into forward and backward information flows.

-Forward information flow BOL to MOL: In general, there are lots of information flows from BOL to MOL since product design and production information are required to deliver products to customers and keep the use of products without any problems. Product usage (or operation) and maintenance period (i.e. MOL phase) is longer than any other lifecycle phases. Thus, technical manuals of products produced at BOL should be kept up to date until the end of product usage. Furthermore, product information for maintenance and spare parts should be accessible at any time. To this end, 'BOL to MOL' information flows are generated. This flow becomes more important since the serviceoriented engineering has been highlighted.

MOL to EOL: Recently, the importance of product recovery process has been increasing since return rate of products rises up due to the expansion of online product purchasing market and strict environmental regulation on products. These motivations make companies have much interest in effective EOL management, which requires MOL to EOL information flows. Owing to various information technologies, a PLM system can gather accurate data related to product usage data during MOL phase, e.g. repair history data, mileage, current status of component, used time, etc. Based on gathered data, at the collecting and dismantling phase of EOL products, EOL product experts in the PLM system can predict degradation status and remaining life time of parts or components. With the information, at the inspection phase, the dismantler can do EOL product recovery optimization, in other words, deciding suitable EOL recovery options such as recycle, reuse, remanufacturing, and disposal, with the objective of maximizing values of EOL products considering product status. To this end, 'MOL to EOL' information flows are generated.

BOL to EOL: A PLM system can gather accurate data related to product lifecycle history at the collecting and dismantling phase of EOL products, e.g. material recycling can be significantly improved because recyclers and re-users can obtain accurate information about ''value parts and materials'' arriving via EOL routes: which components they consist of, what materials they contain, who manufactured them, and other data that facilitate reuse of materials, components and parts, which can be provided from BOL phase. To this end, 'BOL to EOL' information flows should be generated.

-Backward information flow (feedback) Most forward information flows can be directly used as inputs for streamlining the operations of next phases. However, some of them such as feedbacks from MOL and EOL to BOL must be indirect links because they take some times so that they do not directly affect the design and production of the same products that were considered at BOL. However, the feedback information can be used as knowledge for improving the design and production of the same or similar types of next-phased products.

MOL to BOL: In BOL, designers and producers will receive feedback about detailed product information from distributors, maintenance/service engineers, or customers on product usage, about the conditions of retirement, and disposal of their products. Therefore, they will be able to exploit expertise and know-how of other actors in the product lifecycle. For example, mission profile data and maintenance data of products during MOL phase can be a good reference data for improving the design of next similar types of products. We can apply collected product usage data to improve the design of next-phased products via design for reliability. It will also help to enhance production processes and systems. Furthermore, we can gain marketing advantage from improved knowledge of how users utilize a product [START_REF] Xu | Modelling for product information tracking and feedback via wireless technology in closed-loop supply chains[END_REF].

EOL to MOL: Optimization of EOL recovery decision also provides useful information to remanufacturers for making an efficient remanufacturing plan in advance. Furthermore, logistics engineers can improve logistics at EOL (reverse logistics) from collecting to remanufacturing, reuse or disposal. They can get supply volume data for recycle, reuse, remanufacturing, and disposal products in advance from the EOL decision. They can also know demand data for EOL products, e.g. a number of products to be collected, collecting locations, etc. With this information, they can make the best collecting and redistribution plan (reverse logistics plan) with the objective of minimizing total logistics operation cost.

EOL to BOL: In addition, EOL product recovery decision data and product status at EOL dismantling can give useful information to product designers for improving product design with several purposes, e.g. design for reliability, reuse, recycle, and so on. For example, the field data from EOL disassembly activities can act as a source of knowledge to design for improved cost efficiency of disassembly [START_REF] Xu | Modelling for product information tracking and feedback via wireless technology in closed-loop supply chains[END_REF].

Table 1 and 2 show the information flows and relevant applications with necessary information.

Discussion

The lifecycle information flows in PLM may be explicit or implicit. When implementing PLM, according to functions required in a company, some information flows are designed and explicitly built for monitoring them. However, some ad-hoc information flows may be generated later by internal or external stakeholders. In any case, it is important to have the infrastructure for allowing those information flows without any technical problem during product lifecycle. That is, PLM should be the backbone structure of information flows. To this end, there are several issues to be considered. First, to facilitate several information flows, for each lifecycle phase, it is critical to gather lifecycle information. During BOL phase, it seems that lifecycle information can be gathered without any problems thanks to several common information systems such as CAD/CAM, PDM, and MES. The problem is that most lifecycle information has been lost after sales. However, according to emerging wireless communication and sensor technologies such as RFID system, Bluetooth, global positioning system (GPS), and wireless sensor networks (WSN), it becomes possible to gather lifecycle information during product usage period. Second, with an understanding of how people work collaboratively we try to gain insight into how lifecycle information flows through a company during product lifecycle. For the success of PLM, one important point is that not only explicit flows but also implicit flows of lifecycle information generated during off-line tasks should be manipulated in PLM. In this sense, modeling for product lifecycle information and its flow is also important. To this end, useful diagrammatic tools for information flow, e.g. RDF, would be required. Third, several tools and methods for supporting processes should be integrated with information flows. For example, quality function deployment (QFD) method, failure mode and effect analysis (FMEA), and design for X-ability (DFX) should be integrated with forward and backward information flows, respectively. Fourth, information flow is simply transmission of information from one place to another. Over lifecycle, a huge amount of product lifecycle information are generated and modified, and reused for several applications within or through a company. Hence, product lifecycle information should be available at any time and any place with proper format. Furthermore, timely and accurate information is a prerequisite to obtain a competitive advantage to meet this challenge. To this end, infrastructure and methods to keep consistency and accuracy of information and to facilitate collaborations among different participants are critical. Finally, it is necessary to efficiently find information stored in different locations, source systems and formats in one search.

Conclusion

This study has dealt with several aspects of information flows in PLM. It addressed the concept of lifecycle information flows and identified which product lifecycle information are required for streamlining lifecycle operations, and classified them into several types depending on their characteristics. It is worthwhile to study what information flow is and how information flow is created and used during product lifecycle. By understanding the value of lifecycle information flows, better decisions can be made for streamlining product lifecycle operations.

Since this study just dealt with some parts of the study on product lifecycle information flows, the following can be considered as future research works. First, one can investigate more detailed characteristics of information flows. It may also be the challenging issues to develop the ontology-based product information flows sharing. Furthermore, someone may concern how product lifecycle information flows can be traced over the whole lifecycle. Analysis on benefits of information flows can be used to doing ROI analysis of PLM. In addition, it can be one challenging issue to look into the information flows of PLM with social network analysis and mobile/cloud computing environment.

Fig. 1 .( 1 )

 11 Fig. 1. Schema of product lifecycle information flow

Table 1 .

 1 Information flow (forward)

	Direction	Business drivers			Information
	BOL	to	Improving design and	BOM information	Product ID, product structure, part
	MOL		manufacturing collab-			ID,	component	ID	prod-
			oration				uct/part/component design specifica-
							tion, etc.
			Improving service	Information for	Spare part ID list, price of spare part,
			collaboration		maintenance/service	maintenance/service instructions, etc.
					Production	infor-	Assemble/disassemble	instruction,
					mation		production specifications production
							history data, production routing data,
							production plan, inventory status, etc.
	BOL	to	Improving	product	Product information	Material	information,	BOM,
	EOL		reuse and eliminating			part/component cost, disassemble
			environmental effects			instruction, assembly information for
							remanufacturing, etc.
					Production	infor-	Production date, lot ID, production
					mation		location, etc.
	MOL	to	Improving	product	Maintenance history	Number	of	breakdowns,
	EOL		reuse		information		parts/components' IDs in problem,
							installed date, maintenance engineers'
							IDs, list of replaced parts, aging
							statistics after substitution, mainte-
							nance cost, etc.
					Product status in-	Degree of quality of each component,
					formation		performance definition, etc.
					Usage environment	Usage condition (e.g. average hu-
					information		midity, internal/external temperature),
							user mission profile, usage time, etc.
					Updated BOM		Updated BOM by repairing or chang-
							ing parts and components, etc.

Table 2 .

 2 Information flow (backward)

	Direction	Business drivers		Information
	MOL	to	Incorporating voice of	Maintenance and	Ease of maintenance/service, relia-
	BOL		customer into design	failure information	bility problems, maintenance date,
			process	for design improve-	frequency of maintenance, MTBF,
				ment	MTTR, failure rate, critical compo-
					nent list, root causes, etc.
				Technical customer	Customer complaints, customer
				support information	profiles, response, etc.
				Usage environment	Usage condition (e.g. average hu-
				information	midity, internal/external tempera-
					ture), user mission profile, usage
					time, etc.
	EOL	to	Improving supply	Recycling/reusing	Reuse part or component, remanu-
	MOL		chain collaboration	part or component	facturing information, quality of
				information	remanufacturing part or component,
					etc.
	EOL to		Incorporating voice of	EOL product status	Product/part/component	life-time,
	BOL		customer into design	information	recycling/reuse rate of each compo-

Acknowledgements

This research was supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education, Science and Technology (2011C103401).


Mohamed-Foued Sriti
email: mfsriti@ccis.imamu.edu.sa

Philippe Boutinaud
email: pboutinaud@cadesis.com

PLMXQuery: Towards a Standard PLM Querying Approach

Keywords: PLM, Product Data Exchange, Data Semantics, XML, XQuery

Experience with data exchange standards has shown considerable limitations in their integration in commercial tools and their interoperability. The current trend is to use XML as a mean of exchanging data. We present in this paper a new standard approach for querying and exporting data from PLM in XML format using XPath/XQuery languages. An abstraction effort and an adapter implementation were required to make PLM content as usable as XML document. Afterward, the resulting XML content could be directly reused or converted to another format.

Introduction

Early engineering systems were composed from a set of tools that didn't have a significant impact in the market [START_REF] Goldstein | A Brief History of Early Product Data Exchange Standards[END_REF]. Over the years, companies have been increasingly equipped with CAD systems, and, naturally, adopted systems were different from a company to another. Engineers, who are users of these heterogeneous systems, were paralyzed by the inability of sharing and exchanging data related to their products because each system has a proprietary and incompatible data representation format.

To comply with this situation, the main proposed solutions were in form of data exchange standards. First standards were intended to exchange bi-dimensional and tridimensional geometric data. Then, the need to exchange went beyond this, since there were several applications that manage and exchange data throughout product lifecycle.

Besides, through several industrial experiences in PLM (Product Lifecycle Management) integration and customization, we found that there is a considerable redundant work from a project to another in particular for the following applications:

• Reorganization of data and products • Improvement of transfer and data exchange • Automatic documentation generation Usually, these applications need to export or to exchange data. In all cases, provided solutions were specific every time and didn't always include exchange standards. Surely, the best way is to use data exchange standards, but experience with them has shown considerable limitations in their integration in commercial tools and their interoperability. Recently, several researches have been turned to XML strategies, either by exchanging data in a native XML format or by providing an XML representation of the exchange standards.

In this paper, we define a new standard approach for querying and exporting data from PLM in XML format. The paper is organized as follows. In Section 2, we present data exchange technologies, their limitations and the new developed approaches. In Section 3, we present our proposed PLMXQuery approach and we discuss its implementation in Section 4. We conclude in Section 5.

State of the art

To bring new products to the market faster, with more profit and lower cost, companies need to share and exchange data quickly, automatically and safely [START_REF] Saaksvuori | Product Lifecycle Management[END_REF]. Moreover, according to [START_REF] Eynard | Gestion du cycle de vie des produits et dynamique des connaissances industrielles en conception intégrée[END_REF], one of the main PLM goals is to simplify communication and data exchange between collaborators and industrial partners. In fact, data exchange is the building block on which we can find various applications such as data sharing and migration, applications integration, and much more knowledge reuse. Actually, PLM supports many distributed activities and different systems to create, store and share data related to products. This, according to [START_REF] Fenves | Product Information Exchange: Practices and Standards[END_REF], requires robust mechanisms for data exchange and integration. Since PLM data are handled by heterogeneous systems using proprietary formats, in order to exchange data, the first approach that comes to mind is to implement translators between each pair of systems. Considering the huge number of existing systems, it will be worth to find another way specifying an open, common and neutral data representation: exchange standard.

This principle has given rise to many exchange standards in engineering (i.e. IGES, VDA, SET, STEP, etc.) These standards are supposed to be independent from any system or application providers. Naturally, first standards concerned geometric data handled by CAD/CAM systems. According to [START_REF] Rachuri | The Role of Standards in Product Lifecycle Management Support[END_REF], CAD standards had significantly reduced translation costs and improved customers and suppliers interactions. Then, normalization was expanded to cover all product aspects throughout its lifecycle [START_REF] Srinivasan | Open Standards for PLM. PLM Emerging solutions and challenges[END_REF].

On the one hand, data defined by the geometry has limited number of basic concepts (point, line, curve, surface, etc) which are common to physical product aspects. Thus, international normalization efforts are focused on this kind of data. But on the other hand, it is less conceivable to standardize data related to the product lifecycle because these data are represented using different concepts from a system to another. Some works like [START_REF] Szykman | A Foundation for Interoperability in Next-Generation Product Development Systems[END_REF], [START_REF] Lee | A generic model to support rapid product development: an XML schema approach[END_REF], [START_REF]Open product lifecycle data sharing using XML[END_REF] and many others tried to propose standards for exchange data related to product lifecycle. Nevertheless, it is still difficult to specify an exchange standard common to all existing systems around the product lifecycle. Even if we try, standardization will be a long and complex process [START_REF] Lin | A Manufacturing System Engineering Web Ontology Model on the Semantic Web for Inter-enterprise Collaboration[END_REF].

In general, some standards are mature, such as STEP, but they still suffer from:

• Slow integration in commercial tools [START_REF] Eynard | Gestion du cycle de vie des produits et dynamique des connaissances industrielles en conception intégrée[END_REF].

• Lack of interoperability among standards [START_REF] Rachuri | The Role of Standards in Product Lifecycle Management Support[END_REF], [START_REF] Pratt | Introduction to ISO 10303-the STEP Standard for Product Data Exchange[END_REF].

• Large exchange files and significant processing time [START_REF] Fenves | Product Information Exchange: Practices and Standards[END_REF].

• Solutions are specific to their domains and communities [START_REF] Lin | A Manufacturing System Engineering Web Ontology Model on the Semantic Web for Inter-enterprise Collaboration[END_REF], [START_REF] Burkett | Product data markup language: a new paradigm for product data exchange and integration[END_REF].

To overcome some of these limitations, there is strong tendency to use XML as a mean to capture and transform data [START_REF] Pratt | Introduction to ISO 10303-the STEP Standard for Product Data Exchange[END_REF]. For example, to represent STEP files in XML [START_REF] Peak | STEP, XML, and UML: Complementary Technologies[END_REF], PDML (Product Data Markup Language) is proposed [START_REF] Burkett | Product data markup language: a new paradigm for product data exchange and integration[END_REF]. Another example, PLMXML [START_REF]Open product lifecycle data sharing using XML[END_REF], is used to represent products' lifecycle related data.

Beside of data exchange based on translators and standards, we identify a third approach which is based on application integration. This approach defines a process on which several autonomous, distributed and heterogeneous data sources are integrated into one source and they are associated to a global schema [START_REF] Bellatreche | Contribution of Ontology-based Data Modeling to Automatic Integration of Electronic Catalogues within Engineering Databases[END_REF]. By using this approach, data can be exchanged smoothly throughout the predefined mappings between every local schema of each source and the global schema. Application integration can be based on databases, data files [START_REF] Saaksvuori | Product Lifecycle Management[END_REF] (in text format (CSV) or standard format (STEP, XML, etc.)), or semantic web ontologies [START_REF] Lin | A Manufacturing System Engineering Web Ontology Model on the Semantic Web for Inter-enterprise Collaboration[END_REF], [START_REF] Hefke | A Methodological Approach for Constructing Ontology-Based Reference Models in Digital Production Engineering[END_REF], [START_REF] Sriti | Towards a semantic-based platform to improve knowledge management in collaborative product development[END_REF]. Obviously, data integration is almost implemented to integrate applications sharing the same data [START_REF] Burkett | Product data markup language: a new paradigm for product data exchange and integration[END_REF]. It should be thought "a priori" and requires a significant investment. Also, this approach is not easily scalable since it's mandatory to re-integrate different applications if we need to exchange data other than defined at the initial integration.

A new approach has emerged based on remote services invocation. In this context, the OMG has defined PDM Enablers [START_REF]OMG: Product Data Management Enablers. Object Management Group standard specification[END_REF], specifying a set of interfaces providing functionalities that give a simple access to PLM data in standard way. Few years after, the OMG has also developed PLM services [START_REF]OMG: Product Lifecycle Management Services. Object Management Group standard specification[END_REF], an advanced and extended version of PDM Enablers based on Web Services technology. In fact, the current trend in PLM market is that several software providers give access to all or a part of their API functions through Web Services.

PLMXQuery approach

Regarding the different approaches previously exposed, we can discern that there is always need to a more standard and flexible strategy for exchanging data between PLM applications. In fact, we can benefit from effective aspects of previous experiences.

In this section, we present PLMXQuery, a new developed approach which defines a standard way to access and explore PLM content, in order to exchange data, without resorting to a concise specification of limited set of concepts (the case of exchange standards) or a full integration of applications.

Definition

The idea of PLMXQuery approach is to make the PLM content seen as XML document. Then, we can get benefit from XML related technologies, in particular XPath and XQuery, to query (browse and export) PLM content in a standard way. Afterward, resulting (exported) XML content could be reused directly or converted to another format. We can easily use this approach to exchange data punctually (i.e. migration) or permanently (i.e. integration). Also, we can export all the content of the PLM or only a thoroughly selected part of its content.

However, to achieve this idea we are facing a modeling challenge which is to make an abstraction effort on PLM content to make its exploitation possible as XML document. Next, we will show how to overcome this challenge using a standard mapping rules between PLM objects and XML elements. After that, we will present XPath and XQuery languages and the way they will work in our context.

Mapping rules

In order to represent their products through different systems and lifecycle steps, companies need to specify a methodical and functional structures that reflecting data organization depending to each company. For this purpose, every PLM system should implement a data model. The later usually defines basic components: types of objects managed by the PLM system, their properties and relationships. Therefore, a product can be represented inside a PLM system as a set of interconnected objects.

Table 1. Standard mapping rules between PLM objects and XML nodes.

Within the PLMXQuery approach, every PLM component is mapped to a corresponding XML element 1 according to the standard mapping rules, summarized in Table 1, which are applicable to any PLM system. The following example is an illustration: <Item id="00013" name="weldpoint"> <IMAN_reference direction="forward"> <Text id="00017" name="specs"/> </IMAN_reference> </Item>

Querying using XPath and XQuery

XPath and XQuery2 are two standard languages working on XML structures and we'll use them to extract data from a PLM system. XPath is a filtering language based on a context node to designate a portion of an XML document. In reference to the previous XML example, the following one shows how to access the attribute id of element of type Text related to context node (not always the root).

./IMAN_reference/Text/@id XPath could be incorporated inside other XML languages, especially XQuery. The later is an XML query language in which we can declare functions and use conditional and loop statements. The following is an example of a query that uses an XPath expression (text between "(:" and ":)" specifies a comment).

xquery version "1.0"; (: Get a reference to the context object :) declare variable $root := .; (: Get a set of items associated to the context :) declare variable $related := $root/IMAN_reference/Text; (: Beginning of the resulting XML :) <root>{ (: Process related items one by one :) for $item in $related return <text>{$item/@id}</text>} </root> By applying this query on the XML example, it will browse the XML content starting from the context node, then filter and list all paths of interest using XPath expression and then, returns the following XML:

<root> <text>0017</text> </root>
However, XPath and XQuery operate on an abstract data model named XDM 3 . This feature allows the underlying model to be anything other than a structured XML content; PLM content in our case study. Thus, we should implement PLM to XML mapping rules to make the mentioned languages work on PLM through this abstraction. This is what we'll undertake in the following section.

PLMXQuery implementation

For the implementation of PLMXQuery, we propose an adapter for exporting data from PLM system in XML format. PLMXQuery adapter should be able to consider the PLM content as an XML document and to query it using an XQuery engine.

Usually, when working with XML document the query engine tries to browse the whole document to ensure its conformity (a well-formed XML document). This is inconceivable when XML document will be substituted by a PLM database. This situation leads to an important technical challenge which is to have reasonable performances; filtering PLM content and limiting the browsing to only objects of interest.

Next, we present a standard architecture for the adapter including the solution for the raised challenge and the selected XML platform. Then, we present a specific implementation of the adapter for Teamcenter PLM system.

Architecture

Since our approach considers PLM content as XML document queryable using XQuery language, the adapter should be able to implement defined mapping rules and incorporate an XQuery engine. A standard architecture for the adapter is proposed in Error! Reference source not found.. When we need to query the PLM content according to this architecture, we edit a query and then the adapter will: 1) load and compile the query, 2) execute the query on the PLM content, and 3) return the query results in XML format. In order to perform these different steps, the adapter will translate the PLM browsing and access functions (API) to those expected by the XQuery engine. Browsing limitation. One of the major selection criteria of the XML platform on which we will build the PLMXQuery adapter is to restrict the scope of visited PLM objects. To make this possible, we should be able to define the following functions through the selected platform:

• Access function: allows for quickly finding and selecting PLM objects using some of their properties. Then, a selected object could be used as a context node in the query. • Filter function: limits browsing only on PLM object types, relation types and attribute names that we are interested in.

Selected platform. There are several existing XML platforms endowed with XQuery engine. To build the adapter, we need to reuse the most appropriate platform that fits the different PLMXQuery requirements expressed by the following major criteria:

• Adaptability: clear/documented implementation of the XDM abstraction model.

• Performance: restriction and limitation of the visited objects' number.

• Extensibility: definition of additional functions adapted to the underlying model.

• Cost: license, free, etc.

The Table 2 illustrates a summary of the compared platforms against these criteria. In this table, we mention if a platform includes the concerned feature (" ") or not (" "), and if there were difficulties to test or to get information on the feature ("?"). So, it is clear from this comparison table that Saxon is the most appropriate platform.

Teamcenter XQuery Adapter

Teamcenter XQuery is an implementation of PLMXQuery Adapter for Teamcenter PLM system (edited by Siemens PLM Software). In the following we define the mapping rules for this implementation and query extension functionalities.

Mappings. Mapping rules defined earlier are standard and common to all PLM systems. If needed, these rules could be completed by additional mapping rules specific for every PLM system. Table 3 shows additional rules for Teamcenter PLM system needed to map its content with XML.

Table 3. Teamcenter mapping rules.

Extension functionalities. Standard functions provided by XPath/XQuery could be limited to answer specify some queries. The selected XML platform, Saxon4 , allows to define new extension functions that can be reused inside the query allowing more flexibility in query expression. The developed extensions are (with examples):

1. Direct access functions to Teamcenter objects:

(: determine where the extensions are defined :) declare namespace tc= "java:com.cadesis.plmxquery.teamcenter.TeamCenterXQueryExten sion"; (: get TC object to use as a context node :) declare variable $item := tc:getObject("item00001");

2. Filtering functions (types, relations, attributes filters):

<root typeFilter="{tc:setTypeFilter("Item")}" relFilter="{tc:setRelationFilter("Revision,view")}" attFilter="{tc:setAttributeFilter("current_id, current_name,object_type,constraint, ")}" > {.} </root> 3. Downloading files function (: Get set of documents associated to the item :) declare variable $docs := $item/IMAN_reference/Text; (: Beginning of the resulting XML :) <root> { (: Examine all documents :) for $doc in $docs (: We try to get the file using the extension :)

Fig. 1 .

 1 Fig. 1. PLMXQuery architecture.

Table 2 .

 2 Comparison of existing XQuery platforms.

	Plateform	Adaptability	Performance Extensibility	Cost
	Altova XML suite		?
	DataDirect XQuery		?
	Jaxen
	Jaxp
	Xalan
	QtXmlPatterns
	Saxon

http://www.w3.org/TR/xpath/ and http://www.w3.org/TR/xquery/

http://www.w3.org/TR/xpath-datamodel/

http://www.saxonica.com PLM object

The first and the second extensions are developed especially to enhance the adapter performance by limiting the number of visited nodes. But the third function is just developed to download files from PLM into a temporary folder.

All of the adapter for Teamcenter, including extension functions, is based on a common package that defines all of common abstract classes and functions. This package makes easy to implement another PLM adapter.

Conclusion

In this paper we presented the PLMXQuery approach, a new way to extract data from PLM in XML format. The proposed approach considers the PLM content as an XML resource than queries this content using the XML standard query language XQuery. Throughout the implementation of this approach (Teamcenter XQuery adapter) we faced two challenges: 1) defining mapping rules between PLM and XML and 2) restricting number of PLM browsed objects. The developed adapter was based on an XML platform named Saxon, equipped with an XQuery engine. This platform was extensible enough to overcome the raised challenges and to define additional functions used to refine more elaborated queries.

The PLMXQuery approach is very flexible since it doesn't define a limited set of concepts to avoid exchange standards drawbacks and it doesn't recourse to full, complex and slow integration of applications. Using this approach, when we need to have an interface with a PLM system we reuse the standard package to develop its specific adapter. But, for a specific PLM system, only one adapter is developed for several purposes (data export, data exchange, integration, migration, etc) and, for each different application, we just need to write the queries and to configure the query engine (without any modification of the source code).

Teamcenter XQuery adapter is currently under test in a real industrial context and until now we don't have enough results to discuss in this paper.

We believe that we proposed an interesting standard way to access and query PLM content. Using an XML-based approach in a PLM context is very suitable and more advisable since XML is widely used especially in data transfer and communication among heterogeneous applications.


Olivier Lézoray

Stochastic spectral-spatial permutation ordering combination for nonlocal morphological processing

The extension of mathematical morphology to multivariate data has been an active research topic in recent years. In this paper we propose an approach that relies on the consensus combination of several stochastic permutation orderings. The latter are obtained by searching for a smooth shortest path on a graph representing an image. The construction of the graph can be based on both spatial and spectral information and naturally enables patch-based nonlocal processing.

I. INTRODUCTION

Mathematical morphology (MM) is a non linear image processing framework with many applications in filtering, segmentation and classification [START_REF] Serra | Image Analysis and Mathematical Morphology[END_REF]. The construction of morphological operators relies on complete lattices [START_REF] Ronse | Why mathematical morphology needs complete lattices[END_REF] that impose the need of an ordering relationship between the elements to be processed. A complete lattice (T , ≤) is a non-empty set equipped with a total ordering relation, such that every nonempty subset P of T has a lower bound ∧P and an upper bound ∨P. In this context, images are modeled by functions mapping their domain space Ω, into a complete lattice T , and are expressed as f : Ω ⊂ Z l → T ⊂ R n where l is the image dimension, n the number of channels, and T is a non-empty set of vectors. If MM is well defined for gray scale functions, there exists no general admitted extension that permits to perform morphological operations on vectors since there is no natural ordering of vectors [START_REF] Barnett | The ordering of multivariate daa[END_REF].

To cope with this well-known issue, several approaches have been proposed (see [START_REF] Aptoula | Multivariate mathematical morphology applied to colour image analysis[END_REF], [START_REF] Lézoray | Complete lattice learning for multivariate mathematical morphology[END_REF], [START_REF] Velasco-Forero | Mathematical morphology for vector images using statistical depth[END_REF] and references therein). The most recently proposed ones define an ordering relation between the vectors of a set T with the use of h-orderings [START_REF] Goutsias | Morphological operators for image sequences[END_REF]. This corresponds to defining a surjective transform h from T to L where L is a complete lattice equipped with the conditional total ordering [START_REF] Goutsias | Morphological operators for image sequences[END_REF]. We refer to ≤ h as the hordering given by:

h : T → L and v → h(v), ∀(v i , v j ) ∈ T × T v i ≤ h v j ⇔ h(v i ) ≤ h(v j ) . (1)
Then, T is no longer required to be a complete lattice, since the ordering of T can be induced upon L by means of h [START_REF] Aptoula | Multivariate mathematical morphology applied to colour image analysis[END_REF]. When h is bijective, this corresponds to defining a space filling curve [START_REF] Chanussot | Bit mixing paradigm for multivalued morphological filters[END_REF] or equivalently a rank transform [START_REF] Lezoray | Rank transformation and manifold learning for multivariate mathematical morphology[END_REF]. Most approaches for ordering vectors (including horderings) have focused on building an ordering on the definition domain of the spectral vectors of the set T , without taking into account how the corresponding vectors are located in the image support. This provides a general framework, but it can be more interesting to find the best order for a given image rather than looking for the best order in general, as argued in [START_REF] Chevallier | The irregularity issue of total orders on metric spaces and its consequences for mathematical morphology[END_REF]. Recent h-ordering approaches have searched to define total orders that are adapted to only the set T of spectral vectors of an image [START_REF] Lézoray | Complete lattice learning for multivariate mathematical morphology[END_REF]. In addition, this adaptation can consider also the spatial organization of the spectral vectors in the image. Indeed, some vectors of the set T will never have to be compared since they will not appear within a given structuring element. Consequently, having an ordering that considers spatial proximity in addition to spectral proximity can be of strong interest. Some works have considered that trend of spectral-spatial ordering with the use of hierarchical clustering [START_REF] Chevallier | Image adapted total ordering for mathematical morphology on multivariate images[END_REF], binary partition tree [START_REF] Veganzones | Binary partition trees-based spectral-spatial permutation ordering[END_REF] or local hamiltonian path [START_REF] Lezoray | Graph-based ordering scheme for color image filtering[END_REF].

In this paper we propose a new spatial-spectral ordering based on the combination of several stochastic orderings (also called permutations) that each construct a global hamiltonian path on an image. The paper is organized as follows. In section II we present the construction of a global hamiltonian path on an image with 1D permutation ordering on grid-graphs. We then expose in section III how this ordering can take into account both spatial and spectral proximity with the use of nonlocal graphs. Since the proposed ordering depends on a starting pixel, several orders are obtained that are combined using a consensus approach in section IV. Last sections present results and conclude.

II. ONE DIMENSIONAL PERMUTATION ORDERING

We consider that the domain Ω of the image is a grid graph G = (V, E) where vertices V = {v 1 , . . . , v m } correspond to pixels and edges e ij = (v i , v j ) connect vertices with 8adjacency. The notation v i ∼ v j is used to denote two adjacent vertices. With this definition, images are represented as graphs signals [START_REF] Lézoray | Manifold-based mathematical morphology for graph signal editing of colored images and meshes[END_REF] that associate vectors to vertices and are defined by the mapping f : G → T ⊂ R n where T is a non-empty set of vectors (we will consider only RGB color vectors, i.e., n = 3). To each vertex v i ∈ G is associated a vector

v i = f (v i ). The set T = {v 1 , • • • , v m }
denotes the vectors associated to all the vertices of the graph (with a row-major numbering of the image graph vertices). We will use the notation T [i] = v i to denote the i-th element of a set.

To construct an image-adaptive h-ordering, we propose to construct a space-filling curve [START_REF] Chanussot | Bit mixing paradigm for multivalued morphological filters[END_REF] on the grid graph G. This corresponds to the constuction of a global hamiltonian path on all the image: a path that goes through all the vertices of the grid graph and traverses each vertex only one. This was previously explored in [START_REF] Lezoray | Graph-based ordering scheme for color image filtering[END_REF] but with local hamiltonian paths. However, with such a local ordering, the associated dilation and erosion are not strictly speaking morphological operators since they do not commute with the sup. We consider a more general approach and build an order for all the set T in the form of a global hamiltonian path.

Given the set T , the construction of an hamiltonian path amounts to define a sorted permutation P = PT of the vectors of T with P a permutation matrix of size m × m. Let σ be a permutation of the index set I = {1, • • • , m}. If σ(i) = j, then P ij = 1 and 0 otherwise. The induced permutation is

P = PT = {v σ -1 (1) , • • • , v σ -1 (m) } where (σ -1 • σ)
is the identity. Any permutation is not of interest, and spatial and spectral constraints have to be taken into account [START_REF] Chevallier | The irregularity issue of total orders on metric spaces and its consequences for mathematical morphology[END_REF]. To this end we consider the construction of a smooth permutation. The smoothness of an ordered set is expressed by the Total Variation of its elements:

T T V = m-1 i=1 v i -v i+1 (2)
The optimal permutation operator P can be obtained by minimizing the total variation of PT :

P * = arg min P PT T V (3)
This optimization problem provides a permutation such that the corresponding hamiltonian path is the shortest one and is equivalent to solve the traveling salesman problem. This problem being too computationally demanding for large sets, it can be solved using a randomized version of nearest neighbors heuristics presented in [START_REF] Ram | Image processing using smooth ordering of its patches[END_REF], [START_REF] Ram | Image denoising using nl-means via smooth patch ordering[END_REF]. We recall the principle of this algorithm in Algorithm 1. This algorithm starts from an arbitrary vertex and continues by finding the two nearest unexplored neighbor vertices and choosing one of them at random. After the construction of the permutation, we define the hordering from the constructed permutation as h(v i ) = σ(i) and this defines the complete lattice (T , ≤ h ). Given a graph signal f : G → T , a new representation is obtained in the form of the pair (I, P) with I(v i ) = σ(i). When a graph signal is

Algorithm 1 Permutation construction Input: set of vectors T = {v 1 , • • • , v m } and graph G Generate a random probability vector p = [p 1 , • • • , p m ] Randomly choose an index j ∈ I = {1, • • • , m} Set σ -1 (1) = j (or equivalently σ(j) = 1). Set P = {v σ -1 (1) } and J = {σ -1 (1)} for i = 1 to m -1 do Let N (v σ -1 (i) ) = {k|(v σ -1 (i) , v k ) ∈ E} \ J if |N (v σ -1 (i) )| = 1 then σ -1 (i + 1) = N (v σ -1 (i) ) else if |N (v σ -1 (i) )| ≥ 2 then Find the first v j1 and second v j2 nearest neighbors of v σ -1 (i) in N (v σ -1 (i) ) else if |N (v σ -1 (i) )| = 0 then Find the first v j1 and second v j2 nearest neighbors of v σ -1 (i) in I \ J end if end if q i = 1 1+exp v σ -1 (i) -v j 1 -v σ -1 (i) -v j 2 α If (q i < p σ -1 (i) ) then σ -1 (i+1) = j 2 and σ -1 (i+1) = j 1 otherwise. end if P = P ∪ {v σ -1 (i+1) } J = J ∪ {σ -1 (i + 1)} end for
Output: ordered set P and associated index set J encoded in this way, the spectral information is not directly carried by the index I, but is stored in a separate piece of data called a palette: the set P of sorted vectors. Figure 1 presents the obtained permutation and index from an original image. A drawback of the described ordering is its greediness nature. Indeed, as can be seen in Figure 1, the last part of the permutation is not smooth.

The original graph signal f can be directly recovered since

f (v i ) = P[I(v i )] = T [i] = v i .
As exposed in [START_REF] Lézoray | Complete lattice learning for multivariate mathematical morphology[END_REF], [START_REF] Lézoray | Manifold-based mathematical morphology for graph signal editing of colored images and meshes[END_REF], from such a representation, morphological operations can be defined and applied to the graph signal. The erosion and dilation of a graph signal f at vertex v i ∈ G by a structuring element B k ⊂ G are defined as:

B k (f )(v i ) = {P[∧I(v j )], v j ∈ B k (v i )} (4) δ B k (f )(v i ) = {P[∨I(v j )], v j ∈ B k (v i )} . (5)
A structuring element B k (v i ) of size k defined at a vertex v i corresponds to the set of vertices that can be reached from v i in k walks.

III. SPECTRAL-SPATIAL PERMUTATION ORDERING

In Algorithm 1, there is no restriction on the considered graph. However, in section II, we have restricted ourselves to 8-adjacency grid graphs, but there is no reason to do so.

In this section we consider several possible graphs for the construction of the permutation. The graph can use only spatial or spectral information, or both.

• 8-adjacency grid graph (denoted G 0 ): this graph construction connects each vertex to its 8 spatially closest nearest neighbors. • B-adjacency graph (denoted G B ): this graph construction connects each vertex v i to all the vertices contained in a square box of size (2B + 1) × (2B + 1) around v i . • K-Nearest Neighbor graph (denoted G s K ): this graph construction connects each vertex v i to its K nearest neighbors (in terms of spectral distance) within the set of all vertices. The first two kinds of graphs G 0 and G B consider only the spatial proximity information to connect vertices whereas the graph G s K considers only the spectral proximity information and is a nonlocal graph. It can be interesting to construct graphs that combine the use of both spatial and spectral proximity to infer the set of edges. For the graph G s K it is even recommended since its construction cannot ensure that the graph is connected and Algorithm 1 will not be able to reach all vertices. This can be easily obtained by considering the union of the set of edges from the previously considered graphs. Given such a graph construction, with Algorithm 1, a spectral-spatial permutation ordering is then obtained. This can be very interesting since both spatial and spectral proximities are taken into account to build the permutation and this can reduce computation complexity and ensure that relevant nearest neighbors are found. In addition, the spectral proximity used to determine the nearest neighbors within the set of adjacent vertices can consider any distance measure. In particular, instead of using a classical L 2 distance between the vectors v i associated to vertices, one can consider patches p w i = f (x i + t), ∀t ∈ [-w/2, w/2] 2 T of size w × w to obtain a smoother ordering. Figure 2 presents several permutation orderings obtained with different graphs. It can be seen that with G 0 the permutation is not very smooth and the result obtained with a box-constrained graph is much better (graph G 10 ). Finally the permutation ordering can be very different by considering both spatial and spectral information. Last line of Figure 2 illustrates this with two permutations built on the union of a grid graph G 0 and a nearest neighbor graph G s 20 with measuring spectral similarity either with color vectors v i or patches p 3 i of size 3 × 3.

G 0 G 10 G 0 ∪ G s 20 with v i G 0 ∪ G s 20 with p 3 i

IV. CONSENSUS ORDERING

Algorithm 1 is a randomized process that starts from an arbitrary vertex of the graph. As a consequence, at each run of the permutation ordering, if a different starting vertex is chosen, a different result is obtained. In [START_REF] Ram | Image processing using smooth ordering of its patches[END_REF], [START_REF] Ram | Image denoising using nl-means via smooth patch ordering[END_REF], the authors have made the most of this by using several permutations for denoising purposes. We propose to proceed differently and to combine several permutation orderings into a single one. We denote as h i a given ordering and we consider a set of l available orderings. The principle of combining several orderings (also called ranks or permutations) into a single one is known as ranking aggregation, the aim of which being to provide a consensus ranking [START_REF] Ding | A new hierarchical ranking aggregation method[END_REF]. Finding the consensus permutation can be formulated as an optimization problem, which is usually NP-hard [START_REF] Dwork | Rank aggregation methods for the web[END_REF] and heuristic ranking aggregation methods have been proposed. In this paper we consider the following ones [START_REF] Ding | A new hierarchical ranking aggregation method[END_REF].

a) Instant-Runoff aggregation: this aggregation method determines the ranking position of the item with the fewest votes and iterates on the resting items. The process is as follows. One begins by counting the first place votes according to the rankings with respect to the remaining items with R(v j ) = l i=1 δ i (v j ) where δ i (v j ) indicates the first place votes according to permutation h i and is defined by

δ i (v j ) = 1 if v j ≥ hi v k , ∀k = j 0 otherwise (6)
The item with the fewest votes is assigned the lowest position in the obtained consensus rank. b) Borda count: this methods determines the ranking position of all items simultaneously. It assigns each item v j (j ∈ I) a score B i (v j ) based on the positions provided by the permutations h i (i = 1, . . . , l):

B i (v j ) = 1 - h i (v j ) -1 m (7)
According to the scores, Borda count generates a final score

using mean aggregation BS(v j ) = 1 l l k=1 B k (v j ).
The fused ranking is obtained by sorting the aggregated scores of items. c) Weighted Borda count: we propose to modify the Borda count score in order to take into account the smoothness of each permutation. Indeed, in our case, we aggregate permutation orderings that represents smooth shortest paths and we would like to obtain a consensus permutation that is also smooth. To do so, we consider a weighted score defined by where ∇P i (v j ) is a 1D-gradient measured on the permutation ordering from the right and left neighbors of v j in P i Figure 3 illustrates the principle of permutation ordering combination. Given an original image (appears on last row, first column), five different permutation orderings (P 1 , • • • , P 5 ) have been generated using different starting vertices. They are shown in first row of Figure 3. The obtained permutations can be very different from one to another due to the stochastic nature of Algorithm 1. Last row of Figure 3 presents the different consensus permutations that can be obtained with Instant-Runoff, Borda count or weighted Borda count. As it can be seen, the borda count provides a smoother consensus than the Instant-Runoff, and this is enhanced with the proposed weighted version of the Borda count.

B i s (v j ) = B i (v j ) × ∇P i (v j ) (8)

V. RESULTS

Now that we have a method that enables the generation of a consensus permutation from several stochastic ones, we are in position to use the obtained permutation to perform morphological processing of images. In this section we investigate the influence of the different parameters of the approach: the combination method, the graph under construction, the spectral proximity measure. Figure 4 presents an illustration of the influence of these parameters with the result of a closing operation φ = δ with a 5 × 5 square structuring element. The number of combined permutation is 5 and the considered graph is G 10 .

As shown in the two first rows of Figure 4, there is few differences between Instant-Runoff and Borda count consensus combination, and closings with these combination methods are not able to well preserve the object edges in the image. This is not the case for weighted Borda count where the edges are preserved and well-delineated, as expected for a morphological processing. Last row of Figure 4 presents the influence of the graph construction. This corresponds to the weighted Borda count combination of 5 spatial-spectral stochastic permutations with the use of either color vectors or 5 × 5 patches for spectral proximity measurement. As it can be seen the combination of both spatial and spectral can be interesting but only when the proximity measure is accurate enough. Indeed, with color vectors the obtained result does not differ much from the one with only spatial constraints for the graph construction. Inversely, the use of patches has enabled to enhance the result that is much more contrasted, and better preserves edges and texture. This offers a new alternative for nonlocal morphological processing of images. This is all the more visible in Figure 5 Finally we compare the proposed nonlocal processing with the approach of [START_REF] Velasco-Forero | On nonlocal mathematical morphology[END_REF] (denoted NL MM). Figure 6 presents the results of classical MM, non local morphology with [START_REF] Velasco-Forero | On nonlocal mathematical morphology[END_REF] and with our approach. For nonlocal processing 5 × 5 patches are considered. As it can be seen, the approach of [START_REF] Velasco-Forero | On nonlocal mathematical morphology[END_REF] does not provide accurate results and acts more like a gamma correction and artificially enhances background noise. This can be seen on the gradient image that does not correspond to the expected result of a gradient operator. In contrast, our approach has an interpretation that is similar to the classical morphological operators while leveraging the simplification effect to preserve important edges and textures in the image. The gradient image illustrates this and enables a much more accurate extraction of the edges.

VI. CONCLUSION

This paper has introduced a novel approach for nonlocal morphological processing of images. Several stochastic permutation orderings are first constructed so as to obtain smooth paths on a graph. The construction of the graph can benefit from both spatial and spectral information and enables to obtain permutation ordering that rely on patches instead of single colors. The permutation orderings are combined into a single one using a weighted borda count approach to produce a smooth permutation ordering consensus.

Original image B2 δ B2 δ B2 -B2
Classical MM

NL MM [START_REF] Velasco-Forero | On nonlocal mathematical morphology[END_REF] Our approach Fig. 6. Comparison between classical MM, NL MM [START_REF] Velasco-Forero | On nonlocal mathematical morphology[END_REF], and our approach with a 5×5 SE (one approach per line). First line presents the original image.

Columns present respectively an erosion, a dilation and a morphological gradient.

Fig. 1 .

 1 Fig. 1. Example of permutation ordering. First line presents the original image, second line an obtained index I and its associated permutation P.

Fig. 2 .

 2 Fig. 2. Illustration of the influence of graph construction on the obtained permutation ordering.

Fig. 3 .

 3 Fig. 3. Consensus combination of different stochastic permutations.

G 10 ∪ G s 20 with v i G 10 ∪ G s 20 with p 5 iFig. 4 .

 101054 Fig. 4. Influence of the parameters on a stochastic permutation closing with the proposed approach.

Fig. 5 .

 5 Fig. 5. Interest of patch-based processing with, for each row and from left to right, the original image, a local and a nonlocal closing.