María José Verdecho
email: mverdecho@cigip.upv.es

Juan José Alfaro-Saiz

Raúl Rodríguez-Rodríguez

A multi-criteria approach to select suppliers based on performance

Keywords: performance measurement, collaboration, supplier selection

Selecting suppliers is a multi-criteria decision-making problem involving both qualitative and quantitative factors. The aim of this paper is to propose a multi-criteria approach that aids to select suppliers based on two interrelated inputs: overall system performance and supplier performance. With this model, enterprises that are collaborating will have a tool to select suppliers not only based on supplier performance but also aligned with their own strategy increasing the quality of the supplier selection process and, therefore, improving the sustainability of the partnership and their competitiveness.

Introduction

One of the strategies most used to increase/maintain enterprise competitiveness is to establish collaboration relationships with suppliers. Congruent with the need to integrate multiple linked processes in the supply chain, theoretical research advocate that early and extensive supplier involvement results in many benefits, e.g. faster development process, reduced costs, etc. [START_REF] Petersen | Supplier integration into new product development: coordinating product, process and supply chain design[END_REF]. In this context, selecting suppliers becomes a crucial process for manufacturers.

Various studies about customer-supplier relationships point out the importance of considering the performance measurement of the entire supply chain in order to provide products and services that meet the expectations of end customers and promote improvement and innovation of the whole processes. In [START_REF] Mentzer | Managing Supply Chain Collaboration[END_REF], Mentzer defines supply chain collaboration (SCC) as "a means by which all companies in the supply chain are actively working together towards common objectives, and is characterized by sharing information, knowledge, risk and profits". For that reason, it is important to define common performance indicators for all the enterprises that are collaborating as they will aid to focus their efforts towards strategic aspects of their business and, therefore, they will support their competitiveness. In fact, it is acknowledged that the way enterprises measure performance shows the organizational culture and the formulation and deployment of their strategy [START_REF] Pun | A performance measurement paradigm for integrating strategy formulation: a review of systems and framewors[END_REF]. Thus, it is important for those enterprises to define and use a structured performance measurement framework that allows managing performance under various perspectives or dimensions that provide a relevant overview of their performance status. One of the most important performance frameworks developed in the academic literature and business applications is the Balanced ScoreCard (BSC) by Kaplan and Norton [START_REF] Kaplan | The balanced scorecard -measures that drive performance[END_REF]. In fact, possibly due to this acceptance, the BSC developed initially for managing performance of enterprises has been extended by different authors for interorganizational performance management such as the works by Brewer and Speh [START_REF] Brewer | Using the Balanced ScoreCard to measure supply chain performance[END_REF], Bititci et al. [START_REF] Bititci | Measuring and managing performance in extended enterprises[END_REF], Folan and Browne [START_REF] Folan | Development of an Extended Enterprise Performance Measurement System[END_REF] and Alfaro et al. [START_REF] Alfaro | Performance measurement system for Enterprise Networks[END_REF].

In addition, the muti-criteria nature of the process of supplier selection has been widely studied in the literature focusing attention on two main issues: the identification of criteria for the assessment and the application of multi-criteria techniques to pass from the initial criteria to an overall ranking of suppliers [START_REF] Masella | A contigent approach to the design of vendor selection systems for different types of co-operative customer/ supplier relationships[END_REF]. Besides, supplier selection involves considering both qualitative and quantitative criteria [START_REF] Chan | Global supplier development considering risk factors using fuzzy extended AHPbased approach[END_REF]. Therefore, selecting suppliers can be defined as a multi-criteria decisionmaking (MCDM) problem involving both qualitative and quantitative factors.

In this context, it seems reasonable that enterprises that desire to select suppliers need to make their decision based on two main inputs: performance of the enterprises that are collaborating and supplier performance. On the one hand, the enterprises that are collaborating pursue the improvement of the overall system. For that purpose, they should define performance elements (such as performance objectives, performance indicators, etc.) for the whole interorganizational context and select the new supplier that better match those performance elements. In addition, these performance elements are better managed under a structured framework such as the BSC. On the other hand, enterprises need to identify relevant criteria for supplier assessment based on supplier performance as not all the suppliers excel at the same characteristics.

The purpose of this paper is to propose a multi-criteria decision-making approach that aids to select suppliers according to both inputs: overall system performance and supplier performance. With this model, enterprises will have a tool to select suppliers not only based on supplier performance but also aligned with their own strategy increasing the quality of the supplier selection process, its long-term partnership and improving the competitiveness of the whole enterprise association.

The structure of this paper is as follows. First, a literature review of multi-criteria decision analysis methods applied for supplier selection is presented focusing attention on the Analytic Hierarchy Process (AHP) method. Then, the multi-criteria approach to select suppliers is described. Finally, conclusions are exposed.

Literature Review

Several methods have been proposed for solving the supplier selection problem such as vendor profile analysis (VPA), multi-objective programming (MOP), data envelopment analysis (DEA) and analytic hierarchy process (AHP) [START_REF] Chan | Global supplier development considering risk factors using fuzzy extended AHPbased approach[END_REF]. Evaluation and ranking of potential suppliers involves both tangible and intangible criteria. This is because overall assessment of suppliers should not only consider quantitative performance data but also some other criteria that are critical for successful partnerships and are not directly quantifiable, e.g. trust and commitment [START_REF] Mohr | Characteristics of partnership success: Partnership attributes, communication[END_REF]. Therefore, the AHP method developed by Saaty [START_REF] Saaty | The Analytic Hierarchy Process[END_REF] is a useful method to select suppliers as it deals with both types of criteria. In addition, AHP aims at integrating different measures into a single overall score for ranking decision alternatives [START_REF] Rangone | An analytical hierarchy process framework for comparing the overall performance of manufacturing departments[END_REF].

The AHP method has been previously used for supplier selection under a wide variety of applications [START_REF] Perçin | An application of the integrated AHP-PGP model in supplier selection[END_REF]. In [START_REF] Ghodsypour | A decision support system for supplier selection using an integrated analytic hierarchy process and linear programming[END_REF], it is presented an integrated AHP and linear programming method for choosing the best suppliers and placing the optimum order quantities among them. In [START_REF] Masella | A contigent approach to the design of vendor selection systems for different types of co-operative customer/ supplier relationships[END_REF], it is proposed four different vendor selection systems (VSSs) depending on the time frame (short-term versus long-term) and the content (logistic versus strategic) of the co-operative customer/supplier relationships using an AHP framework. In [START_REF] Akarte | Web based casting supplier evaluation using analytical hierarchy process[END_REF], it is proposed an AHP model for casting supplier assessment based on four groups of criteria: product development capability, manufacturing capability, quality capability, and cost and delivery. In [START_REF] Km | Application of AHP in project management[END_REF], it is applied AHP in the field of project management to select the best contractor to perform the project based on six criteria: experience, financial stability, quality performance, manpower resources, equipment resources, and current workload. In [START_REF] Muralidharan | A Multi-Criteria Group Decision making Model for Supplier Rating[END_REF], a multi-criteria group decision making model for supplier ranking based on AHP is developed by combining group member's preferences into one consensus ranking. The criteria used to rate suppliers are quality, delivery, price, technical capability, financial position, past performance attitude, facilities, flexibility and service. In [START_REF] Huan | A review and analysis of supply chain operations reference (SCOR) model[END_REF], an AHP model to structure SCOR (Supply Chain Operations Reference) model metrics to evaluate overall supplier efficiency is proposed. In [START_REF] Chan | Interactive selection model for supplier selection process: an analytical hierarchy process approach[END_REF], it is developed a selection model for supplier selection process using AHP. In [START_REF] Kahraman | Multi-criteria supplier selection using fuzzy AHP[END_REF], a multi-criteria supplier selection procedure using AHP is presented. The first level criteria used to compare suppliers involve: supplier, product and service criteria. In [START_REF] Bittencourt | A systematic approach for VE partners selection using the SCOR model and the AHP method[END_REF], it is developed an AHP approach for virtual enterprise partner selection using the SCOR model and the AHP method. In [START_REF] Chan | Global supplier development considering risk factors using fuzzy extended AHPbased approach[END_REF], it is presented an AHP approach to select global suppliers according to five criteria: cost, quality, service performance, supplier profile and risk factor.

Regarding the combination of AHP and BSC, in [START_REF] Clinton | Implementing the balanced scorecard using the analytical hierarchy process[END_REF], it is developed a model to implement a BSC framework. In [START_REF] Searcy | Aligning the balanced scorecard and a firm's strategy using the analytic hierarchy process[END_REF], it is proposed a model to align the BSC and a firm's strategy using the AHP. In [START_REF] Chiang | Decision Approach for Long-Term Vendor Selection Based on AHP and BSC[END_REF], a model for long-term vendor selection based on vendor performance is presented. In [START_REF] Bhagwat | Performance measurement of supply chain management using the analytical hierarchy process[END_REF], a model for selecting performance indicators for supply chain management is presented. In [START_REF] Lee | A fuzzy AHP and BSC approach for evaluating performance of IT department in the manufacturing industry in Taiwan[END_REF], an approach for evaluating performance of IT department in the manufacturing industry in Taiwan is exposed. In [START_REF] Cebeci | Fuzzy AHP-based decision support system for selecting ERP systems in textile industry by using balanced scorecard[END_REF], it is proposed a decision support system for selecting ERP systems in textile industry by using the BSC.

However, there is not a specific model developed for selecting suppliers that integrates performance information of suppliers as well as the overall system performance. For this reason, the purpose of the remaining of this paper is to present a multi-criteria BSC-AHP model for supplier selection that fills this research gap. With this approach, enterprises that are collaborating and have defined a BSC framework (or desire to do it) will have a tool to select suppliers based not only on specific performance data of suppliers but also aligned to their common strategy and, therefore, have a tool to improve their competitiveness. In addition, supplier performance and overall performance are interrelated as there is probably a supplier that can contribute more than the other suppliers to enhance overall efficiency. With this approach, both inputs are connected to provide an overall rating of suppliers for decision-makers.

3 The proposed Multi-criteria Approach for Supplier Selection

Description of the Model

The AHP method structures the decision problem in a hierarchy of levels. These levels are linked by unidirectional dependence relationships. In the upper level of the hierarchy, it is defined the ultimate goal of the decision problem. Then, the criteria that contribute to achieve the goal stand in the second level. In the next levels, intermediate subcriteria and attributes that compose the hierarchical structure are located. Finally, in the last level, the decision alternatives are established. Using levels allows decision makers to focus on a small set of decisions [START_REF] Saaty | The Analytic Hierarchy Process[END_REF]. By making pairwise comparisons and using the fundamental scale of Saaty [START_REF] Saaty | The Analytic Hierarchy Process[END_REF], the AHP method provide relative weights to each element within a level depending on its contribution to an element linked to it that is located on the immediate upper level.

In order to apply AHP, we have defined four phases. In the first phase, the criteria and attributes to improve overall efficiency are defined and the AHP model is composed. Once the AHP model is obtained, the second phase consists of making pairwise comparisons within each level and obtaining the relative priorities. The third phase aims at calculating the overall priorities of the decision alternatives. Finally, the fourth phase deals with sensitivity analysis of the solution provided. It has to be noted that this paper deals with the description of the phase 1.

The multi-criteria model developed for selecting suppliers is founded on two main interrelated inputs: overall performance and supplier performance. Figure 1 shows the hierarchical structure of the BSC-AHP model. The main goal is to select a supplier to improve overall interorganisational efficiency so that this goal stands on the top of the hierarchical structure. In order to manage this goal, different performance indicators are defined. The performance indicators that are to be monitored by the partners are the main criteria of the model as the achievement of the objectives related to these performance indicators will benefit the overall performance. Thus, at this phase, it is suggested to define performance objectives and indicators according to the four performance perspectives of the BSC [START_REF] Kaplan | The balanced scorecard -measures that drive performance[END_REF]: financial perspective, customer perspective, internal business process perspective and innovation & learning perspective. In order to support these performance indicators, supplier performance information has to be evaluated under different supplier dimensions. Therefore, there is a direct link between the performance of the supplier and the performance of the overall system. That is the reason why supplier dimensions stand on a layer below overall performance indicators as the performance of the supplier has to contribute to the consecution of the performance of the overall system. In order to define the supplier performance data, the model uses the conceptual framework by Croom [START_REF] Croom | The dyadic capabilities concept: examining the processes of key supplier involvement in collaborative product development[END_REF] for supplier involvement which considers both operational and relational competencies and comprises three dimensions: product, structure and interaction dimensions.

Finally, the last level is composed of the potential suppliers (1, 2, … n). Suppliers will be pairwise evaluated to know which one best performs on the performance dimensions previously defined. Thus, the BSC-AHP model is composed by these four main components: overall goal, performance perspectives, supplier attributes and potential suppliers. The product criterion comprises the main attributes that are to be addressed when assessing the product dimension of the supplier. Some relevant attributes of the product dimension include [START_REF] Chan | Global supplier development considering risk factors using fuzzy extended AHPbased approach[END_REF][START_REF] Akarte | Web based casting supplier evaluation using analytical hierarchy process[END_REF][START_REF] Kahraman | Multi-criteria supplier selection using fuzzy AHP[END_REF]: quality, price, development time, flexibility, and research and development (R&D) initiatives. Quality is one of the most important product attributes. It relates to the historical rejection rate during a period of time of the products delivered by the supplier. Rejection is due to deviations from specifications in the design, manufacturing, or packaging of the product. It also considers deviations from the specified quantities or delivery dates in the customer order. The second attribute is the price of the product as it affects to the bottom-line. Development time refers to the competence of the supplier to design, develop and launch products within the agreed period of time according to the product specifications. Flexibility involves the response time of the supplier when product changes are needed. It also considers the response time to new orders or order modifications during the development and manufacturing stages. Finally, R&D initiatives measure the ability of the supplier to provide support during product development and manufacturing. It is an important attribute as most products, after launching, demand continuous improvement to remain competitive. The second criterion is structure. Structure comprises the capabilities/procedures for developing products and processing materials/components as well as the systems to facilitate control, co-ordination and communication through organizational and interorganizational systems [START_REF] Croom | The dyadic capabilities concept: examining the processes of key supplier involvement in collaborative product development[END_REF]. The first attribute within the structure criterion is dedicated cross-functional team which assesses the human compromised capabilities of the supplier into the relationship. The second attribute is project management methodology which measures the degree of knowledge and implementation of project management practices in the supplier organization as well as the compatibility with the project management practices of the rest of enterprises. The third attribute is quality methodologies. The fourth attribute is Information Technology and Information Systems (IT & IS). It assesses the extent of technology implementation and interoperability of supplier information systems in order to send/receive and use the information exchanged with the rest of enterprises. The role of the technology is an important attribute as effective collaboration it is highly influenced by seamless communication between supplier and manufacturer. The fifth attribute is process alignment. It evaluates the extent of business process interoperability defined as the "ability of different processes to work together and exchange information, data, control information, etc." [START_REF]Interop. Interoperability ontology. Interop-vlab platform[END_REF]. The sixth attribute is complementary capabilities. It measures the degree of interdependence on assets as well as the capacity on development/manufacturing so that the collaborative relationship can develop and manufacture higher variety/amount of products to increase the market share. Finally, the financial profile attribute assesses the past and current financial condition of the supplier in order to support/invest in the long-term [START_REF] Chan | Global supplier development considering risk factors using fuzzy extended AHPbased approach[END_REF]. The third criterion is the interaction criterion which deals with the relational side of collaboration. Based on [START_REF] Mohr | Characteristics of partnership success: Partnership attributes, communication[END_REF], the relational attributes considered in the AHP model are: coordination, commitment, trust, information sharing, and conflict management. Coordination involves the tasks that are to be taken for linking activities performed by the different members in a seamless manner. Commitment refers to the willingness of the supplier to perform effort on behalf of the relationship. It is the establishment of the foundation of the relationship and it is based on being supportive in solving problems together. A high level of commitment provides the context for the achievement of individual and mutual goals. Trust is based on the belief that the partner is reliable and will fulfil its responsibilities acting fairly. A partner trusts another partner if considers that decisions made by this last one will be in the interest of both parts. Information sharing considers the timeliness, accuracy, adequacy and completeness of the relevant information exchanged. Finally, conflict management measures the degree of intensity and conflict resolution mechanisms that exist between manufacturer and supplier.

Conclusions

This paper introduces a multi-criteria approach to select suppliers for collaborative relationships based on two types of performance information: supplier performance information and overall performance of the collaborating enterprises. Performance data of the suppliers is the common performance information considered in the literature for supplier selection problems. In addition, we have introduced the BSC of the whole collaborative enterprises as it is important that the supplier selected should contribute the most to the overall efficiency improvement. Therefore, in our approach, supplier performance information is linked to the common performance indicators defined by all the enterprises that are collaborating. With this approach, enterprises will have a tool to select suppliers aligned with their own strategy increasing the quality of the supplier selection process, its long-term partnership and improving the competitiveness of the whole enterprise association.

Fig. 1 .

 1 Fig. 1. BSC-AHP model: main components Figure 2 shows an example of a detailed BSC-AHP model for partner selection. As can be observed, the second level (performance perspectives) is further deployed to show their performance indicators within each performance perspective. Similarly, the supplier attributes within each dimension are also shown. Based on Croom's work, the attributes comprising the dimensions (product, structure and interaction) are described as follows.The product criterion comprises the main attributes that are to be addressed when assessing the product dimension of the supplier. Some relevant attributes of the product dimension include[START_REF] Chan | Global supplier development considering risk factors using fuzzy extended AHPbased approach[END_REF][START_REF] Akarte | Web based casting supplier evaluation using analytical hierarchy process[END_REF][START_REF] Kahraman | Multi-criteria supplier selection using fuzzy AHP[END_REF]: quality, price, development time, flexibility, and research and development (R&D) initiatives. Quality is one of the most important product attributes. It relates to the historical rejection rate during a period of time of the products delivered by the supplier. Rejection is due to deviations from specifications in the design, manufacturing, or packaging of the product. It also considers deviations from the specified quantities or delivery dates in the customer order. The second attribute is the price of the product as it affects to the bottom-line. Development time refers to the competence of the supplier to design, develop and launch products within the agreed period of time according to the product specifications. Flexibility involves the response time of the supplier when product changes are needed. It also considers the response time to new orders or order modifications during the development and manufacturing stages. Finally, R&D initiatives measure the ability of the supplier to provide support during product development and manufacturing. It is an important attribute as most products, after launching, demand continuous improvement to remain competitive. The second criterion is structure. Structure comprises the capabilities/procedures for developing products and processing materials/components as well as the systems to facilitate control, co-ordination and communication through organizational and

Fig. 2 .

 2 Fig. 2. BSC-AHP detailed model.

Acknowledgments. This work has been developed within the framework of a research project funded by the Polytechnic University of Valencia, titled "Design and Implementation of Performance Measurement Systems within Collaborative Contexts for aiding the Decision-making Process", reference PAID-06-08-3206.


María José Verdecho
email: mverdecho@cigip.upv.es

Juan José Alfaro-Saiz

Raúl Rodríguez-Rodríguez

COL-PMS: a collaborative performance measurement system

Keywords: performance measurement system, collaboration, interorganizational relationships

Enterprises look for new strategies to increase/maintain their competitiveness. One of these strategies consists of establishing collaboration relationships with other enterprises. Collaborating enterprises need tools and methods that facilitate the management their performance. Performance Measurement Systems developed in the literature have overlooked the development of frameworks that include the social side of collaboration relationships under an integrated and structured approach. The purpose of this paper is to introduce the COLlaborative Performance Measurement System (COL-PMS) framework which overcomes this gap.

Introduction

Global competition, higher specifications of products/services, and fast technological changes are some of the factors that are boosting enterprises to adopt new business models to remain competitive in the marketplace. In this context, some enterprises that traditionally operated with their suppliers under a transactional approach, have shifted their strategy and adopted collaborative initiatives. Collaboration involves two or more independent enterprises working together to align their processes with the goal of creating value to end customers and stakeholders with greater success than acting alone [START_REF] Simatupang | Applying the theory of constraints to supply chain collaboration[END_REF]. It implies sharing information, resources, responsibilities, knowledge, risk and profits among the partners [START_REF] Mentzer | The Nature of Interfirm Partnering in Supply Chain Management[END_REF]. In addition, in the current business environment, information and communication technologies play a fundamental role increasing the interoperability among the different enterprises and supporting the different processes and activities developed by the collaborating enterprises.

Performance measurement is "a process of quantifying the efficiency and effectiveness of pass action" [START_REF] Neely | The Performance Prism: The Scorecard for measuring and managing business success[END_REF]. The complexity of collaboration environments requires new tools that aid to define and collect the necessary information for measuring the performance of enterprises. For that reason, Performance Measurement Systems (PMSs) should include in their structure special characteristics to support this issue. In supply chain management, interorganizational relationships rely on "hard" (technological and infrastructural) and "soft" (social and behavioural) aspects [START_REF] Harland | Supply Chain Management: Relationships, Chains and Networks[END_REF][START_REF] Burgess | A proposed integrated framework for analysing supply chains[END_REF]. Around 50% of interorganizational performance drivers are considered to rely on people factors while process and technology factors represent 30% and 20% respectively [START_REF] Zaklad | The social side of the supply chain[END_REF]. In fact, in real assessments of business relationships, "hard" aspects have been addressed while "soft" aspects appear to be overlooked [START_REF] Staughton | Operational performance gaps in business relationships[END_REF]. For that reason, one of the lacks of PMS for collaborative contexts is the inclusion of mechanisms and elements that aid to manage jointly hard and soft aspects under an integrated framework.

The purpose of this paper is to introduce a PMS for collaborative contexts, called (COLlaborative Performance Measurement System (COL-PMS)), that fills this gap by including both (hard and soft) aspects within its structure in order to provide a tool for managing the performance of collaboration relationships more efficiently and effectively.

The structure of this paper is as follows. First, a literature review of PMS for interorganizational contexts is presented. Then, the COL-PMS framework is described. Finally, conclusions are exposed.

Background

The behavioral, social or relational side of collaboration consists of a set of aspects or elements that describe the interaction among the different partners. In [START_REF] Mohr | Characteristics of partnership success: Partnership attributes, communication[END_REF], it is suggested that successful partnerships rely on three types of characteristics: attributes of the partnership (commitment, coordination, trust, etc.), communication behavior (such as information sharing) and conflict management techniques.

Based on [START_REF] Mohr | Characteristics of partnership success: Partnership attributes, communication[END_REF], a literature review regarding PMS for interorganizational environments is performed to analyze if PMS developed support the behavioral side of collaboration. In [START_REF] Brewer | Using the balanced scorecard to measure supply chain performance[END_REF], a PMS is proposed using the Balance Scorecard (BSC) [START_REF] Kaplan | The balanced scorecard -measures that drive performance[END_REF] to measure supply chain (SC) performance. Their work presents the classical four perspectives (business process, customer, financial, and innovation and learning), generic goals of the SC and examples of performance indicators for these goals. Regarding the social elements, two of the goals to accomplish within the innovation and learning perspective are "partnership management" and "(shared) information flows". For these goals, the performance indicators "product commitment ratio" and "number of shared data sets with respect to total data sets" are defined. However, other aspects could be included to manage SC performance such as trust and commitment.

In [START_REF] Gunasekaran | Performance measures and metrics in a supply chain environment[END_REF], it is presented a framework for supply chain performance measurement that classifies metrics within three levels of management: strategic, tactical and operational. The authors also indicate the importance of measuring partnerships and present a list of partnership evaluation parameters such as "level and degree of information sharing" and extent of "mutual assistance in problem solving". Nevertheless, the framework could be extended by considering other elements such as trust, coordination and conflict management.

In [START_REF] Chan | Feasibility of performance measurement system for supply chain: a process-based approach and measures[END_REF], it is developed a process-based PMS for supply chains. The work establishes a set of steps to decompose the core supply chain processes into lower level processes. The PMS associates goals, responsibility and function, and their respective performance measurements to each level and each element within each level (subprocesses, activities). It identifies that the strategy of the supply chain should be defined to identify the core processes. Nevertheless, social collaborative elements (information sharing, trust, etc.) can be intuited to be within the subprocesses and activity borders, but they are not explicitly defined.

In [START_REF] Bititci | Measuring and managing performance in extended enterprises[END_REF], it is proposed a PMS for Extended Enterprises (EEs). The PMS comprises three levels: enterprise, business unit and extended processes. The work considers strategic and operational coordinating measures which should include collaboration measures. The authors present a case study where the performance measurement of 'number of inter-partner strategic conflicts' is included in the EE scorecard. However, the work does not detail the extent of collaborative elements to be measured through the coordinating measures.

In [START_REF] Folan | Development of an Extended Enterprise Performance Measurement System[END_REF], it is presented a PMS for EE. The PMS consists of two frameworks: a structural and a procedural framework. The structural framework comprises two levels: individual enterprise and EE PMS. At the individual level, each node PMS is composed of four perspectives: internal, supplier, customer and EE perspectives. However, the PMS could be further detailed by considering the social elements.

In [START_REF] Chalmeta | Performance measurement systems for virtual enterprise integration[END_REF], it is developed a PMS methodology for virtual enterprises. It defines two levels: virtual and individual enterprise. The work exposes the necessity to assess relationships within the PMS. The authors present a case study where the elements 'culture' and 'interpersonal relationships' are included in the 'virtual enterprise' and 'learning and growth' perspectives respectively but they do not specify how they are to be measured.

In [START_REF] Alfaro | Performance measurement system for Enterprise Networks[END_REF], it is presented a PMS for supply networks composed of a methodology and a framework. The work identifies that equity and trust have to be maintained through the functional levels, perspectives and performance structure. However, the PMS does not consider how to integrate and measure these collaborative elements.

In [START_REF] Bititci | Managing synergy in collaborative enterprises[END_REF], it is presented an audit model to assess an enterprise readiness for collaboration. Although the model is not structured as a PMS, it is considered within this group of the typology because it intends to evaluate the antecedent stage of a collaboration relationship. The model comprises four perspectives: strategic, operational, cultural and commercial. The cultural and commercial perspectives pursue to assess the compatibility of culture (trust, management style, information sharing, etc.) and commercial position (risk, investments, etc.) among enterprises. The work could be further extended by considering other social aspects (cooperation, coordination, conflict management, etc.).

In [START_REF] Romero | A conceptual Model for Virtual Breeding Environments Value System[END_REF], it is developed a PMS for Virtual Organizations Breeding Environments (VBEs). The objective of a VBE is to prepare its member's organizations for their potential involvement in a collaborative opportunity-based virtual organization. The PMS is based on the four perspectives by Kaplan and Norton [START_REF] Kaplan | Strategy maps: converting intangible assets into tangible outcomes[END_REF]. Collaboration elements are incorporated in all four PMS perspectives such as developing collaborative opportunities (outcome perspective), partnership development (stakeholder perspective), trust and relationship management (internal perspective), and culture, alignment of goals and collaboration (learning and growth perspective).

However, although the work exposes the strategic map and performance indicators for measuring collaborative elements, a PMS structure is not explicit.

In [START_REF] Westphal | Measuring collaboration performance in virtual organizations[END_REF], it is proposed a PMS for measuring collaboration performance in virtual organizations. The PMS is based on the four perspectives by Kaplan and Norton [START_REF] Kaplan | The balanced scorecard -measures that drive performance[END_REF] and a fifth perspective for measuring collaboration that comprises five subperspectives: reliability, flexibility, responsiveness, commitment and communication. However, it would be necessary to explain further the extent of the five subperspectives and how these subperspectives are related to the rest of perspectives in order to define an integrated and solid PMS.

As a conclusion, it can be stated that although some of the PMSs reviewed include the measurement of some social elements in their structure, there is a clear lack of a collaborative PMS that considers the social side of the collaboration that allows managing the performance of collaborating enterprises under a solid performance structure. The framework proposed on this paper aims to fill this research gap.

COL-PMS Framework

Description of COL-PMS

From the literature review, it can be observed that there is a need of methods, systems and procedures that establish the steps to be followed to manage performance within collaborative contexts considering the social side of collaboration and following an integrated approach. The COL-PMS framework introduces these characteristics based on the PMS developed by Alfaro et al. [START_REF] Alfaro | Performance measurement system for Enterprise Networks[END_REF] which is founded on three phases: 1) definition of the strategic framework, 2) definition of the process framework and 3) monitoring. The characteristics of a PMS for collaborative environments are related to the requirements that should be covered by the PMS in order to be considered solid and integrated. This implies that the PMS should provide all the necessary functionalities to approach the context for which it was developed. Additionally, this PMS should support the decision-making process of the enterprises and entities that collaborate. For that reason, it is necessary that the PMS considers two levels: interorganizational level (where collaboration takes place) and individual enterprise level. Both levels should be aligned in order to keep traceability among the performance elements that are to be defined.

At the individual enterprise level, COL-PMS derivates from the vision and strategy and reflects the most important aspects of the business. If this concept is extended within the interorganizational context, it can be said that it is a process of strategic planning for all the partners and implies a common understanding of their aims what facilitates the evaluation and degree of success reached in their objectives and strategies. Thus, COL-PMS starts with a strategic approach for its adequate interpretation and application. Therefore, the starting point of COL-PMS is the definition of the strategic framework (phase 1). Figure 1 shows the composition of the COL-PMS generic framework which distinguishes between two types of frameworks: strategic and process framework. In detail, the definition of the strategic framework needs to incorporate all the performance elements (philosophical planning (mission and vision), stakeholder requirements, objectives, strategies, critical success factors and key performance indicators (KPIs). All these elements at defined for the four performance perspectives [START_REF] Kaplan | The balanced scorecard -measures that drive performance[END_REF]: financial, customer, process and learning & growth. These perspectives aid to structure performance measurement following relationships of cause-effect.

Fig. 1. COL-PMS Framework

In addition, it is necessary to introduce one perspective oriented to manage collaboration relationships that covers the social side of collaboration. This perspective supports the consecution of the other four perspectives, due to the fact that is related to the aspects of the collaborative culture which provides a mediator effect or social climate above the components of the rest of perspectives. Therefore, this perspective will consist of a set of objectives, strategies, critical success factors and KPIs related to relational characteristics such as the ones described by [START_REF] Mohr | Characteristics of partnership success: Partnership attributes, communication[END_REF]: relationship attributes (trust, commitment, cooperation, etc. Commitment refers to the willingness of the supplier to perform effort on behalf of the relationship. It is the establishment of the foundation of the relationship and it is based on being supportive in solving problems together. A high level of commitment provides the context for the achievement of individual and mutual goals. There are different types of commitment depending on the effort (to continue the relationship), loyalty, contract duration or their combination.

Coordination involves the tasks that are to be taken for linking activities performed by the different members in a seamless manner. The coordination degree between manufacturer and supplier is an important attribute of the relationship as it allows moving together towards the achievement of mutual objectives.

Trust is based on the belief that the partner is reliable and will fulfill its responsibilities acting fairly. A partner trusts another partner if considers that decisions made by this last one will be in the interest of both parts. There are different types of trust depending if they are founded on deterrence, reliability, competence, goodwill, loyalty or their combination.

Information sharing considers the timeliness, accuracy, adequacy and completeness of the relevant information exchanged. It is the degree of private information that enterprises share in order to perform their activities and make decisions. It is one of the main flows through the companies and it defines the depth and width of the other two main flows (material and monetary) and develops the flow of social relationships.

Finally, conflict management measures the degree of intensity and conflict resolution mechanisms that exist between manufacturer and supplier. The existence of conflict is inherent to interpersonal as well as interorganizational relationships. However, the manner this conflict is managed is essential to the long-term and stability of the relationship.

It is not the purpose of this paper to provide a full overview of social characteristics of collaboration relationships but to present the elements that a PMS should have to manage them. For that reason, the social characteristics described are only given as examples that are to be considered but do not conform a definitive list.

In addition, it has to be noted that collaboration relationships perspective is mainly related to the interorganizational functional level as it is the level in which interorganizational collaboration takes place. However, social aspects between companies are not isolated from the individual enterprise level as it happens with the other four perspectives. Therefore, the consecution of the interorganizational level is linked to the individual enterprise level.

On the other hand, in the definition of the process framework (phase 2), it is not required to incorporate the philosophical elements (mission and vision) or the stakeholder requirements, excepting special circumstances that require it. Once the strategic framework is obtained, performance elements of the process framework (objectives, strategies, critical success factors and KPIs) are defined for those key business processes associated to the collaborative context. These key processes are processes directly linked to a common product/service produced by the partners or processes that support the success of the production of those products/services.

In the previous phases, all the performance elements have been defined. This last phase aims at monitoring of all those elements in order to know which are the most important elements, what levels are the most relevant from a performance management point of view and where are located special indicators for a specific partner. For that reason, it is convenient show graphically the deployment of all the elements by elaborating two types of diagrams: graphics of global deployment and graphics of partial deployment. The graphics of global deployment represent the deployment of the basic performance measurement elements for all the functional levels (interorganizational context level and individual enterprise level). Consequently, there will be one graph for the objectives, another for the strategies, another for the critical success factors and, finally, another for the key performance indicators. Then, such graphics will also be constructed for both the strategic and the process framework. Figure 2 shows an example of graphic of global deployment for the objectives.

Conclusions

This paper has reviewed the literature regarding interorganizational PMSs in order to analyse if the social aspects of collaboration are managed through a solid and integrated PMS structure. Based on the gaps coming from the literature review, we have introduced a new collaborative PMS, called COL-PMS, which includes the social side of collaboration within its structure in order to provide a tool for managing the performance of collaboration relationships more efficiently and effectively. This framework considers five performance perspectives in order to manage collaborative performance. In fact, there is a need to consider a specific perspective for managing the relational aspects of collaboration relationships as they act as a mediator element for reaching the elements that compose the other performance perspectives. In addition, we have described the elements that integrate the COL-PMS framework so that enterprises that desire to collaborate or are collaborating have a tool for aiding to define and collect performance management information.

  ), communication behaviour (information sharing) and conflict resolution techniques. The descriptions of these characteristics are as follows (Boddy et al., 2000; Hanfield and Betchel, 2004; Lejeune and Yakova, 2005):

Fig. 2 .

 2 Fig. 2. Graphic of Global Deployment: Objectives. The graphics of partial deployment complements the graphics of global deployment providing the combination of elements necessary for measuring performance for each level (interorganizational level and individual enterprise level)

Acknowledgments. This work has been developed within the framework of a research project funded by the Polytechnic University of Valencia, titled "Design and Implementation of Performance Measurement Systems within Collaborative Contexts for aiding the Decision-making Process", reference PAID-06-08-3206.


Sofiane Lagraa
email: sofiane.lagraa@inria.fr

Jérome François
email: jerome.francois@inria.fr

Knowledge Discovery of Port Scans from Darknet

Port scanning is widely used in Internet prior for attacks in order to identify accessible and potentially vulnerable hosts. In this work, we propose an approach that allows to discover port scanning behavior patterns and group properties of port scans. This approach is based on graph modelling and graph mining. It provides to security analysts relevant information of what services are jointly targeted, and the relationship of the scanned ports. This is helpful to assess the skills and strategy of the attacker. We applied our method to data collected from a large darknet data, i.e. a full /20 network where no machines or services are or have been hosted to study scanning activities.

I. INTRODUCTION

Computers connected to a network use many services by mainly relying TCP/UDP protocols. Port or IP scanning (also known as sweeping) is one of the most common techniques by attackers to discover open ports in preamble of an attacker or an intrusion through those ones. Hence, scanning methods are part of network-based discovery techniques, still for emerging threats like Advanced Persistent Threats [START_REF] Brewer | Advanced persistent threats: minimising the damage[END_REF]. Therefore, an indepth understanding of scanning techniques is necessary for improving security: detection, prevention or forensics. There are three main types of scans: vertical, horizontal and block scans. Vertical scan is described as a single IP being tested on multiple ports. Horizontal scan is described as trying scan against a group of IPs for a single port. Block scan is a combination of both of them. All of them cannot.

This paper aims at making a deeper and more sophisticated analysis of vertical scans by seeking the relationship of commonly scanned TCP ports. Our approach relies on building dependencies among those latter and extract the predominant roles of certain ports within sequence of consecutive targeted ports as well as extracting relationships among them, i.e. discover groups of commonly scanned ports. This can help the security analysts to guard against attacks and improve prevention and detection tools. For example, given the scans performed on the following successive ports 80, 591, 8008, 8080 and 443, we want to discover the relationship of commonly scanned ports with their dependencies, then the significant service ecosystem is that these ports are used for "HTTP" traffic. Thus, the analysts can update the web server security. We have validated the method on real data collected from a darknet in order to observe and understand [START_REF] Balkanli | On the analysis of backscatter traffic[END_REF] the behavior of different scans and (2) extract the "modi operandi" of targeted ports.

In this paper, our contributions are as follows: 1) a graph-based model of port scans relationships, 2) a knowledge discovery methodology using graph mining techniques based on the proposed model,

3) an application of our methodology on real darknet data. The rest of the paper is organized as follows. Section II briefly presents the background on darknet and related work of analysis techniques of ports scans. Section III provides an overview of dataset and highlights research questions. Section IV and Section V present the first and the second part of our approach concerning the port-scan graph model and port-scan graph analytic, respectively. Finally, Section VI draws conclusion and discuses the future work.

II. BACKGROUND AND RELATED WORK

A vertical port scanner sends a message to each port of predefined list and waits for a certain replies. For example, a common manner (TCP SYN-scan) to discover TCP open ports is to try to initiate new connections with SYN-flagged packets (TCP SYN scan). If the port is open, a SYN/ACK packet is sent back while a RST packet is representative of a closed port.

From darknet data we have collected, we conduct the study over all TCP-SYN packets which represent 93,79% of TCP packets. The darknet (network telescopes) refers to the servers configured to trap adversaries and collect suspicious data. Since these servers run without interacting with no legitimate users (attackers) in passive mode and correspond to unused hosts or devices, any observed traffic destined to them raises suspicion and hence necessitates investigation. They do not offer Internet services, nor does it use any. Darknet has been used in the past to extract different insights on probes or scanning activities [START_REF] Bellovin | There be dragons[END_REF], [START_REF] Fachkha | Darknet as a source of cyber intelligence: Survey, taxonomy, and characterization[END_REF], bots, DDoS (Distributed Denialof-Service) attacks due to victims reply (backscatter) packets to spoofed IP addresses [START_REF] Moore | Inferring internet denial-of-service activity[END_REF]. For instance, DDoS attacks and scanning activities can be detected using the existing techniques and tools such as IDS Suricata1 , topological analysis technique [START_REF] Coudriau | Topological Analysis and Visualisation of Network Monitoring Data: Darknet case study[END_REF], and statistical techniques [START_REF] Durumeric | An internet-wide view of internet-wide scanning[END_REF]. However, these tools and techniques provide scanning profiles of ports such as the number of countries are responsible for the majority of large scans, the number of performed scans, and the targeted services.

Darknets are also known as darkspace, blackhole monitors, unused IP addresses, or network telescopes. To harmonize the terminology, we use the word darknet throughout this paper. A darknet is a whole subnetwork, which is announced over Internet such that packets sent to the IP addresses are properly routed over. This subnetwork is defined by the prefix length. It does not host any services and so no legitimate traffic is supposed to reach it. The entity hosting the darknet is then silently collecting all incoming packets, i.e. without replying to any of them. They have been proved to be appropriate monitoring techniques for security, e.g. to observe botnets, scans or distributed denial-of-service attacks [START_REF] Bellovin | There be dragons[END_REF], [START_REF] Fachkha | Darknet as a source of cyber intelligence: Survey, taxonomy, and characterization[END_REF], [START_REF] Balkanli | On the analysis of backscatter traffic[END_REF]. Although there can be primarily seen as very naïve collector, especially when compared to more advanced Internet security monitoring approaches like honeypots, they have been demonstrated to be complementary [START_REF] Francois | Activity monitoring for large honeynets and network telescopes[END_REF]. In [START_REF] Balkanli | On the analysis of backscatter traffic[END_REF], the authors discovered patterns of darknet traffic and how it changes over time.

In [START_REF] Dainotti | Analysis of a "/0" stealth scan from a botnet[END_REF], the authors presented the measurement and analysis of a 12-day world-wide cyber scanning campaign targeting VoIP (SIP) servers using a darknet. Similarly, in [START_REF] Durumeric | An internet-wide view of internet-wide scanning[END_REF], behavior of horizontal scanning scans have been researched on. In [START_REF] Van Riel | Inetvis, a visual tool for network telescope traffic analysis[END_REF], the authors presented an animated 3-D scatter plot visualization of port scanning on darknet data. Finally, in [START_REF] Coudriau | Topological Analysis and Visualisation of Network Monitoring Data: Darknet case study[END_REF], a topological data analysis technique is able to analyze and visualize a large number of IP packets in order to make malicious activities patterns easily observable by security analysts. Among these patterns, they found scanning activities observed by a darknet.

Compared to previous approaches, it is worth to mention that our work focuses on vertical scan. Besides, most of current studies either rely on evaluating trends over targeted ports or behavior of a scan towards a single ports. Our approach tends to automatically discover dependencies among targeted ports rather than deducing those relations thanks to observation like in [START_REF] Dainotti | Analysis of a "/0" stealth scan from a botnet[END_REF]. In that sense it is closer to work related about automated discovery of network service dependencies [START_REF] Peddycord | On the accurate identification of network service dependencies in distributed systems[END_REF]. However, the goal of the latter is to discover normal dependencies that exist in benign applications whereas ours establish relations in attacker behaviors when performing TCP scans. To achieve that, we provide behavior pattern of port-scan through graph mining methods which provide intrinsic analysis of scanned ports where the classical statistical tools do not allow to discover.

III. DATASET OVERVIEW AND RESEARCH QUESTIONS

Darknet data used in this paper are collected from a /20 darknet network (i.e., 4096 addresses) for 2 years (Nov.2014 ∼ Nov2016). During this period 2,884,539,435 packets were captured representing 500 GB of data with an increasing trend over the months as shown in Figure 1. As highlighted in Table I, the number of TCP packets represents more than 78% of packets. This is a challenging task to analyze port-scan strategies from a huge amount of traffics.

Since, there are 65,536 possible TCP ports, we segment the scanned port numbers on three ranges according to Internet Assigned Numbers Authority (IANA): system or well-known Fig. 1: Number of packets according to the months ports (0-1023), user or registered ports/or vendors use for applications (1024-49151), and the dynamic and/or private or ephemeral ports (49152-65535). The difference uses of these ranges is described in RFC6335 [START_REF] Cotton | Internet Assigned Numbers Authority (IANA) Procedures for the Management of the Service Name and Transport Protocol Port Number Registry[END_REF]. Figure 2 shows the segmentation of the different scanned Port numbers during the 24-months. The system ports are frequently targeted compared to the user ports and dynamic ports. Indeed they contain the majority of well used standardized services, especially web services (HTTP and HTTPS) which are now a de facto standard for majority of applications. However, as attackers may also target specific and specialized ports, we conduct the study over all 2,125,080,102 TCP-SYN packets which represent 93,79% of TCP packets. Indeed, through priori analysis with the Suricata IDS2 and our own tool [START_REF] Coudriau | Topological Analysis and Visualisation of Network Monitoring Data: Darknet case study[END_REF], we discovered that most of TCP-SYN packets are related to port scanning even if other types of traffic like DDoS are also present. Discarding those few ones is risky as they are being mixed with scans. Keeping their related packets for analysis has zero impact since they are targeting a unique TCP port although our approach aims to find correlation over multiple TCP ports.

In this paper, we focus on the last four months from 08-2016 to 11-2016 which represent a higher number of TCP-SYN packets and more significant results. Table II represents the top-10 of probed ports in each month. We see that the services relating to traffic, servers and database are most targeted services in scans. However, these statistical results do provide the relationships between the probed ports.

Our aim is to discover, analyze the relationship groups of ports probed together, and so understand the attacker scanning strategies, especially by answering the following questions:

• Does it exist particular ports which are frequently probed in vertical scanning ? In order to answer these questions, we propose a graphbased modelling approach for scanned ports through the time.

IV. PORT-SCAN GRAPH MODEL

Different steps are needed for transforming TCP-SYN packets into graph describing TCP ports and their relationships from a scanning perspective. Source and destination IP addresses and ports have been widely used in the intrusion detection domain [START_REF] Hang | Entelecheia: Detecting P2P botnets in their waiting stage[END_REF], [START_REF] Beigi | Towards effective feature selection in machine learning-based botnet detection approaches[END_REF]. They are good indicators of attack traffic flows and targeted services. In our context, we are interested in the scanning process to successive ports. Relying on a graph representation will allow to condense information and thus to model global knowledge from multiple instances, i.e. multiple scans involving different IP addresses.

A. Port numbers transformation and representation

Definition 1 (Port sequence): Let P be the set of all TCP ports. Assuming, a source and destination IP address, respectively IP src and IP dst , we denote S((IP src , IP dst ), T s , T e ), the sequence of targeted ports by IP src to IP dst between the starting time T s and the ending time T e where T s < T e . S is thus a list of TCP destination TCP ports ordered by time: S =< (p 1 , t 1 ), . . . , (p n , t n ), . . . >, where p i ∈ P , T s ≥ t i ≤ T e such that t i+1 > t i . Figure 3 shows an example of a port sequence with 5 targeted ports {25, 26, 27, 443, 8080} at time 50, 60, 70, 87 and 98 respectively. In our particular case of the darknet, no replies are sent back and no service are running. As a result, the sequences of ports thus only represents relations among attackers attempts rather than dependencies among multiple ports used by the applications [START_REF] Peddycord | On the accurate identification of network service dependencies in distributed systems[END_REF]. In practice, the length of port sequences can be very long, and can reach up to a million of TCP port instances over one month. In addition, the ports can be redundant in a sequence in most of the time. To characterize causal relations between two successive ports in a port sequence, we introduce the notion of the port-scan graph model as an intuitive graph representation for successive scans in all port sequences.

A port-scan graph is constructed from multiple successive scans. In other way, it is a directed graph that represents successive relationships between targeted ports in port sequences. Specifically, each vertex represents a port number p i and each edge (p i , p j ) indicates that a scan after to port p j occurs after a scan to the port p i , where p i = p j . p i = p j means that we neglect successive probes to identical part. As introduced in section III, the advantage of such an approach is automatically discard DDoS attacks from the analysis which are the main type of mixed traffic with scans in darknet data. Formerly, port-scan graph over a set of type of ports V is a labelled directed graph G = (V, E, β):

• V is the set of observed port numbers from from P . • E is a set of edges in G. Let p u and p v be be two port numbers in V . There is an edge (p u , p v ) ∈ E if and only if there exists a dependency rule p u lp u ,pv ----→ p v • β is a function that assigns for each edge (p u , p v ) the number of dependency occurrence l pu,pv .

The represented associations between all port numbers represent the signature features of all port sequences, i.e. the most and common behavior of IP source addresses targeting IP destination addresses.

V. PORT-SCAN GRAPH ANALYTICS

Graph algorithms or graph analytic are analytic tools used to assess structural properties of the graph and, as a final objective, to determine strength and direction of relationships between objects represented as vertex in a graph. For example, in a graph representing relationships (such as "interacting" or "probing" another port or IP address) between machines, graph analytic can to extract the following knowledge:

• the centrality of a node (TCP port), i.e. its predominant presence in shortest paths between other nodes (shortest scanning sequences) • the clusters of ports being well connected in the graph, i.e. detecting dense partitions. Such knowledge allows to answer the different questions raised in sectionIII.

To evaluate how relations among targeted ports evolved in time, our analysis is monthly-based. Hence, 24 unique graphs have been created from the two years of collected data. The built graphs contains of up to 6284 vertices and 589117 edges and very low density.

A. Centrality measures

Centrality score measure the communication importance of a vertex in terms of how central it is.

In this paper, we study two popular types of centrality: degree (baseline), and betweenness (flow-based).

1) Degree centrality: The degree centrality is the simplest and most popular centrality measure and gives a highly local view of the graph around each node.

Indeed it is the number of neighbors of a node. In a directed graph, each vertex has an indegree and an outdegree. Let G = (V, E) and ∀v ∈ V . Indegree of vertex v is the number of edges which are coming into the vertex v. The indegree of v is denoted deg -(v). Outdegree of vertex v is the number of edges which are going out from the vertex v. The outdegree of v is denoted deg + (v).

2) Betweenness measure: The betweenness centrality measures are flow-related by considering the information flowing through edges. The most well-known centrality in this group is the Freeman's betweenness centrality [START_REF] Freeman | Centrality in social networks conceptual clarification[END_REF]. It measures how much a given vertex lies in the weighted shortest paths of other vertices. Let δ st = δ ts denote the number of shortest paths from s ∈ V to t ∈ V , where by convention δ ss = 1. Let δ st (v) denote the number of shortest paths from s to t that some v ∈ V lies on. betweenness =

s =v =t δ st (v) δ st (1)
The betweenness centrality allows us to measure the probability that a sequence from the port s to the port t goes through the port v. s and t have the route of the ports sequences. Betweenness centrality is a measure of the influence or importance of a vertex over the flow of scanned ports between every pair scanned ports under the assumption that information primarily flows over the shortest path between them.

3) Results: We list top 10 port-scan patterns in Table III based on both centrality measures. The betweenness provides us the ports commonly used during the scanning, but not necessary most frequent ports, which thus differs our work from the usual study. We discover, for a months, there exists at least not-assigned targeted ports.

We discovered that the degree centrality and betweenness highlight some period of times where probes targets certain kind of services (close in their use):

B. Community analytics measures

Community analytic algorithms aims to find dense subgraphs (also called clusters or communities) in a graph, in which its vertices are more connected within the cluster than with the nodes outside the cluster.

1) Modularity: Modularity measures the density of the partition of a graph into subgraphs called modules. It measures the density of links inside communities as compared to links between communities [START_REF] Newman | Modularity and community structure in networks[END_REF]. The modularity measures essentially compares the number of links inside a given module with the expected value for a randomized graph of the same size and same degree sequence.

Figure 4 shows the densely connected clusters of vertices, with sparser connections between clusters. The ability to detect such clusters of vertices is significantly important to discover the dense group of the common targeted ports at the same time whereas centrality measures are focused on individual ports. Groups within scanned ports graphs might correspond to consecutive probes communities, or service communities.

For computing modularity class, we use the algorithm developed in [START_REF] Blondel | Fast unfolding of communities in large networks[END_REF] for community structure discovery in large graphs.

The modularity of a partition of a graph [START_REF] Newman | Finding and evaluating community structure in networks[END_REF] can be written as

Q = M m=1 [ l m L -( d m 2L ) 2 ] (2)
where the sum is over the M modules of the partition, l m is the number of links inside module m, L is the total number The principle of the algorithm is based on distinct community divisions process which is represented as follows:

1) Separate each vertex solely into m community.

2) Calculate the increase of Q for all possible community pairs. 3) Merge the greatest increase in Q. Example: the port number 22 is assigned port number, which corresponds to the Secure Shell (SSH). In contrast, the port number 26 is an unassigned port number. In addition to extract strongly connected targeted ports, these graphs highlight the homogeneity or heterogeneity of clusters regarding the types of ports within a single cluster assuming one of two criteria, either IANA ranges or distinction between assigned and unassigned ports.

Clusters mainly contain user ports or vendors use for applications (1024-49151) but few of system ports (0-1023) and dynamic and/or private ports (49152-65535). It also exists clusters with only user/vendor ports (as for instance C6, C7, C8 and C10 in 5a). The same phenomena appears regarding the distinction between assigned and unassigned ports. As shown, there is no strong relationship between the homogeneity of clusters based on the two differentiation criteria. This supposes different underlying scanning strategies. In addition, through a manual inspection of clusters, we particularly discovered two types of strategy when targeted unassigned ports: either in a random or an incremental probing of TCP ports.

Furthermore, we analyze the heterogeneous and homogeneous clusters containing assigned ports. As clusters containing not-assigned ports, it also exists consecutive probes of services such as: user and private ports.

We discover other important clusters which contain nonconsecutive probes and not randomly probes. These graph clusters are completely connected and contain particular scanned ports: such as gdp-port: 1997, x25-svc-port: 1998, and tcp-idport: 1999. Those observation clearly indicates, first, that our method is able to find proper correlation (based on the semantics of the ports) and, second, that attackers may leverage intelligent scanning to attack a dedicated type of systems.

VI. CONCLUSION AND FUTURE WORK

In this paper, we have proposed a new approach based on graph analytic techniques for grouping the scanned ports. We provide a method to help the security analysts to understand what clusters of services are commonly being targeted, the importance and semantic of clusters over the scanned ports, and different strategies of the port scanning. Our experimental results, over real data collected in a darknet, highlight the ability of our method to discover unknown specific co-targeted ports to the same or different types of services. Our future plan consists on performing a semantic distance measure over TCP ports and predict the future scanned ports.

Fig. 2 :

 2 Fig. 2: Scanned Port numbers segmented on three ranges: System Ports (0-1023), User Ports (1024-49151), Dynamic and/or Private Ports (49152-65535)

Fig. 3 :

 3 Fig. 3: Port sequence

4 )

 4 Repeat 2 & 3 until the modularity Q reaches the maximal value.

Fig. 4 :

 4 Fig. 4: Clusters of vertices

Fig. 5 :Fig. 6 :

 56 Fig. 5: Top-10 clusters between 08-2016 and 11-2016

TABLE I :

 I Number of packets per protocol

•

  Does it target successive ports of the same segment or different ports of different segments ?

	08-2016	09-2016	10-2016	11-2016
	Port	Number	Port	Number	Port	Number	Port	Number
	telnet	106,891,746	telnet	122,793,736	telnet	183,601,219	telnet	229,054,687
	ms-sql-s	7,788,167	3d-nfsd	9,615,076	3d-nfsd	19,775,823	pcanywherestat	28,614,678
	ssh	3,921,299	microsoft-ds	8,285,083	microsoft-ds	7,975,819	3d-nfsd	26,786,260
	ms-wbt-server	2,241,857	ms-sql-s	7,885,440	ms-sql-s	7,096,882 1,090,220

• Are there a strongly relationship between set of ports? Do they belong to the same targeted ecosystem services such as set of messaging or database services ?

TABLE II :

 II Top-10 of scan probes

	(logscal)	1e+08 1e+09	0-1023 1024-49151 49152-65535
	ports
	Number of scanned	1e+06 1e+07
		100000
			11-2014	12-2014	01-2015	02-2015	03-2015	04-2015	05-2015	06-2015	07-2015	08-2015	09-2015	10-2015	11-2015	12-2015	01-2016	02-2016	03-2016	04-2016	05-2016	06-2016	07-2016	08-2016	09-2016	10-2016	11-2016
													Month-Year

  In 10-2016, the same as the last month, with additional printing protocol. • In 11-2016, the centrality is around the transfer protocols such as http: 80, https: 443, http-alt: 591, telnet: 23 and servers protocols. There exists a large flow to network services, database service, computer network authentication protocols, and transfer protocol, in 08-2016, 09-2016, 10-2016, and 11-2016, respectively.

	• In 08-2016: the centrality concerns different services and
	protocols such as http-alt: 8008 (HTTP Alternate), irdmi:
	8000 (Intel Remote Desktop Management Interface) or
	memcache: 11211 (Memory cache service).
	• In 09-2016: the centrality is around the computer net-
	work authentication protocols and distributed control sys-
	tems such as Kerberos: 88 and d-s-n: 8086 (Distributed
	SCADA Networking), respectively.

•

TABLE III :

 III Top-10 of services discovered with the betweenness and degree measures of port-scan graph of links in the graph, and d m is the total degree of the nodes in module m.

https://suricata-ids.org/

https://suricata-ids.org/

ACKNOWLEDGMENTS

This work was partially funded by HuMa, a project funded by Bpifrance and Region Lorraine under the FUI 19 framework. It is also supported by the High Security Lab hosted at Inria Nancy Grand Est (http://www.lhs.loria.fr).

ms-sql-s 9,221,479 http 2,123,664 ssh 3,148,455 ssh 3,589,657 microsoft-ds 7,766,815 http-alt 1,257,436 ms-wbt-server 2,376,006 ms-wbt-server 2,479,072 ssh 5,290,784 https 1,238,662 http 1,729,630 http 2,019,860 cwmp 4,380,958 mysql 1,199,221 http-alt 1,052,039 https 1,121,846 http 3,262,717 dtserver-port 847,182 https 1,035,851 mysql 1,058,366 http-alt 1,111,872 smtp 464,722 mysql 950,356 http-alt 873,097 https


François Hild

Jean-Marc Domergue

Frederick A Leckie

Anthony G Evans

may come

Tensile and flexural ultimate strength of fiber-reinforced ceramic-matrix composites


François Hild

Alain Burr

Frederick A Leckie

published or not. The documents may come

Matrix cracking and debonding of ceramic-matrix composites


Eero Ivask

Sergei Devadze

Raimund Ubar

Collaborative Distributed Computing in the Field of Digital Electronics Testing

Keywords: collaboration, distributed computing, task partitioning, task scheduling, load balancing, priorities, fault simulation, model security

Computation tasks used in digital design flow for test quality evaluation can require a lot of processor and memory resources. To speed up execution and to overcome memory restrictions, a collaborative computing approach was proposed in this paper. Web-based system architecture allows seamlessly aggregate many remote computers for one application. Efficient collaboration requires credit based priority concept, issues of task partitioning, task allocation, load balancing and model security must be handled. Experimental results show feasibility of proposed solution and gain in performance.

Introduction

Today, sizes of digital electronic circuits are still increasing dramatically according to Moore's law, which states that transistor density on integrated circuits doubles about every two years. This trend is predicted to continue at least for another decade, posing serious problems for test engineers.

To ensure fault-free products, devices must be carefully tested with appropriate test inputs during manufacturing. Widely used process in test engineering is fault simulation: it is used to evaluate the fault coverage of the test sets, it is often required for automatic test pattern generation, fault diagnosis, test compaction, built-in-selftest optimization, etc. For some computation intensive tasks, like test generation, fault simulation step is carried out many times, making simulation speed one of the key issues for overall task performance.

One solution to meet the time-to-market for larger circuits could be the improvement of the fault analysis algorithms. However, plentiful of different methods proposed during last decades leaves little room for improvement. Another way to solve the problem is to divide computation tasks and combine available computer resources. In this paper, we focus on collaborative computing. We assume that participants have computers that are not fully loaded and this load is rather uneven in time. Working is more efficient when every participant has the possibility to run his tasks using collective resources. Effective collaboration requires credit based priorities, good task scheduling and load balancing.

Concept of infrastructure of the current solution was initially inspired from MOSCITO [1], [START_REF] Schneider | Internet-based Collaborative Test Generation with MOSCITO[END_REF] used for example in European VILAB cooperation project. Original system was implemented as client-server Java application. It allowed invoking single work tools remotely and organizing them into predefined automated workflows. Users in different locations cooperated via sharing their software tools in joint workflows. However, task partitioning and parallel execution was not supported. Major limitation for extensive Internet based use was TCP/IP socket based communication that required dedicated communication ports in firewalls on server side and also on the client side for each application and for end users.

More flexible web-based solution for remote tool usage following some key ideas of MOSCITO was proposed in [START_REF] Ivask | WEB-Based Environment: Remote Use of Digital Electronics Test Tools[END_REF]. Socket communication was replaced with HTTP, Java Servlets were used on server side and Applets along with Java applications were used on client side, also database was introduced. In current paper, this concept is revised and improved to support distributed computing and collaboration via effective resource sharing.

There exist also some general-purpose frameworks for distributed computing. BOINC [START_REF]Open-Source Software for Volunteer and Grid Computing[END_REF] for example, built with PHP scripts and AliCE [START_REF] Teo | Distributed Geo-rectification of Satellite Images using Grid Computing[END_REF] that uses Java based Jini technology. Both have drawback of relying on remote procedure calls, which is restriction in firewall-protected networks.

The paper is organized as follows: web-based infrastructure is described in section 2. In section 3 priorities concept is explained. Task partitioning and task allocation are handled in sections 4 and 5. Section 6 discusses model security problems. Section 7 presents workflow with distributed computing. Experimental results are presented in section 8 and conclusions are given in section 9.

Distributed Environment

We have used Web-based client-server concept in our solution. There is Master server, several workstations and arbitrary number of users (Figure 1). Each user is associated with certain workstation, he is owner or he belongs to the team. Simulator application instances run on workstations. Simulator is provided with network communication abilities, it is wrapped with additional software layer -agent is created. For each simulator there is dedicated agent. Agent and simulator reside on the same computer. User accesses only Master. Users and Agents work in "polling" mode, Master is working in "answering" mode. Master and Agents must be started by administrators first. Thereafter, user can initiate a task, which is passed to Master where it is stored until a free Agent is asking for a new task. Each user has its own server-side workspace in the database. When task is complete, Agent passes results to Master where results are assembled and stored again until user will ask for results.

Implementation

Infrastructure of the system is built with Java applet/servlet technology. Communication flow between system components can be seen in Fig. 1. Tomcat is a servlet container that is used in the official reference implementation for the Java Servlet and JavaServer Pages technologies. Tomcat and servlets running on it play important role in order to access intranet resources on workstations and MySQL database. Simulator tool is implemented in C language. Wrapper agent for simulator is written in Java, which has excellent support for network programming. System components can run on different computing platforms. instances must run on their native platform. Master servlet resides usually separately from agents, they must not reside in the same local area network. Each agent can reside on different local area network. Since accessing hard drives for Java applet is restricted for security reasons by default, then GUI applet has been signed digitally, with selfsigned certificate for simplicity.

Use of applet/servlet approach means that general communication is based on HTTP protocol. Firewall traversal is therefore no problem as only one web server port must be configured on Master server. There is no need for opening extra ports in the firewalls on the user side as it is the case with other solutions (which would be major restriction). Communication could be secured via SSL encryption, when necessary. Data passing between user and Master and between workstations is implemented following Transfer Object (TO) design practice -information is passed once as data bundle with serialized Java objects.

Data handling takes place in coordinator servlet on Master server. Problem is that web-based HTTP communication is naturally stateless. Normal HTTP session is valid only for short time, but simulation process may run much longer. We must identify users and store all their data. Users can then come back and receive their results later. Data module has three layers: presentation layer, business logic tier (data base queries, etc.) and physical database. First two layers are implemented in Java. User is accessing database via presentation layer, not directly. User tier consists of several functions to execute business layer queries. Database access is implemented using Data Access Object (DAO) design practice. Data access is using also Tomcat built-in connection pooling to speed up DB transactions.

Workstation Master Server

Coordination

Priorities

Important prerequisite for collaboration is that performance of each participant is overall enhanced; nobody should experience unfair lack of computing power, especially these participants who previously have donated most of their resources. Solution here is to introduce a priority system based on participants credit. Credit for host computer calculated as:

Credit = Credit + Mode • HostPerf • CompTime • LoadFraction . (1)
In formula (1), Mode equals "1" when computer is donating and "-1" when consuming; HostPerf referes to performance index of the computer, it is calculated initially after executing sample calibration task; CompTime is the time host computer has devoted to the current running task; LoadFraction is number between 0 and 1, inclusively. Credit rating of the host increases while other participants are consuming its processor time, at the same time ratings of the quests will decrease the same amount. Credit numbers can go negative, it shows who is contributing and who is not. Participant with higher credit rating will have higher priority and will be served sooner.

Task Partitioning

There are several methods to parallelize task execution in digital test engineering: algorithm can be parallelized, circuit model can be partitioned into separate components, fault set and test pattern data can be divided. In this paper, we rely on model partitioning, but technically it is possible to improve the overall performance by adding at the same time also test pattern parallelism concept. Faults are included already in simulation model, therefore fault list itself is not partitioned. Granularity of the task is important issue to consider. The smaller the network latency, the smaller slices of the task are useful. However, Internet has relatively big network latency, therefore larger model pieces are suitable -communication between partitions should be avoided.

Our simulation method and graph based model partitioning is described in [START_REF] Devadze | Parallel Exact Critical Path Tracing Fault Simulation with Reduced Memory Requirements[END_REF]. Process of fault simulation is divided into number of parallel sub-processes. For each sub-process, a partial calculation model is constructed. In the current implementation, no analysis is conducted to find the optimal selection of partition for minimizing the size of overlapped area. Instead of that, partition is selected randomly taking into account only the required task size and amount of available memory. Algorithm uses internal memory counter to keep track of the currently allocated memory and stops the construction of partial calculation model when maximally allowed amount of allocated memory is reached. During model partitioning some overlapping likely occurs, as we want to avoid interdependences. These repetitive model parts are simulated several times, resulting some speed penalty. In fault coverage point of view results will be accumulated.

Task Allocation

Optimal task allocation ensures that all computers would stop computing at the same time. Several aspects have to be considered. Very slow machines should get only a small fraction of total load, workstation, which has fault, should get no load at all. Distribution of the tasks should happen within short time interval. Scheduler has to be fast not to delay the execution of the tasks. Scheduling objective is to minimize overall execution time, which is actually time of the longest executing subtask. In real-life situations tasks are accumulated and then scheduled in batches [START_REF] Hwang | Scalable Parallel Computing: Technology, Architecture, Programming[END_REF].

There could be possibly two goals while allocating tasks to different computers: 1) maximize the speed of particular task execution for particular user 2) maximize system overall throughput from the perspective of all users. We assume that in case of collaborative networking, the latter is more important; we can use adaptive approach depending on the situation.

Scheduling Algorithm

Initially, idle agents are polling the master server. Each agent announces its tolerated load threshold, it's current load and current performance index. Agents can determine its performance index by host profiling i.e. executing a small sample task and measuring the completion time. Thereafter, Agents prepare it's profiling message for Master server. This message includes also time stamp, so Master can measure the transfer time and determine network latency index for this particular Agent (it is assumed that clocks of both hosts are synchronized). Master collects the information initially for some predetermined time, obtained coefficients are applied later -faster machines will get larger tasks respectively.

Let us assume, that tasks are already submitted to the Master. Scheduler takes the task with highest credit rating and selects an Agent with least load index in the list, given that Agent's load is smaller than allowed threshold. Thereafter, scheduler takes the task with next highest credit rating and searches for next Agent satisfying the threshold condition above. This repetitive process continues until there are no tasks in the list.

The Agent selection idea in this intuitive scheduling strategy corresponds to Bestfit algorithm described in [START_REF] Donald | Fundamental algorithms[END_REF] and used for memory allocation problem. When dimension of scheduling becomes larger, i.e. number of agents and tasks grows, it is possible to switch over to a First fit algorithm also described in [START_REF] Donald | Fundamental algorithms[END_REF] -we look for first Agent in the list who's parameters satisfy condition above. This ensures that scheduling itself is not delayed.

When higher-priority task is submitted to scheduler, and there is no idle Agent available in the system, then scheduler has to suspend or reallocate some of the current tasks if expected completion time of the priority task is greater than migration time of the current task. Candidate task for migration is the one who has the lowest priority and longest execution time. Migration means that Agent notifies Master server and sends intermediate result of simulator tool to Master, thereafter Agent terminates the simulation task. Scheduler finds new target candidate for the task under migration. It will be the Agent with the lowest credit rating and with the best performance rating. Task must be also migrated when load on the host increases above allowable threshold, i.e. if local computing activity increases, as our goal was that participants in collaboration may not suffer. Exception is only the case, when such task belongs to the host owner. Load threshold is adjustable by each participant. Further information on load balancing can be found for example in [START_REF] Eager | Zahorjan: Adaptive load sharing in homogenious distributed systems[END_REF], [START_REF] Shivaratri | Load distributing for locally distributed systems[END_REF].

Circuit Model Security

In this paper, we assume that server host is trusted -circuit models must be uploaded first. Other hosts of participants can be less trusted, because to a certain extent, we can overcome model security problem automatically by partitioning the model itselfpartial model is not very valuable for potential malicious agent. We may even enforce that model pieces are scheduled to the same processors during repetitive executions of the same circuit model, disabling the potential collection activity. However, identifying the right pieces of model must be handled. It is possible to compute the hash value of the model piece and save it to database. When later within limited time frame same hash value is computed for some model piece, then this task is scheduled to the same processing unit. Less limiting strategy is to ensure that not all the model pieces will be scheduled to the same processing unit. Malicious collector still will have no complete puzzle. Let us note that it is much harder to assemble the meaningful full model from pieces than just to recognize the same model piece. Computation of hash values could be performed in parallel with separate execution threads on multiprocessor computer.

Simulation Workflow

First, user specifies parameters and design file location (see Fig. 1). In addition, size of the simulation task can be predefined by user. Thereafter users Applet contacts with coordinating Master server and described parameters along the model are passed automatically. Task coordinator process on Master stores all requests from user(s) in the database. Simulation agents constantly poll the Master and if any subtask is scheduled by coordinator, then agents receive the appropriate parameters and design file and will start actual native simulator tool executable.

Simulator first constructs simulation model taking into account of the size limit of the subtask. While reaching the limit, it saves the breakpoint information into local file system. Simulation agent then reads the breakpoint information and passes it to the Master where it will be stored for other simulation agents. When next idle agent is polling, then it will receive the circuit file along parameters and breakpoint information. Agent starts a copy of simulator tool on the their host. Simulator first constructs simulation model again, but this time it is not starting from beginning, but restores from breakpoint up to the point when task size limit will be reached. New break point information will be again saved to local file system and later passed to Master server database by agent. Simulation agents then wait until their subtasks will be completed and report results back to Master server. Process repeats until there are no simulation sub tasks left. Note that simulators have been started subsequently, but thereafter they run concurrently. Total starting delay is small compared to runtime. Finishing order of simulators depends. It may not necessarily be the same as starting order as simulation speed depends on the piece of the circuit model -some pieces are more complicated to simulate. When all simulators are finished, then Master assembles sub results into final result and stores in database. Results are passed to user when requested.

Experimental results

In experiments we measured communication overhead, memory gain by model size reduction, overall simulation speedup and scalability of the solution when processor number increases. Simulation was carried out on UltraSPARC IV+ 1500Mhz servers. Tomcat servlet engine and MySQL database were running on two cores AMD Athlon 64 6000+ 3 GHz processor with 2 Gb memory. User applet was also running on the similar Athlon machine. Circuit file loading takes less than second for the largest 3,2 Mb size circuit file B17C on the user computer. File transfer to the database and user notification takes about 6 seconds. Thereafter, simulation agent receives circuit file from Master server 3-4 seconds later. Total communication delay was approximately 12-15 seconds (8%) in average compared to local single processor solution with current sample circuits [START_REF]Example circuits[END_REF]. Simulation results are presented in Table 1

Conclusions

We have presented how web-based distributed computing concept can be used collaboratively for digital circuits fault simulation. We introduced credit based priority management, addressed task scheduling and load balancing problem, handled model security issues. In experiments, web communication overhead was sufficiently small, depending on the size of the circuit and the number of test vectors simulated. In practice, circuits and vector sets may be much larger -overhead would be consequently smaller. Further improvement could be the use of compacted models. Model partitioning allowed reducing required memory amount up to 4 times and at the same time simulation was also speed up 3,2 times (pure simulation speedup 3,4 times) compared to single processor simulation. Further speedups can be achieved by combining model partitioning and test set partitioning in the future.

Model partitioning algorithm could be possibly improved to support finer granularity. However, for moderately sized collaborative consortium, granularity might be sufficient, especially when combined with test set partitioning, since in reality there is usually more than single circuit model in use at given time -total number of tasks to be scheduled may be larger and appropriate for number of computers available. Model partitioning is useful in sense that it offers basic circuit model security since entire model needs to be handed over to central server only.

Fig. 1 .

 1 Fig. 1. System components and communication

Table 1 .

 1 . B17C circuit for example contains part of the Intel 386 processor, repeated three times. In simulation experiments 100K test patterns were applied to circuits. Distributed simulation with model partitioning.

	Circuit	B17C	B21C	B22C
	Number of logic gates	31008 18966 27599
	Max model partitions	13	12	13
	Max model build time,s	0,24	0,32	0,37
	Max subtask simul. time, s	214	146	195
	Model size reduction, times	4,1	2,5	2,6
	Pure simulation speedup, times	3,4	2,7	3,1
	Distrib. simul. speedup, times	3,2	2,5	2,9

Acknowledgement

The work has been supported by Estonian SF grants 7068, 7483, EC FP7 IST project DIAMOND, ELIKO Development Centre and European Union through the European Regional Development Fund (Research Centre CEBE).