Marie-Hélène Deredempt
email: marie-helene.deredempt@astrium.eads.net

Vangelis Kollias

Zhili Sun

Ernest Canamares

Philippe Ricco

Spacecraft Data Handling Architecture based on AFDX network

come

Introduction

Satellite Architecture in space domain is basically composed of two major parts: platform and payload.

Avionics platform is composed of functions chains components including launcher interface, solar array deployment, thermal control, power control, orbit management and vehicle attitude control. It manages also the interface with ground systems for commanding and observability. These functions are mainly realized by a redounded On Board Computer (OBC) connected to platform units such as sensors, actuators or remote interface units.

Each of these functions is done by acquisition of sensors (star tracker as optical device that measures position of stars using photocells camera, sun sensors as solar cells or telescope device that sense the direction of the sun, magnetometer as device that senses magnetic field strength). They realize processing and provide commands to actuators (reaction wheels as electric motor, driven rotors made to spin in the direction opposite to that required to reorient the vehicle) Due to the lack of processing resources on most of those equipment and devices, exchanges with the central computer are realized on a master slave scheme with synchronization by buses events. This architecture based on MIL-STD-1553 [START_REF]ECSS-E-ST-50-13C: Interface and communication protocol for MIL-STD-1553B data bus onboard spacecraft[END_REF] and predefined buses activities are extremely robust and easily validated.

Payload avionics for earth and observation mission is often composed of specific equipment connected to OBC by SpaceWire [START_REF]14C : SpaceWire protocol identification[END_REF] links. Figure 1 shows the EAS SAVOIR architecture. Some others links coexist in such architecture for some specific needs (RS485, RS422 and proprietary links). This leads to a complex design, sometimes driven by contractors with too many and heterogeneous systems. This has consequence on power consumption, weight (wires and systems), maintenance, configuration, provider dependency, integration and validation.

Space context

Such avionic architectures are mainly due to a specific business model and geographical return. Business in terms of number of pieces of equipment does not help to promote standardization, harmonization and inter-operability. Standardization for communication above SpaceWire and MIL-ST-1553 are proposed by ESA only for the first layers of the OSI model. But this has begun to change and the major reasons are detailed hereafter:

• The cost to integrate equipment in terms of hardware and software for protocol, data management and database will be no more acceptable.

• The number of payload equipment that have to be connected to the central computer will increase significantly for the future science and observation mission. Architectures are based on routed SpaceWire network and lack of determinism will become a major issue when the bandwidth and the number of equipment increase. Determinism on interface traffic could be implemented using MIL-STD-1553 buses driven by a bus controller as orchestrator and predefined scheduling scenarios. On routed SpaceWire, determinism is not possible without the required a software synchronization solution

• The central computer will need to realize more functions for autonomy and operation, will also need more industrial parameters. These will have an impact on the bandwidth.

• The cost of processing resource decreases for connected device. They will realize more closed loop automation. They have to be uploaded and they have to be configured and controlled.

In such a context, a critical architecture design based on deterministic network will improve reliability, availability, quality of service (QoS) and meet standardization requirements to reduce costs.

In aeronautical domain, Avionics Data Communication Network (ADCN) is built using determinism redundant network based on Ethernet and standardized by ARINC664 part 7 specifications (AFDX). Systems connected to the network are composed of processing and I/O boards and a specific End System (ES) in charge of the network management. The network is composed of intelligent switches that make traffic policy, control and buffering. A specific database is available to provide configuration of End Systems, switches, applications, integration test beds and flight test installation equipment. Figure 2 At data link level (MAC), notion of Virtual Link (VL) and Bandwidth Allocation Gap (BAG) are the basic concepts for determinism. This is completed by standard IP and UDP layers as network and transport layer. Sampling port and queuing ports are used to make the link with applicative partitions to receive data from the network or transmit data to the network. This completes ARINC653 (Time and Space Partitioning) inter partition communication by an inter system communication using the same mechanism for interoperability.

Objectives of MISSION project

Experimentation on time and space partitioning was already done for space domain through ESA and CNES initiatives (IMA for Space [START_REF]DASIA May 2012 Dubrovnik: Integrated Modular Avionics for spacecraft : software architecture and requirements[END_REF] [3], LVCUGEN [START_REF] Galizzi | TSP-based solution) and first porting feedback[END_REF]) and have focused mainly on input/output (I/O) management, CPU overhead introduced by the virtualization and impact on the process, methods and tools for designing above the technology. Those studies have brought all the necessary material to go further and make a complete Integrated Modular Avionics (IMA) for space domain based on deterministic network.

The first objective of this study is to provide a tailorisation of the AFDX standard referring to space constraint and to evaluate the needs for applicability in space domain.

The second objective is to define the network topology based on an existing operational mission and a possible future mission. This has to be done on data profile exchanged between systems in such reference architecture.

The third objective is to design, develop and test use cases. Implementing Ethernet on space avionic will be a major step and it seems that realizing first AFDX above SpaceWire could help to promote the solution for a future implementation. This is the reason why we have planned to experiment on AFDX above SpaceWire and above Ethernet.

The fourth objective concerns the configuration. Airbus experience in AFDX shows that the configuration of such a network is the base to secure operational condition and improve development phases. The network and its configuration are considered as a system on its own. The last objective is to provide a modelisation of the switch for Ethernet and SpaceWire.

We have first made a tailorisation of ARINC664 standard and defined capabilities requirements for the applicability of the standard in space domain above physical layer of Ethernet and SpaceWire.

Based on Astrium experience on spacecraft architecture, the project defines a representative network architecture from different earth and observation missions and provides a network topology with associated network traffic. ARINC664 demonstrators over Ethernet and over SpaceWire are defined.

For succeeding to promote this technology in space domain, we have to consider some constraints and keep in mind the following aspect:

• Due to business model, implementing AFDX protocol in all the systems of the avionic will not be possible from scratch. Steps are required and development of a gateway (IO module) to interface new world and legacy ones could be needed. The topology proposed is based on that concept.

• Implementing AFDX in connected equipment will probably not be possible if it requires specific material and if it is hardware software codesigned, even if FPGA technologies and System on Chip (SoC) for space domain allow now to envisage having a processor. A modular design of the IP could allow selecting only the minimum required modules for limiting footprint on FPGA or ASIC resources needed for radiation space domain constraints. This will allow also to implement all functionalities for others domains (aeronautical and automotive) or space when technologies will be available.

• Implementing an AFDX switch will not be possible if it requires specific hardware. It could be integrated in an existing system such as central computer or mass memory (in System On chip with SpaceWire or Ethernet interfaces)

Tailorisation of the standard

Requirements for addressing and quintuplet definition (MAC, IP source and destination and UDP source and destination addresses) are kept and the AFDX frame is considered for Ethernet and for SpaceWire. Figure 3 shows AFDX frame. Requirements on network redundancy, integrity checking, jitter and latency are taken as defined in the standard. Notion of sub virtual link, fragmentation and reassembly, jumbo frame are considered as options. Fragmentation is an interesting feature to allow maximum TM to be transferred without specific software development. Sub virtual link could be needed if the maximum number of VL is not sufficient. This does not seem to be the case in the space domain.

Concerning redundancy of equipment, we are facing the following specific conditions in space domain:

• In cold redundancy, one function is operational and the redundant part is not powered. The same bus profile must be defined at configuration level with different addressing for the same software data structure.

• In warm redundancy, there is one active part of the function. The redundant one is powered for reduced functionality or to be already powered in case of failure of the nominal one.

• In hot redundancy, both operational and redundant functions are powered on and data profile is redounded. This has an impact on configuration (all parameters have to be defined) and bandwidth but could be managed easily at application level with communication ports and specific data structure.

Network topology for the demonstrator

The demonstrator for this study will be based on representative operational spacecraft avionics architecture. The current topology based on MIL-STD-1553 and routed SpaceWire is presented on figure 4 and a network centric equivalent architecture is proposed on figure 5. The AFDX network is composed of 18 end systems divided into four groups as follow:

• 6 instruments which are all remote-sensing, and considered as End System in this use case as relevant equipment like imagers used for solar atmosphere measurements or to acquire information on magnetic field. The choice of these instruments to be End System is headed by the high bit rate needed and their remotesensing capabilities.

• We have implemented 3 switches to limit the number of connection on each.

AFDX above Ethernet

Designing protocol in a modular way with configuration on threshold (number of ports and VL) will allow selecting limited functionality for a first future implementation on radiation hardness technologies. Such modules (switch and ES) could be integrated in a System on Chip (SoC) for On Board Computer (OBC) and mass memory.

The demonstrator will be based on SoC hardware components which include both processing units and programmable logic (FPGA). Based on previously developed products, a conservative approach is to start with a basic physical layer management using modified IP for MAC inside the In order to use the same IP protocol modules for AFDX above SpaceWire, the AFDX frame has to be considered. To be compatible with ECSS-E-ST-50-12C SpW Standard, we have decided to consider LogicalAaddress (LA) and Protocol Identifiers (PID) as part of Ethernet destination address and change the fixed fields to 0xFF for LA and 0xF0 to 0xFE for PID. This will guarantee the same structure for designing protocol, capability for the switch to identify an AFDX or a legacy device. Figure 7 shows the AFDX over Ethernet and AFDX over SpaceWire frame formats.

Standard SpaceWire CODECs can be used but the flow control mechanism will be neutralized (i.e. SpW CODEC will not be modified but the flow control will be bypassed at higher level either at driver level or at IP core interface level) in order to have deterministic and non-blocking behavior at the network. This can be achieved by allocating enough receive buffer memory at receiving ports and dropping incoming packets in case of buffer overflow (a receive buffer can be used which will automatically receive data no matter if a receive pointer/descriptor is available or not. In case that no receive pointer/descriptor is available the data are lost but even in this case the data are read out of the SpW CODEC without blocking the link). SpaceWire router will not be used anymore and switches will operate in store and forward mode and wormhole routing will not be used.

The implementation must accept every link speed between 10 and 200 Mbps. The links automatically start at 10 Mbps until they are connected and then automatically reach the nominal speed as defined by the configuration. Figure 8 illustrates the AFDX over SpaceWire demonstrator

Figure 1 :

 1 Figure 1 : ESA SAVOIR reference architecture

  illustrated A380 ADCN..

Figure 2 :

 2 Figure 2 : A380 Avionics Data Communication Network (ADCN)

Figure 3 :

 3 Figure 3 : AFDX frame

Figure 4 :

 4 Figure 4: spacecraft avionics architecture

Figure 5 :

 5 Figure 5: Network centric approach for Spacecraft

FPGA.

  The main differences between Ethernet MAC layer and AFDX are: • multiple MAC addresses for addressing the same device • Control precisely the time to start for each frame transmission. • Redundancy management with 2 physical connections. Fig 6 shows AFDX over Ethernet.

Figure 6 :

 6 Figure 6: AFDX over Ethernet Demonstrator (Zyng board)

Figure 7 :

 7 Figure 7: AFDX over Ethernet and AFDX over SpaceWire packet formats

Figure 8 :

 8 Figure 8: AFDX over SpaceWire Demonstrator

Configuration and feasibility analysis phase

Configuration tables for AFDX End Systems, Switches and Applications are needed. A powerful configuration tool is being designed in order to help during specification phase of network topology for feasibility analysis as shown in Figure 9 and 10. This tool manages only few parameters such as message ID, source application, destination application and non-functional dynamic properties to create messages, virtual links and communication port and a complete dynamic of the network traffic. Once validated, this configuration can be considered as input of the tool for modification without changing ports for application already developed and validated with this configuration. Using spare during the design phase will help to avoid requalification of the network The Network Topology Management System (NTMS) provides functionalities regarding to the network topology management. These functionalities include the management of the end systems, switches in the network and the interconnection between them. The most important functionality of the NTMS is the ability of importing the network topology configuration from an XML file to the internal data structures.

The Message Management System (MMS) is in charge of managing the message structures (including the concept of data element, data sets, functional data sets, etc.) and provides importing functionality which allows the user to import already defined data exchange information from an XML file to the internal data structures. MMS also assign ports to the messages.

The Virtual Link Management System (VLMS) is in charge of creating the virtual links, adjusting its parameters and finding a path through switches. VLMS also manages the distribution of messages in virtual links.

The data profile of the representative avionic architecture shows that nearly 40 VLs have to be created from each OBC to the devices. Considering work already done on state of art, radiation characterization, design constraints in terms of footprint and resources needed, and requirements for configuration, validation, observability, schedulability and determinism, we can confirm the applicability of the standard in space domain if modularity is provided for switches and end system design. Data profile analysis shows that the solution is applicable for the defined topology even for managing space FDIR and systems redundancy.

The fact that AFDX is possible above SpaceWire using described frame profile could help to promote the solution in space domain with a space network.

Demonstrators will confirm the feasibility and complete the results to give space agencies and prime contractors, the guideline for applying such a technology in space domain and a sufficient background to encourage the development of an operational prototype based on candidate equipment for switches and end systems


Silvia Mazzini
email: silvia.mazzini@intecs.it

John Favaro

Alessandra Martelli

Laura Baracchi

Intecs S P A Via

Umberto Forti

Security and Safety Modelling in Embedded Systems

Keywords: security, safety, embedded, certification, model Congress Topics: Dependability: Safety, Security, Certification. Process, methods and tools: model-based system engineering. Domain: embedded systems

The SESAMO project directly addresses the root causes of problems arising from the convergence of safety and security in embedded systems at architectural level, where subtle and poorly understood interactions between functional safety and security mechanisms impede system definition, development, certification, and accreditation procedures and standards. Intense market innovation is being held back by this root cause: the absence of a rigorous theoretical and practical understanding of safety and security feature interaction.

Introduction

SESAMO (Security and Safety Modelling in embedded systems) is an ARTEMIS JU Project begun in May 2012 and lasting three years, coordinated by Intecs S.p.A. SESAMO is developing a component-oriented design methodology based upon model-driven technology, which jointly addresses safety and security aspects and their interrelation for networked embedded systems in multiple domains (e.g., avionics, transportation, industrial control). Safety and security have long been identified as two very important attributes of overall system dependability [AVIZ 2001]. The convergence of safety and security is happening everywhere in mission-critical embedded systems domains and today's methodologies are inadequately prepared for it. There are many subtle interactions and interdependencies among safety, Quality of Service (QoS) and security. Indeed absolute security is often less important than the quantifiable trade-offs between performance and securityas resource constraints may make it infeasible to guarantee absolute security in all circumstances. In addition, the introduction of security related considerations into embedded systems tends to invert the priorities of certain non-functional attributes. In traditional network Cybersecurity, confidentiality and integrity trump (that is, they are more important than) availability. But in embedded systems, it is invariably just the opposite: availability and integrity trump confidentiality. Furthermore, additional non-functional attributes of security come into play in the embedded area, including autonomy, timeliness, and isolation. Finally, in general, safety trumps security in the embedded area.

Making a system more secure can make it safer, e.g. by improving code quality to achieve a higher level of assurance. But making a system more secure can also make it less safe: for example, the authentication and authorisation might meet critical real time requirements in "most" cases, but not always, making it impossible to certify and use the system in a safety critical environment. Conversely, QoS constraints placed on systems by safety functions could preclude implementation of adequate security mechanisms, making the system unqualifiable for security critical use. In each case, not only are systems rendered inadequate in one or the other environment, but both the suppliers and users of the systems lose opportunities in a growing area of the mission-critical systems market. Safety analysis techniques, both qualitative and quantitative, have been developed over the years and are now deployed. On the other hand, the development of quantitative security analysis techniques has been only recently investigated with significant effort. We need to progress on both quantitative aspects of security analysis and the integrated quantitative and qualitative analysis and design of safety and security. These involve at least specification methods able to cope with both aspects from requirements elicitation to system design and analysis. We need thus an enriched language able to deal both with security aspects, such as cryptography modelling, and probabilistic/stochastic ones, including those for describing performance. Similarly we need an integrated validation framework that permits the use of multiple evaluation techniques in an organized manner. One can then consider both orthogonal analysis approaches, where one aspect is considered after the other, as well as fully composite ones, where all the aspects are considered (optimized when necessary) at once.

On a system level, the safety and security properties and concepts differ and means potentially collide when designing dependable systems for safety and security. For a start, assumptions between classical safety and security environments have traditionally been different as exemplarily depicted in the following table.

"Classical Safety-Oriented Dependability" versus "Classical Security"

Safety Security

Assumes trustworthy operators Assumes fault free system

Assumes closed system Assumes open connected system

Assumes time response from dedicated resources Assumes shared generic platform Furthermore, the technical means to achieve dependable operation in safety and security have been different, although some concepts have deceptively similar names. Consider the example of ensuring avoidance of altering or disclosure of information. In the safety world, the goal is to use error detection codes that maximize the so-called "Hamming distance" (the minimum distance between encoded words) to avoid unrecognized corruption of data. In the security world, encryption uses the same mathematical basis, but the goal is to minimize the Hamming distance and equally distribute code words over the available code space (in order to not disclose and protect change of any information). So a very good error detection code cannot be very secure at the same time. As we have seen in the past, in the development of real world systems it is not possible to simply test systems in order to find such subtle issues of safety and security feature interaction. It is necessary to develop systems from the beginning to meet such joint requirements. Model based approaches offer a promising avenue to handling the emergent aspects of safety and security in a technologically and methodologically sound fashion. Through the appropriate application of constraints on the way in which systems are constructed and executed (the so-called computational model of a system), it becomes possible to construct systems with the properties of composability (that is, they can be constructed from reusable parts that retain their individual properties when combined into larger systems) and compositionality (that is, the overall emergent properties of the system constructed under these appropriate constraints can be deduced from the individual properties of its parts). Such approaches offer the promise of a major advance not only in the way that systems are constructed but also in the way that they are accredited and certified (at the part level rather than only at the system level). Thus, a main objective of SESAMO is to provide a model-driven process for the compositional development of safety and security critical systems, by allowing modelling and reasoning about systems at different abstraction levels.

Building blocks

One important activity in SESAMO is the identification of a sufficiently large and useful set of mechanisms for safety and security that can form building blocks to be used in the construction of systems in the domain of interest to resolve (or balance) conflicts inherent in safe and secure system development. This is the constructive aspect of SESAMO. The term "building block" has been used within the project in a broad sense, in order not to prejudice the identification of mechanisms of all types. Bringing their background and experience to bear upon the activity, consortium members have identified and contributed building blocks of different levels of granularity. Some are large, architectural level building blocks such as: Time and space partitioning; Virtualisation; Redundancy / Diversity of architectural components.

Others are smaller, more specific building blocks, including:

Signature generation and verification; Protocols and monitoring; Encryption / decryption; Plausibility checks.

Still others are process building blocks, such as joint safety and security FMEA. They are being classified within the project with respect to their safety aspects, security aspects, and most importantly, their characteristics that lead to either synergies or conflicts between safety and security.

Analysis methods

A further important activity is the identification and description of analysis and assessment techniques suitable for supporting the analysis of systems constructed using the building blocks and integrated methodological approach and tool chain of SESAMO, as well as their assessment according to current requirements or those to be recommended by the consortium as a result of the project work. It is viewed as a continuation of the constructive aspect of SESAMO, whereby building blocks are accompanied by contextual analysis of both qualitative and quantitative types. Without analysis, the enabling mechanismsthe building blocksof SESAMO are useless for certification or accreditation.

An example of a technical analysis technique under study is a quality calculus, from the Technical University of Denmark [NIEL 2012], for which a type system has been developed for checking the extent to which safety and security goals have been met in the system under analysis. To deal with conflicting safety and security goals, the calculus has been extended with a primitive for "endorsing" data to a higher trust level (accepting violations of the explicit flow) to explore the effects of relaxing security in controlled situations, thereby lowering costs. The calculus has been applied to a realistic automotive engineering scenario in which the CD-player is used not only for playing music (a non-critical context) but also for updating the functionality of the car (a critical context).

An example of an analysis technique under study under the lead of SESAMO partner Adelard with an eye toward assessment and certification is security-informed safety cases (see [ASCE 2013] and [PAUL 2012]). The use of safety cases in the context of safety-related assessment is a well proven approach in several domains now, from nuclear to railway to automotive. In the context of safety and security related systems, we believe that an effective approach is to enhance the existing safety case methodology and use it to demonstrate and communicate security requirements in addition to safety. In this way we can benefit from a mature, effective and time-tested safety case approach and have both safety and security properties considered in an integrated manner within a security-informed safety case. This approach should also give us a better understanding of the interactions between system safety and security aspects and help us to address the potential issues associated with their interrelation.

Decision support

It is a premise of SESAMO that safety and security cannot always be reconciled in a perfect "win-win" manner. The elaboration of decision support strategies is a natural consequence of this premise. To date there has not been a significant amount of structured work in analyzing the trade-offs between safety and security, and some of the work in the project is aimed at establishing a firm basis for decision support development.

Reaching over to results in the economic domain, one activity in this area currently being pursued in the project is the exploration of Pareto frontiers, where the balance between security and safety can be seen as a multiple criteria optimization problem that can be approached by Pareto optimality (that is, the design space is characterized as involving multiple criteria while achieving overall optimal use of resources such that no further improvements can be made without penalizing one of the parameters). A quantitative approach is being investigated that introduces probabilities and metrics, as well as quantitative verification using stochastic model checking. [NIEL 2013] In our opinion, equally important as the quantitative aspect of decision support is the qualitative aspect represented by the different safety and security cultures, which have potentially clashing value systems. For example, it is commonly stated in the safety communitywell represented in the SESAMO consortiumthat during a crisis, safety takes precedence over security. Within the consortium, there is also a strong security community, which notes that authenticity must also be given its due within a combined safety/security context; it would be imprudent, they argue, not to check that orders to change course do in fact originate from authorised sources rather than intruders. The existence of qualified and robust safety and security cultures within the SESAMO consortium is providing an important opportunity to explore this interaction among cultural value systems that is also a strong factor within any framework for decision support. It is also a strong factor in any examination of a joint methodological approach to safety and security related development.

A Combined Methodology?

The safety community in particular has established standards in a number of domains governing safety-related embedded systems development, including avionics (e.g. Given the importance of both safety and security in mission-critical systems, it is natural to consider the inclusion of both within a single standard. But to date there has been little cross-fertilization between the safety and security communities developing these standards [PIET 2012]. Even the vocabulary used in the two communities is often confusing and contradictory [START_REF] Piet | [END_REF]]. Thus, incorporating security requirements into safety related standards (and vice versa) has been extremely challenging to date.

Nevertheless, an increasing amount of activity is becoming evident, both in the technical community (e.g. [START_REF] Hans | [END_REF]], [REIC 2012]) and in the standardization committees, reflecting the increasing need within the embedded systems industry. An example in standardization is the latest version of IEC 61508 [IEC 2010], the domain-independent safety standard for electrical and electronic systems. In its latest edition, it introduces for the first time requirements related to security. Clause 1.2 k) of Part 1 of the standard " … requires malevolent and unauthorised actions to be considered during hazard and risk analysis …"

This opens the door to the integration of security requirements also into the domain specific safety standards for which it serves as the "mother" standard. This has process already begun: for example, Draft EN 50126-5:2012 states that "… the Safety Case shall demonstrate that […] misuse-based failures on external interfaces do not adversely impact on the safety integrity of the system."

However, this "opening" is in at least one respect less a door than a Pandora's Box. Once the step is taken of including both types of requirements in a standard, a much more difficult problem comes into the foreground: the process. Mature development processes for safety related embedded systems have existed for years, and lifecycle activities for security related development (e.g. [HOW 2006]) have also been defined. But the integration of safety and security related development processes has been a controversial topic.

A case in point is the automotive safety standard ISO26262 [ISO 2011], another of the domain-specific "children" of the generic IEC61508 standard. In 2013, the ISO26262 standardization committee (ISO TC22/SC3/WG16) began to debate the inclusion of security related considerations in the standard. During the debate, a strong position was presented that security and safety are very different areas, requiring different skills, and that the respective processes must remain separate (see also [START_REF] Czer | [END_REF]). This position argued that, rather than including specific security requirements in the standard, interaction points between the two (separate) processes be identified, and requirements defined to establish appropriate communication channels. Another position presented preliminary ideas for what could happen at the interaction points, such as combining FMEA with security analysis (e.g. by adding failure modes to the FMEA representing security attacks).

This concept of establishing points of contact between parallel safety and security lifecycle activities is also more or less in line with the approach being taken within the EUROCAE/RTCA community, where ongoing work is examining the addition of security related activities to the heavily safety-oriented ARP4754 system development process [ARP 2010].

The SESAMO approach is proceeding along these lines. Figure 1 illustrates how SESAMO positions itself within this scheme.

Figure 1: Safety and security lifecycle activities

Two types of SESAMO intervention can be identified:

1. Trade-off analyses at the interaction points. These are particularly relevant for those lifecycle activities and mechanisms that tend to be purely safety or security related. For example, cryptography and authentication is squarely in the security domain. The analysis methods developed in SESAMO can make it possible during communication at the interaction points to judge the effects of the activities in each parallel process on the other, and to provide appropriate decision support based on the results. Note that this tends to concern the building blocks that are architectural in nature. 2. Joint lifecycle activities. This is a deeper intervention, where the lifecycle activities are actually combined, such as joint hazard and threat analysis, or joint FMEA and attack analysis. Note that this tends to concern the process building blocks rather than the architectural building blocksbut it is not always the case. For example, a "redundancy" building block (and its accompanying analysis methods) could be instrumental in a joint safe and secure architectural design activity.

An important point to observe is that the SESAMO approach assumes that, although the processes are parallel, they are working on a single set of workproducts: the two types of intervention described above are intended to support the development of this single set of workproducts with the appropriate set of safety and security attributes.

The approach of parallel processes with "weak" trade-off interactions and "strong" interactions for joint activities has the advantage of providing a smooth migration path, starting from the separate processes of today and gradually identifying and implementing architectural and process building blocks that promote an ever-closer integration of the processes, while continuously approaching the Holy Grail of a fully integrated process.

Tool Support

A model-based approach to tool support for the integrated SESAMO safety and security methodology is being taken.

The elaboration of a cross-domain tool chain has been organized as a bottom-up process, involving first an examination of the tools made available by the consortium partners for their relevant characteristics. These relevant characteristics might be either domain-specific (e.g. automotive) or technique-specific (e.g. assurance management). Secondly, the possibilities for integration of the individual tools are being explored, often in a pair-wise manner, in order to arrive at an understanding of the overall possibilities for integration. As a specific example of the model-based approach being pursued in the project, the pair-wise integration of two toolsets is presented in the following.

The medini analyze toolset [IKV 2013] has been developed for the automotive domain, and supports a high level of process integration for the ISO26262 safety standard. Possibilities have been identified for the integration of securityrelated activities including threat analysis, attack trees, security FMEA, and security informed safety cases. Figure 2 illustrates the integration of facilities for integrating the consideration of security violations as malfunctions, which allows them to be treated in a harmonious fashion alongside safety related malfunctions in further lifecycle activities. For example, also illustrated is the incorporation of the security violations into the FMEA, to create the type of joint lifecycle activity described in the previous section on the methodology.

Another toolset available for integration into the SESAMO toolchain is CHESS [CHES 2013], an Eclipse-based framework that supports a domain-independent process for general model-based, dependability-related development. Its UML metamodel based profile is extensible to relevant SESAMO concepts. CHESS has particularly well developed facilities for schedulability and dependability analysis, which can support the analysis and trade-off techniques being elaborated in SESAMO. Thus, a pair-wise integration with medini analyze has been undertaken to cover the development lifecycle using the most appropriate tool for each phase. Medini analyze is used for the system design, safety and security management, risk/hazard analysis, ASIL association, etc. CHESS is used for software design, schedulability/dependability analysis, trade-off analysis, and code generation. Figure 3 illustrates the process, as explained in the following.

Conclusions

Key elements of the SESAMO approach are: a methodology to reduce interdependencies between safety and security mechanisms and to jointly ensure their properties; constructive elements for the implementation of safe and secure systems; procedures for integrated analysis of safety and security; an overall design methodology and tool chain that utilize the constructive elements and integrated analysis procedures to ensure that safety and security are intrinsic characteristics of the system.

The relevance of the SESAMO results is guaranteed by the involvement of large partners with significant economic interests in safety and security critical systems in several domains, a sound group of technology providers (including SMEs) and prestigious research entities (academia and institutes) with deep and complementary multi-domain expertise.

The characterisation and assessment of the applicability of SESAMO results from a multiple domain industrial perspective is based on elaboration of the following use cases: integrated modular avionics, motor control in automotive and industrial environments, car infotainment system, safe & secure communication in railway and medical applications, smart-grid security, security of SCADA systems in oil and gas refineries.

SESAMO has also adopted an instrument of consultancy to ensure the maximum impact possible within the communities of potential end users: the SESAMO Expert Advisory Board (EAB), consisting of important industrial organisations external to the project with significant activity and needs in the key areas addressed by SESAMO.

Mid-term results have been presented to the EAB in a public event, the 1 st SESAMO Industrial Day in Vienna on November 20 th 2013, where the challenges of the project objectives and results have been recognised, positive feedback as well as useful hints and challenging guidelines for future work have been collected. Final results are expected by the end of 2014.

  [DO-178C]), railway (e.g. [EN 2010]), and the automotive industry (e.g. [ISO 2011]). A number of security related standards have also been developed, such as ISO 27002 [ISO 2013] establishing guidelines for information security management in an organization, IEC 62443 for industrial communication networks [IEC 2013], and the ISO/IEC 15408 Common Criteria [IEC 2009] for security assurance.

Figure 2 :

 2 Figure 2: Integration of security related activities in medini analyze (Source: ikv++)

Figure 3 :

 3 Figure 3: medini analyze / CHESS process 1. The safety related activities of the ISO26262 development process are followed, enhanced by security related activities as discussed in the previous section on methodology (whereby some joint activities such as security and safety FMEA are carried out). Note also the reference to joint safety and security activities for the elaboration of a security informed safety case using the Goal Structured Notation (GSN) facilities provided by the ASCE tool of consortium partner Adelard [ASCE 2013]. 2. Medini analyze SysML based models representing the system architecture and requirements are imported into CHESS in the System View and Requirements View respectively, using Model-to-Model (M2M) transformations written in the Operational QVT language [QVT 2013]. An import facility was created in CHESS for this purpose (Figure 4) and the CHESS profile was extended with SysML stereotypes to represent Medini elements (such as controllers, actuators and sensors) and to include specific safety/security related properties (such as ASIL information). 3. The software modelling process proceeds in CHESS, where the software is designed and elements in the software model are linked to system level architectural elements and requirements, thus providing support for system and software co-engineering. In preparation for trade-off analysis (e.g. the effect of cryptographic mechanisms on system schedulability), real-time properties are specified and schedulability analysis is launched in CHESS. Schedulability analysis is executed by the MAST tool (Modelling and Analysis Suite for Real-Time Applications) developed by CHESS partner University of Cantabria [MAST 2013]. Based upon the results of the analysis, the best solution may be chosen for code generation. Alternatively, the user may return to the medini analyze environment for further development as described in the next step. 4. The software models and analysis results are imported back into medini analyze, through another Model-To-Model transformation. As noted above, the results of the analysis may lead to necessary changes in the architecture or in the mechanisms used in the architecture (e.g. weakening of the cryptographic mechanisms in order to permit the safety-related scheduling deadlines to be respected).

Figure 4 :

 4 Figure 4: Import of medini analyze models into CHESS

Concepts: "Classical" Safety-Oriented Dependability versus "Classical Security" Safety addresses Security

  The following table lists some similarities with slightly different approaches to address either safety or security.

	Partitioning (independent redundant		Partitioning (firewalls / router filters;
	channel)					VPNs)
	Design audits (code reviews / testing)			Design audits (open source code)
	Redundancy	(multiple	nodes	and
	channels)				Avoidance
	Selection of redundant channels (voters /		Selection of actors (authentication)
	selectors)
	Error detection codes				Encryption
	Parameter monitoring / limit checks			Traffic monitoring
					Sensing	Signature checks
						Anomaly checks
	System diagnostics / mutual test		Correlation	Intrusion state estimation
	Fail-silent shutdown of redundant node or	Isolation	Rerouting (router filtering, IP shunning);
	channel					host shutdown
	Fault masking				Recovery	Rerouting	(routing	filtering,	IP
						shunning); host shutdown
	Reboot				Repair	Backup site
	Hardware repair				Scrub and re-install

Acknowledgements

The work presented is based on the results of the ARTEMIS SESAMO Project, Grant Agreement 295354. The authors would like to thank all the members of the consortium for their contributions to the results described therein.


Jingyi Bin

Sylvain Girbal

Daniel Gracia Pérez

Arnaud Grasset

Alain Mérigot

Studying co-running avionic real-time applications on multi-core COTS architectures

come   L'archive ouverte pluridisciplinaire

Studying co-running avionic real-time applications on multi-core COTS architectures Jingyi Bin * † , Sylvain Girbal † , Daniel Gracia Pérez † , Arnaud Grasset † and Alain Merigot * * Fundamental Electronic Institute, France † Thales Research & Technology, France

Abstract-For the last decades, industries from the safetycritical domain have been using Commercial Off-The-Shelf (COTS) architectures despite their inherent runtime variability. To guarantee hard real-time constraints in such systems, designers massively relied on resource over-provisioning and disabling the features responsible for runtime variability.

The recent shift to multi-core architectures in the embedded COTS market worsened the runtime variability problem as contention on shared hardware resources brought new variability sources. Additionally, hiding this variability in additional safety margins as performed in the past will offset most if not all the multi-core performance gains.

To enable the use of multi-cores in this domain, it has become essential to finely characterize at system level the application workload, as well as the possible contention on shared hardware resources.

In this paper, we introduce measurement techniques based on a set of dedicated stressing benchmarks and architecture hardware monitors to characterize (1) the architecture, by identifying the shared hardware resources and their associated contention mechanisms. (2) the application, by identifying which shared hardware resources it is sensitive to. Such information would guide us toward identifying which applications can run smoothly together without endangering individual worst-case execution times.

I. INTRODUCTION

Industries from the safety-critical domain such as the avionic, automotive, space, healthcare or robotic industry, have been using Commercial Off-The-Shelf (COTS) architectures rather than in-house solutions to reduce both the non-recurring engineering costs (NRE) and the time-to-market (TTM) [START_REF] Baker | Lessons learned integrating COTS into systems[END_REF], while fitting their exponential needs in performance and functionalities [START_REF] Bailey | Study report on anionics systems for the time frame 2007[END_REF], [START_REF] Ebert | Embedded software: Facts, figures and future[END_REF], [START_REF] Dvorak | NASA study on flight software complexity[END_REF], as illustrated by Figure 1.

In the avionic industry, the real-time software embedded in aircrafts is characterized by a few hundreds of small and independent applications running together with a few larger and very communicating applications. Embedded COTS multicore platforms are an expected trend for next generation aircrafts, providing a low-cost opportunity to run concurrently these small independent applications. However, as COTS providers are mainly targeting the consumer electronic market, mostly driven by best-effort performances, the safety-critical industry has to face more and more runtime variability issues [START_REF] Predator | Design for predictability and efficiency[END_REF], [START_REF] Ungerer | Merasa: Multicore execution of hard real-time applications supporting analyzability[END_REF].

Concurrently, the safety-critical software is characterized by stringent hard real-time constraints and missing a single deadline may have some catastrophic consequence on the user or the environment. Therefore, time predictability is a major concern, and the safety-critical industry is heavily relying on resource over-provisioning to mitigate such an unacceptable deadline-miss risk.

A common practice to guarantee the deadlines of a safetycritical application with single-core architecture is to determine the application Worst-Case Execution Time (WCET). This WCET computation usually relies on analysis tools based on static program analysis tools [START_REF] Wilhelm | The worst case execution time problem, overview of methods and survey of tools[END_REF], [START_REF] Puschner | Guest editorial: A review of worst-case execution-time analysis[END_REF], detailed hardware model, as well as measurement techniques through execution or simulation [START_REF] Heckmann | Verifying safety-critical timing and memory-usage properties of embedded software by abstract interpretation[END_REF]. However, these analysis techniques and tools are not currently able to provide an exact computation of the WCET, only delivering an estimated upper bound, introducing some safety margins as depicted in Figure 2. A direct extension to the measurement-based analysis to multi-core COTS processors would involve several co-running applications. However this shift worsened the runtime variability problem as contention on shared hardware resources bring new variability sources even in the case of independent 1 2 3 4 5 q q q q q q q q q ADPCM FFT SHA susan pedestrian CRC32 blowfish patricia radar perf. slowdown over standalone median runtime (a) 1 2 3 4 5 q q q q q q q q q ADPCM FFT SHA susan pedestrian CRC32 blowfish patricia radar perf. slowdown over standalone median runtime (b) 1 2 3 4 5 q q q q q q q q q ADPCM FFT SHA susan pedestrian CRC32 blowfish patricia radar perf. slowdown over standalone median runtime (c) Fig. 3. Runtime variability over 600 iterations of reference applications running (a) standalone, (b) concurrently with 2 benchmarks stressing the shared memory path, and (c) concurrently with 7 benchmarks stressing this resource. co-running applications. Despite all the improvements in the WCET estimation domain [START_REF] Kirner | Using measurements as a complement to static worst-case execution time analysis[END_REF], [START_REF] Ferdinand | Program analysis and compilation, theory and practice[END_REF] over the last decades, the over-estimation remained mostly constant as the predictability of the architecture decreased [START_REF] Wilhelm | The worst case execution time problem, overview of methods and survey of tools[END_REF], thus making the use of WCET analysis tools difficult for real industrial programs running on multi-core COTS architectures [START_REF] Kirner | Obstacles in worst-case execution time analysis[END_REF], [START_REF] Mezzetti | On the industrial fitness of wcet analysis[END_REF]. Possible interference on shared hardware resources among co-running tasks significantly increases the complexity of timing analysis, forcing it to have a full knowledge of co-running tasks at software level, and detailed resource contention models at hardware level.

In Figures 3(a,b,c), we show that hiding the worst-case variability with additional safety margins as it was performed in the past is not an option for multi-core COTS. Figure 3(a) illustrates, with distribution violin plots [START_REF] Hintze | Violin Plots: A Box Plot-Density Trace Synergism[END_REF], the runtime variation for some applications running standalone on 8-core barebone platform. To mimic a single-core configuration, the other cores are idle. In such a configuration the runtimevariation remains very low (below 1%).

However, when introducing co-running benchmarks, the runtime variation of each application around the previously computed standalone median is increasing rapidly as depicted in Figure 3(b) showing runtime variation in presence of 2 corunning benchmarks stressing the shared memory path. The impact on the average runtime is not significant, however, for the worst-case, we observe an average variation of 71% and a maximum variation of 361%.

Increasing the number of co-runners as depicted in Figure 3(c) furthermore degrades the runtime variation, up to an average variation of 87% on the worst case and a maximal variation of 396%. Additionally, the average and minimal execution times start to be impacted as well, with a variation of 54% on the average runtime.

Another study [START_REF] Nowotsch | Leveraging multi-core computing architectures in avionics[END_REF] from EADS also exhibits that using actual WCET analysis techniques for multi-cores would force the industry to multiply the WCET by a value close to the number of cores being used, providing no performance benefits over single-cores.

As considering the impact on such a large runtime variation would lead to unsustainable WCET margins far above the performance benefits of multi-core systems, it is critical to control this variation by providing a detailed characterization on how each co-running application is behaving relatively to the shared hardware resources.

In this paper, we present measurement techniques allowing us to (1) characterize the underlying architecture, identifying the shared hardware resources responsible of most of the runtime variation, (2) identify which shared hardware resource each application is sensitive to, and (3) predict which applications could run smoothly together without endangering individual worst-case execution times.

II. MEASUREMENT TECHNIQUES

In this section we present two different measurement metrics and tools: Hardware Monitors and Stressing Benchmarks. These metrics allowed us to study and characterize the behavior of co-running independent applications, as well as the behavior of the contention mechanisms of the multi-core architecture.

A. Hardware monitors

Most recent architectures include some special on-chip hardware, the hardware Performance Monitoring Counters (PMC), that provides to the system accurate information of hardware events. Once collected from special purpose registers counting the occurrence of architectural events, this information provides performance information [START_REF] Sprunt | The basics of performance-monitoring hardware[END_REF] on the applications, the operating system, and the underlying hardware behavior. Therefore, it is usually used to guide the programmer into better tuning the applications to maximize their performance through various optimizations [START_REF] Salapura | Power and performance optimization at the system level[END_REF], [START_REF] Azimi | Online performance analysis by statistical sampling of microprocessor performance counters[END_REF]. This monitoring information is not only useful for performance tuning, but also to characterize workloads, allowing us to quantify hardware resource utilization at application level and providing some clues to better understand runtime variability [START_REF] Duesterwald | Characterizing and predicting program behavior and its variability[END_REF]. In a multi-core context, hardware monitors are an opportunity to observe contention phenomenons at the level of the shared hardware resources.

The real source of the variation could however remain hidden. For example, in current processors we can monitor the number of cache misses that an application has, but we cannot distinguish between cache misses inherent to application behavior and cache misses caused by the interaction with other applications accessing the same shared cache [START_REF] Zhao | Cachescouts: Fine-grain monitoring of shared caches in cmp platforms[END_REF].

The special purpose registers available from the microarchitecture instruction-set are confined to count on-die related events. In some architectures, it would imply not being able to gather some events related to the last cache level, as well as the interconnect, and the DDR controller. As contention on these resources can actually be the bottleneck of the architecture, we cannot afford not collecting this information.

However for recent embedded architectures, the integrators are proposing some monitoring facilities at platform level. These monitoring features are most of the time dedicated to debugging through proprietary hardware probes but allow to count events at all the platform levels, including the number of DRAM refresh, page switch, and so on. This ability is a proof of the existence of some additional (most of the time undocumented) platform-level hardware counters that could be exploited through some reverse-engineering.

B. Stressing benchmarks

To better characterize concurrent accesses to shared hardware resources, we defined a large set of stressing benchmarks, each dedicated at stressing a particular potentially shared hardware resource.

On one hand, solely running a set of stressing benchmarks on the target system allows us to characterize the underlying architecture, by identifying which resources are effectively shared, as well as providing some information related to undisclosed features, such as resource-level contention mechanisms.

On the other hand, by selecting some of these stressing benchmark as co-runners for an application and studying the performance impact on the application, we are able to characterize how a black-box application behaves relatively to a particular hardware resource.

As an extension of the work proposed in [START_REF] Radojkovic | On the evaluation of the impact of shared resources in multithreaded cots processors in time-critical environments[END_REF], each resource stressing benchmark is dedicated at introducing a high-load onto one of the processor hardware resources. As a consequence, the resource-stressing benchmark is providing a good upper-bound estimate of the potential slowdown that a set of simultaneously-running applications may cause to the target application with respect to the studied resource.

Stressing benchmarks are directly written in assembly code to minimize the impact on the other resources the stressing benchmark do not aim at stressing. Even though stressing a single resource is not necessarily possible (like stressing the L3 cache without impacting the L1 and L2 caches), we tried to reduce the effect by minimizing the number of lines impacted in the L2 cache while stressing the L3.

We separated stressing benchmarks into different categories corresponding to the part of the architecture these benchmarks target: pipeline-level benchmarks (stressing the FPU unit, the load/store unit, the branch unit, ...), the i/o level (stressing pcie, ethernet, or serial interfaces), the memory level (stressing on-die caches, platform caches, or the ddr-controller)

C. Representing runtime variability

In this paper, we are considering safety critical applications that requires to control their runtime variability to ensure that their worst execution time is below the hard real-time deadlines.

Runtime variability however, is not only characterized by a minimum and a maximum runtime: The statistical distribution of the runtimes also provides some useful information such as the rarity of the worst case, the distribution relatively to the median runtime, ... To represent this distribution of runtimes, we relied to violin plots [START_REF] Hintze | Violin Plots: A Box Plot-Density Trace Synergism[END_REF] such as the one presented in Figure 4.

III. METHODOLOGY

As previously described, safety-critical systems typically consist on a multitude of independent applications. To maintain / reduce NRE costs while improving usage / performance embedded COTS multi-cores are the targeted hardware solutions.

Especially, the avionic domain is relying a lot on subcontracting and is facing an important challenge during the integration phase, dealing with the integration of both grayand black-box components, while part of the hardware being undisclosed as using COTS architecture.

Our objective is therefore to characterize how each application is individually accessing each potential shared hardware resource, to anticipate at integration time, how these applications will impact each other when running concurrently on the multi-core.

During our study depicted by Figure 5, we started with characterizing the architecture to identify the shared hardware resources of the system. We then focused on characterizing the applications to identify which resources each application was accessing.

A. Architecture characterization

The objective of architecture characterization is twofold: first to identify the effectively shared hardware resources of the target architecture, and second to discover undisclosed architecture features (such as the exact structure of the NOC) that can have a significant impact on runtime and contention.

To perform the architecture characterization analysis, we first run a particular resource stressing benchmark on the target multi-core processor in isolation to collect all the hardware monitor information. We then replicate this stressing benchmark up to the number of available cores to collect hardware monitor information again. Comparing these results allows us to quantify the contention mechanism of this stressed particular resource, while reproducing this study for every potentially shared resource enables us to identify each effectively shared resource as well as the maximum throughput available on each of these resources. Also, varying the deployment pattern of stressing benchmarks on the architecture enables us to learn undisclosed architecture features, such as the structure of the interconnect, the cost of the coherency traffic, the number of shared ports on a shared cache and so on... Finally, running all the different stressing benchmarks together may allow us to identify correlated resources and monitors, as well as constant monitored values. Such information is useful to discover useless monitors and stressing benchmarks. For example intituively in a multi-core architecture with distributed L1 and L2 caches, the monitors counting L1 hits and misses are useless when studying the impact of an application running on a core on another application running on a different core, and so are the stressing benchmarks stressing the L1 cache, as co-running application will never cause a contention on this resource; on the other side monitors counting the number of requests send by a core to the interconnect might cause contention between applications. At the end of this phase, a subset of hardware monitors and stressing benchmarks appropriate to the target architecture are identified.

If the target architecture supports different configuration modes (such as with some caches enabled or disabled, with optional contention, with some cache partitioning, etc...), then the whole architecture characterization phase could be replicated for every configuration to identify the most suitable configuration for a particular application domain.

B. Application characterization

The purpose of the application characterization analysis is to understand why a particular application performance, and thus its execution time, varies when the application is running with other applications in the same multi-core. As most these variation are due to conflict on shared hardware resources access, it is critical to identify which of such resources each application requires, as well as to quantify the usage of the resource by the application.

To perform the application characterization analysis, we run each application with every stressing benchmark from the subset identified by the architecture analysis, gathering hardware counters information related with shared hardware resources. Application slowdowns allow us to identify to which resources this application is sensitive as well as a quantification of the sensitiveness thanks to the observed performance variability.

Also, it enables us to evaluate the minimal share of the resource the application needs to be provided with, prior to facing significant performance degradation.

Finally, repeating the process on every application would allow us to identify applications likely to run concurrently on the system without significantly degrading individual worstcase execution times.

IV. EXPERIMENTAL SETUP

This section first presents both the hardware and the software configurations we considered for our study; and second quantifies the associated design space size. The associated results will be presented in Sections V and VI.

A. Hardware configuration

The experiments presented in this paper are performed with the 8-core Freescale P4080 Development System [START_REF] Freescale | P4080 Product Summary Page[END_REF]. This system, depicted in Figure 6 and Table I Freescale Semiconductor 3

Block Diagram

Figure 1 shows the major functional units within the P4080. -All cores 1.0-V -Two cores up to 1.0-V to achieve higher frequencies -Up to 6 cores down to .9-V for lower power consumption at lower frequencies • Maximum memory data rates of 1066 MHz (DDR2), 1600 MHz (DDR3) -DDR: 1.8-V for DDR2, 1.5-V for DDR3 (conforms to JEDEC standard) Besides core-related performance monitors, the P4080 platform provides us with a set of poorly documented memorymapped registers allowing us to monitor interconnect activity, L3 cache and DDR accesses.

Architecture tuning. Most multi-core COTS provide some amount of configurability to adapt users' requirements. For example, caches help to reduce data transfer time between cores and memory to improve overall system performance, but they also bring performance variability when they are shared among co-running applications. To improve predictability, most embedded architectures allow to simply disable caches, some provide some degree of hardware partitioning, and few allow the caches to be configured as SRAM memories.

Our P4080 platform provides us with two shared L3 caches each connected to a dedicated DDR controller. Beyond allowing us to enable one or both of the L3 cache / DDR controller pairs, the P4080 L3 cache offers some hardware partitioning support, allowing us to shift from a cache fully shared by the cores to a cache where each core has a dedicated pre-allocated memory space. While the fully shared cache will in principle provide better overall performance by better fitting asymmetric load balancing scenarii, the pre-allocated cache will provide better predictability by preventing evictions due to other cores.

In the context of safety-critical systems, the optimal configuration is characterized by low performance variability and sufficient average performance. However, it is not obvious to intuitively infer the most suitable configuration. For instance, partitioning may degrades performance below acceptable throughput, and activating the second L3 cache / DDR controller only makes sense if it does not compromise the predictability of the interconnect.

We have selected four configuration candidates described in Table II to be challenged for performance and predictability. The experiments realized to determine the optimal configuration for our safety-critical context is described in Section V-B.

B. Software configuration

As a proxy for various independent applications co-running on a safety critical system, we used a subset of the MiBench benchmark suite [START_REF] Guthaus | Mibench: A free, commercially representative embedded benchmark suite[END_REF], a set of embedded benchmarks from various domains of the embedded market: automotive, consumer, office, networking, security and telecommunication. We selected a subset of 7 mibench benchmarks: ADPCM, CRC32, FFT, blowfish, SHA, patricia, and susan to be ported for barebone testing on the P4080 hardware platform, eliminating operating system requirements such as system calls and dynamic memory management.

We completed this suite of small benchmarks with two larger industrial-level applications developed internally at Thales including additional hard real-time constraints: 1) an airborne radar application based on the Space-Time Adaptive Processing (STAP) algorithm [START_REF] Wicks | Space-time adaptive processing: a knowledge-based perspective for airborne radar[END_REF] to detect targets in the presence of both clutter and jamming. 2) a pedestrian detection application based on the Viola & Jones shape recognition algorithm [START_REF] Viola | Robust real-time object detection[END_REF] to detect pedestrian on security camera footage.

All the applications presented in this paper were run barebone on the platform, without any operating system to minimize variability. We eliminated preemption which has to be strictly controlled for safety-critical systems, by having each core running a unique benchmark.

Applications were loaded using a hardware debug probe managed through the CodeWarriork [START_REF]CodeWarrior Development Tools[END_REF] software provided by Freescale.

C. Design space

Considering the available number of hardware monitors (∼ 200), the total number of stressing benchmarks (∼ 1000), and the total number of possible mappings on an 8-core architecture the the experimental space of the methodology presented in Section III can be quite large.

Let A be the number of applications to characterize, S the number of stressing benchmarks, M the number of available hardware monitors, C the number of cores, I the number of iterations of repeating each experimental scenario, and N the number of monitor each core is able to measure at once. The total number of possible experiments is:

1 N (AM C(S + 1) (C-1) + SM C(S + 1) (C-1) )I
Considering that average execution time of a single experiment to be 100ms, and the order of magnitude for the values presented in Table III

TABLE III. ORDER OF MAGNITUDE OF THE DESIGN SPACE

To deal with such a design space we setup an automatic framework. As depicted in Figure 5 we perform an overall architecture characterization prior to performing the application characterization.

Beyond the characterization aspect presented in Section V, the architecture characterization phase also allows us to filter out unnecessary hardware monitors and stressing benchmarks by identifying those that are not related to performance variability. For instance, hardware monitors and stressing benchmarks related to the L1 data cache were filtered out by our framework, as the L1 data cache is not a shared hardware resource and therefore not responsible for performance variability in a co-running context.

Beyond this first step towards the reduction of the design space size, the Section V and the Section VI will show how more detailed characterization of both the architectures and the applications helps to furthermore reduce this space.

V. ARCHITECTURE CHARACTERIZATION RESULTS

This section regroups all the architecture characterization related results. We present first how our methodology allowed us to identify undocumented architecture features, and second the experiments that enabled us to select the most suitable hardware configuration for safety-critical real-time applications. Finally, we detail the experiments that allowed us to quantify each hardware resource availability.

A. Identifying undocumented architecture features

Embedded COTS architectures come with detailed ISA and block diagram, but many aspects of the micro-architecture remain undisclosed such as the exact SoC network topology, contention, arbitration and prefetcher mechanisms. If such information is not necessary to guarantee correct functional behavior, it could have a significant impact on the timing behavior, that is as much important for safety-critical real-time systems.

The architecture characterization phase is an opportunity to learn about these undisclosed features and mechanisms. More particularly, our experimental P4080 platform features a complex CoreNet interconnect to connect all the cores to two L3 platform caches, each connected to a DDR controller. The topology of this interconnect has a significant impact on how the memory of one core will interfere with the traffic of other cores.

The architecture characterization allows us to quantify the competition on the CoreNet interconnect resource, as well as the impact of both the interconnect and the mapping on the application runtime variability.

To characterize the interconnect, we used the default platform configuration appearing as first row of Table II. By running and monitoring various stressing benchmarks within this configuration, we managed to figure out that the eight cores are organized as two clusters of four cores.

The experiment illustrating this cluster effect is depicted in Figure 7. It shows the runtime variability of running three instances of a particular stressing benchmark of the 8-core architectures, mapping and monitoring the first instance on core #1, while varying the mapping of the other two instances on the other available cores. 1.30 q q q q q q q q q q q q q q (1,4,5) (1,5,6) (1,6,7) (1,3,4) (1,7,0) (1,2,3) (1,0,2) perf. slowdown over standalone median runtime mapping Fig. 7. Runtime variability while mapping three instances of a stressing benchmark on different cores.

Three different distributions can be identified in the figure:

The first one, corresponding to the first three violin plots, corresponds to mapping the monitored instance alone while running the two other instances on the other cluster. It exhibits only a small performance degradation with a maximum of +7.6% and relatively small variance between the three mappings. The second distribution, illustrated by the next two violin plots, corresponds to mapping the monitored benchmark on the same cluster as one of the two other instances and shows a bigger performance degradation with a maximum of +15.6% and low variability. Finally, the third distribution, illustrated by the last two violin plots corresponds to running all three instances on the same cluster and exhibits the largest performance degradation and an important variability (from +16.3% to +23.0%).

As a conclusion, due to the cluster effect, performance variability of applications is placement dependent for the P4080 platform. However within each 4-core cluster the performance does not depend on the placement, enabling us to reduce the number of mapping to be tested, and therefore allowing us to reduce the overall design space.

B. Identifying the optimal hardware configuration

In Section IV-A we identified several hardware configurations presented in Table II. Selecting the most appropriate configuration for safety-critical applications is not straightforward as two criteria have to be maximized: 1) predictability, ensuring low performance variability, and 2) sufficient minimal performances assuring the worst case execution time will be below the application deadlines of the hard real-time system.

To evaluate these different hardware configurations, we designed a set of stressing benchmarks dedicated at stressing the different shared hardware resources along the memory path including the CoreNet interconnect, the L3 caches and the DDR controllers.

While monitoring the stressing benchmark running on core #1, we varied the number of stressing benchmarks running on the remaining cores. Figure 8 shows the distribution of the observed runtime of core #1 while varying the number of corunners, during 200 iterations of these experiments. q q q q q q q q 1 2 3 q q q q q q q q 1 2 3 q q q q q q q q 1 2 3 The y-axis on the left shows the observed speed down compared to when running the stressing benchmark standalone for each particular configuration. The y-axis on the right correspond to the overall runtime.

Performance variability of the different configurations can be obtained by comparing the height of the various distribution violin plots relatively to the left y-axis, tallest plots being the one with the largest variability. Worst performance of the various configurations can be observed with the top-most point of each violin plot. The associated runtime appears on the right y-axis.

The configurations exhibiting the lowest variability are the configurations with a partitioned L3 cache appearing in Figures 8(b) and(d). On the other hand, enabling a second L3 cache and associated DDR controller in configurations in Figures 8(c) and (d) also allow the system to reduce performance variability by providing some load balancing, while increasing the overall performances.

To put it simply, on one hand, activating the second L3 cache and associated controller brings both more predictability and more performance. On the other hand, shared L3 caches are providing more performance while partitioned L3 caches are offering more predictability.

To compare these two last configurations, we collected worst execution times of Figures 8(c) and (d) into Table IV. Even by offering a larger variability, the shared configuration provide better overall performance leading to lower worst case runtimes than the partitioned configuration. Therefore the worst case upper bound has the opportunity to be lower for the shared configuration, making the dual-controller shared configuration the most efficient setup for our safety critical system.

Even though we shown that the configuration with two controllers and shared L3 caches could be the most efficient for safety critical systems as allowing a lower upper bound for worst execution time, avionic applications usually favor partitioning as it allow to minimize interferences, here eliminating eviction due to other co-running tasks. For this reason we focused on the configuration with two controllers but with partitioned L3 caches for the remaining of the paper.

Using this configuration is also in cope with the cluster organization identified in Section V-A, and will benefit from the same design space reduction options as applications sharing the same cluster / L3 cache / memory controller will compete on the same shared hardware resources, whereas application placed in different clusters will not.

C. Selecting the adequate mapping

Selecting an optimal mapping with the considered dualcontroller partitioned hardware setup is important as the amount of resource competition between tasks will be tied to their core placement, large competition for tasks running on the same cluster, and low to no competition for tasks running on different clusters.

This optimal mapping largely depends on the application to be considered. In a system only running critical blackbox tasks with the same level of criticality, a fair load balancing of the tasks between the cluster should be privileged.

In a mixed critical system running a few very high-critical applications together with some lower-critical ones, designers may want to keep one cluster for the high-critical tasks, and the other cluster for the low critical tasks. This way, the possible impact of low-critical tasks on high-critical ones is minimized by reducing the sources of resouce competition between them. q q q q q q 1 2 3 q q q q q q q q 1 2 3 Figure 9 presents these both setups. We placed the benchmark with highest criticality level in the first cluster, and then augmented the number of co-running benchmarks. In Figure 9(a) new benchmarks were placed to ensure fair load balancing, in Figure Figure 9(b) new benchmarks, considered of lower criticality were placed first on the other cluster.

While 9(a) corresponding to the fair load balancing exhibit similar and more regular variabilities, for 9(b) the performance variability starts to be impaired only when the number of co-runners do not fit anymore into the second cluster. As a consequence, mixed-critical system should better not try to use all the avialable ressources to ensure that the low-critical traffic does not impact the high-critical one.

D. Quantify resource availability

With a hardware setup selected, the last step of the hardware characterization is to identify the available amount of each hardware resource, quantifying maximum throughput, available number of concurrent accesses and so on.

The most important resource to be characterized (and the least documented) is the CoreNet interconnect connecting the core clusters to their dedicated L3 cache and DDR controller, as well as to the other off-chip resources.

Evaluation of the CoreNet interconnect. Very few details on the topology of the CoreNet interconnect are available, and the only information available in the P4080 reference manual is the 0.8 Tbps coherent read bandwidth.

To better characterize this interconnect, we need to figure out its maximum throughput corresponding to the amount of traffic needed to saturate the interconnect. Remaining strictly below this saturation value would allow us to minimize the variability due to the interconnect, while getting closer to the saturation value would mean a significant increase of the runtime.

We also need to figure out how this available bandwidth is distributed among the clusters. Is the total bandwidth shared by all the cores, or is this bandwidth partitioned per-cluster?

To collect this information, we set up a dedicated stressing benchmark performing misses in the L1 and L2 caches to maximize possible CoreNet usage, and tuned this stressing benchmark to be able to select the CoreNet stressing level (the number of CoreNet accesses per CPU cycle). The results of the related experiments are presented in Figure 10. CoreNet transaction per cycle perf. slowdown over standalone median runtime q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q (a) total CoreNet load while running 4 co-runners in 1st cluster CoreNet transaction per cycle perf. slowdown over standalone median runtime q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q (b) total CoreNet load while running 8 co-runners in both clusters CoreNet transaction per cycle of cluster1 perf. slowdown over standalone median runtime q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q (c) cluster1 CoreNet load while running 8 co-runners in both clusters In a real-time context, to enforce that the activity of a cluster does not impact the activity of the other one, we therefore need to make sure that this maximum bandwidth is not reached.

Evaluation of the DDR. Another important shared hardware resource to consider are the DDR controllers and the associated DDR memory.

To similarly evaluate the maximum throughput and saturation value of each DDR controller, we shifted back to the hardware setup with one unique L3 cache and associated DDR controller. This allowed us define a new stressing benchmark aiming at stressing the DDR controller with the load of 8 different running cores performing misses in the L3 cache. DDR access per cycle perf. slowdown over standalone median runtime q q q q q q q q q q q q q q q qq q q q q q qq q qq q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q Fig. 11. Runtime variability versus DDR controller accesses to identify each DDR controller maximum bandwidth again an inflection point when saturating the DDR controller when reaching 0.042 accesses to the controller per CPU cycle.

VI. APPLICATION CHARACTERIZATION RESULTS

In the previous section we quantified the different available shared hardware resources. In this section, we will focus on identifying the resource requirements of the applications. The application runtime variability while stressing a particular resource will allow to determine the share of the hardware resource that each application requires.

We will start by figuring out the optimal number of iterations required to fully capture the runtime variability of each application, and then quantify the resource usage of each application. Such an information about resource usage could later be used to determine which applications could run smoothly together.

A. Optimal number of iterations to capture variability

To capture runtime variability of a particular application, each experiment involving this application has to be run several times in successive iterations. A large enough number of iterations will be able to capture the whole runtime variability of the application, while a not sufficiently large number will miss the runtime with the rarest distribution. As missing the worst execution time is not an option, we need to figure out what is this optimal number of iterations allowing to fully capture the runtime variability.

To empirically determine this optimal execution iteration number of a particular application, we setup an experiment performing successive executions of this concurrently with a resource-stressing environment. Every 100 iterations, we collected the runtime distribution since the beginning of the execution.

Figure 12 shows such runtime distribution results for the Adpcm benchmark. The shape of a violin plot corresponds to the captured behavior of the application. Therefore, comparing the violin plot shapes enables us to figure out if the optimal iteration value has been reached. For these experiments, stopped the iteration counter when we obtained three consecutive identical violin plots. For Figure 12 the optimal number of iteration is therefore 1000.

Identifying the optimal number of iterations to capture the runtime variability of each application allows us to reduce the 1.30 q q q q q q q q q q 100 200 300 400 500 600 700 800 900 1K perf. slowdown over standalone median runtime number of iterations Fig. 12. Runtime variability collected with different number of iterations for application Adpcm. overall design space. We applied the same methodology to identify the optimal number of iterations for every experiment. We found that the largest number of required iterations was 1000, and the average number was 200.

B. Capturing resource utilization

Section V allowed us to quantify the maximum number of resource available in the architecture. We now want to capture the resource requirements of each application from the software setup described in Section IV-B.

To perform this measurement, we ran each application standalone, collecting the same hardware monitors as the ones which allowed us to quantify the CoreNet and the DDR controller maximum load in Section V-D. Table V lists both the average and peak number of resource usage for the considered applications. The maximum average usage remains quite low: 2.23% of the available CoreNet resources, and 11.68% of the DDR resources. The peak usage however is significant with Adpcm using as much as 47% of the available DDR bandwidth.

C. Predicting co-running behavior

With the resource usage of each benchmark and the total amount of available resources quantified, we can use this information to predict the behavior of co-running applications.

Looking at the peak usage of the DDR resource for the ADPCM application if Table V, up to two instances of AD-PCM should run fine on the same cluster with low performance impact compared to the standalone version; 3 and 4 instances should start to exhibit significant speeddown. Performance slowdown with difference number of co-running ADPCM.

Figure 13 depicts the performance variability while running an increasing number of ADPCM instances on a single cluster. All speeddown are normalized to the standalone execution time of ADPCM. Running two concurrent instances of the ADPCM benchmark produces a slight maximum performance degradation of +2%. This is in cope with the fact that the DDR controller is just below saturation. When running three concurrent iterations the maximum increases to +5%, and to +8% when co-running four different instances.

Even though the impact on runtime behavior is not that high, the behavior is correctly captured. The reason why the maximum performance degradation is only 8% is due to the fact that the average DDR controller usage of ADPCM is only 9%, far away from the peak usage of 47%.

To better capture the impact level on the architecture as future work, resource usage distribution could be used instead of peak usage for co-running performance prediction.

VII. CONCLUSION AND NEXT STEPS

In this paper, we presented a methodology and its associated automatic framework allowing us to characterize both the hardware and the safety-critical software relying on hardware monitors available in multi-core architectures and stressing benchmarks.

From the hardware point of view, we successfully quantified the shared hardware resource availability, an identified some undisclosed hardware features. From the software point of view, we were able to capture the median and peak resource utilization of the applications, allowing us to perform a first prediction on co-running application behavior.

Fig. 1 .

 1 Fig. 1. Evolution of code size in space, avionic and automotive embedded systems

Fig. 2 .

 2 Fig. 2. Estimation of the Worst-Case Execution Time, and the over-estimation problem

Fig. 4 .Figure 4

 44 Fig.[START_REF] Duesterwald | Characterizing and predicting program behavior and its variability[END_REF].Example of violin plot to represent runtime distribution of two different applications Figure4is composed of two violin plots providing some information about the runtime distribution of two applications A and B. For each application the variability is characterized by the bottom-most point and the top-most point that correspond to the minimum and maximum observed running times. The black dot in the violin represents the median running time. Finally the width of the plot for a particular runtime (r in the figure) is proportional to the density of the runtime population with such a runtime.

Figure 1 •

 1 Figure 1. P4080 Preliminary Block Diagram 1.2 Operational Parameters • Top speed bin e500mc core frequency of 1.5 GHz at 1.0 V • Core supply voltage options: -All cores 1.0-V -Two cores up to 1.0-V to achieve higher frequencies -Up to 6 cores down to .9-V for lower power consumption at lower frequencies • Maximum memory data rates of 1066 MHz (DDR2), 1600 MHz (DDR3) -DDR: 1.8-V for DDR2, 1.5-V for DDR3 (conforms to JEDEC standard) • Local bus: 3.3-, 2.5-, or 1.8-V • Operating junction temperature range is 0-105C. • Max Power dissipation target <30 W: 8 x 1.5 GHz, 1.0-V operation • Package: 1295-pin FC-PBGA (flip-chip plastic ball grid array)

Fig. 6 .

 6 Fig. 6. Freescale P4080 block diagram

Fig. 8 .

 8 Fig. 8. Runtime variability of one of the stressing benchmarks while varying the number of co-running instances.

  fair load balancing between tasks with the same criticality level

Fig. 9 .

 9 Fig. 9. Comparing the runtime variability of different balancing techniques.

Figure 10 (

 10 Figure 10(a) depicts the correlation between the performance variability and the CoreNet load while running four instances of our stressing benchmark on the first cluster. The figure clearly shows a brutal degradation of performance when reaching 0.219 CoreNet transactions per CPU cycle. This inflection point corresponds to when the interconnect becomes saturated just before reaching the maximum bandwidth.

Figure 10 (

 10 Figure 10(b) describes the same correlation when running eight instances of the stressing benchmark. The performance degradation now occurs when the number of CoreNet transactions reach 0.406 transaction per CPU cycles. This nearly doubled value means that the full available CoreNet bandwidth

Fig. 10 .

 10 Fig. 10. Performance slowdown variability versus CoreNet load to identify CoreNet maximum bandwidth

Figure 11

 11 Figure11shows the correlation between the performance variability of the benchmark monitored in core #1 and the number of accesses to the DDR controller. The figureexhibit

  over standalone median runtime number of co-running Adpcm

Fig. 13 .

 13 Fig. 13.Performance slowdown with difference number of co-running ADPCM.

  , is composed of eight e600mc PowerPC cores coupled with private L1 data+instruction caches and a private L2 unified cache. The eight cores communicate with two shared off-chip L3 platform caches through the proprietary CoreNet interconnect. Each L3 cache is itself connected to a dedicated DDR controller.

	Core	8 Power Architecture e500mc at 1.5GHz
	Pipeline	7-stage pipeline, superscalar, out-of-order
	Distributed L1 caches	32kb, 8-way associative, 64-byte line, PLRU
	Distributed L2 cache	128kb, 8-way associative, 64-byte line, PLRU
	Performance monitors	162 performance monitor counters
		4 special registers per core
	Shared L3 cache (x2)	1MB, 32-way associative, 64-byte line, PLRU
	Memory controller	Two DDR memory controllers
	Interconnect	CoreNet Coherency Fabric

TABLE I

 I

	.	FREESCALE P4080

SPECIFICATIONS

Hardware monitors. Each e600mc PowerPC core provides 162 different performance monitors, and has the ability to P4080 Features QorIQ™ P4080 Communications Processor Product Brief, Rev. 1

  • Local bus: 3.3-, 2.5-, or 1.8-V • Operating junction temperature range is 0-105C. • Max Power dissipation target <30 W: 8 x 1.5 GHz, 1.0-V operation • Package: 1295-pin FC-PBGA (flip-chip plastic ball grid array)

		QorIQ P4080		128-kbyte Backside L2 Cache	Power Architecture™ e500mc Core 32 D-Cache 32-kbyte I-Cache			1024-kbyte Frontside L3 Cache 1024-kbyte Frontside L3 Cache		64-bit DDR2/DDR3 Memory Controller 64-bit DDR2/DDR3 Memory Controller
	eOpenPIC
	BootROM Internal PreBoot Loader Security Monitor	PAMU	PAMU				PAMU		CoreNet™ Coherency Fabric PAMU			PAMU	Peripheral Access Mgmt Unit
	Power Mgmt						Frame Manager	Frame Manager					Real Time Debug
	SD/MMC SPI	eLBC	Security 4.0	Queue Mgr		Parse, Classify, Distribute	Parse, Classify, Distribute	RapidIO Message Unit (RMU)		2x DMA	Watchpoint Cross Trigger
	2x DUART							Buffer			Buffer
	Clocks/Reset 4x I 2 C 2x USB 2.0/ULPI	Test Port/ SAP	Pattern Match Engine 2.0	Buffer Mgr	10GE	1GE 1GE	1GE 1GE	10GE	1GE 1GE	1GE 1GE	PCIe	PCIe	sRIO	PCIe	sRIO	Perf Monitor Aurora CoreNet Trace
	GPIO
	CCSR											18-Lane 5-GHz SERDES

TABLE II .

 II FOUR CONFIGURATIONS OF P4080

  , it would requires us 10 19 years to exhaust such a design space.

	Applications	A	9
	Stressing benchmarks	S	1000
	Hardware monitors	M	200
	Cores	C	8
	Iterations	I	100
	Performance monitor registers	N	4

TABLE IV .

 IV WORST EXECUTION TIMES (IN MS) FOR THE MONITORED CORE WHILE VARING THE NUMBER OF RUNNING BENCHMARKS.

Table V

 V provides the resource usage information for each application, this usage being computed as a ratio to the previously quantified saturation value: 0.219 requests per CPU cycle for the CoreNet interconnect, and 0.042 requests per CPU cycle for the DDR controller.

		average	peak	average	peak
	Application	CoreNet	CoreNet	DDR	DDR
	ADPCM	0.86%	9.04%	4.52%	47.13%
	CRC32	0.93%	1.30%	4.77%	6.90%
	FFT	0.19%	13.38%	0.38%	31.43%
	blowfish	0.14%	0.72%	0.74%	3.74%
	SHA	0.25%	2.15%	1.33%	11.19%
	patricia	0.07%	0.19%	0.35%	0.97%
	susan	0.40%	2.96%	2.01%	15.44%
	airborne radar	2.23%	3.06%	11.68%	16.19%
	pedestrian detection	0.10%	4.29%	0.48%	22.86%

TABLE V .

 V CORENET AND DDR LOADS OF STANDALONE APPLICATION