Dirk Von Wissel

Vincent Talon

Vincent Thomas

Benoist Grangier

Lukas Lansky

Michael Uchanski

Linking model predictive control (MPC) and system simulation tools to support automotive system architecture choices

Today's automotive industry must introduce advanced powertrain technologies as a consequence of the stringent environmental regulations and strong market expectations. This leads to the increase of vehicle variants and to the growth of the powertrain architectures complexity. Hence, the development of hardware and software can no longer be decoupled. Those activities have to run in parallel, starting from simulation and advanced engineering, continuing to the detailed engineering phase, and ending in validation and calibration. In the so-called Model-Based Systems Engineering (MBSE) approach, simulation models including both hardware and control systems are used to first make decisions on possible architectures. This paper presents a new MBSE approach that allows powertrain hardware selection to occur early during the simulation stages. The process combines Model Predictive Control (MPC) for the control system with a physical modeling software package for the hardware. This combination of MPC and physical modeling addresses several practical difficulties that typically hinder attempts at MBSE for hardware selection.

Introduction Context

Today's automotive powertrains must delight customers with performance, reliability, and low noise while simultaneously meeting increasingly stringent regulations on CO 2 and other emissions, all at a competitive cost and fast time-to-market. Embedded software, electronics, sensors and actuators play a critical role and acts as a "glue" to make combustion systems, boosting systems, cooling systems, exhaust after-treatment systems, batteries, and transmissions perfectly integrate for optimal vehicle performance.

The development of the mechanical powertrain hardware and the software functions controlling them can no longer be performed independently. As a result, many automakers implement hardware selection processes where many powertrain variants are built. An electronic control system is then designed and calibrated for each variant. The variant that best meets system-level requirements moves to the next development stage. But this process is money and time-consuming and prevents manufacturers and suppliers to stay in the competition race.

Given the financial incentives, most automotive companies have tried some form of Model-Based-System-Engineering for hardware selection applying system simulation in particular. This implies to have access to the "right" models for both the hardware and the software.

Difficulties frequently encountered with MBSE for hardware selection

Models for the plant (engine, transmission, and vehicle) are generally available at the earlier stages of the product development process. Often generated during R&D and advanced projects, they are not necessarily well adapted to be applied for an innovative and efficient hardware selection process. In particular, the models generated in the with a powertrain subsystem design perspective are often too complex and slow to be combined efficiently with control tools. The present paper illustrates the requirements for getting the right modeling approach for plant model adapted to hardware selection and the implications for the deployment of models throughout the complete product development process.

On the other hand, when models of the plant are available, obtaining the corresponding control models is always a more difficult task. One of two methods is usually employed:  Adaptation of existing production control approaches -This approach provides a path to the implementation on physical prototypes later in the process. However, simulation time can be slow because the production strategy block diagrams often include tens of thousands of elements. Personnel gaps are an even more serious issue. Hardware designers are usually experts in boosting, transmissions, or system-level simulation tools, but have limited skills in complex control strategy diagrams.

 Simplified custom controllers -A simplified control is created from scratch to cover the extra capabilities and degrees of freedom required for the new technology brick. This approach offers fast simulation times and understandable control models. But also come with disadvantages: the time and expertise required to build the controller from scratch is often substantial. Also, since it is often built quickly, controller performance can often be substantially different from that of a well-tuned production controller. As a result, controller tuning differences could be mistaken for hardware performance differences. Finally, the controller used in simulation is often a dead-end with no path to implementation on embedded computing platforms.

Novelty of the proposed approach

The process described in this paper improves on the second method by making it faster to develop custom controllers for new hardware: The novelty is to use physical models with MPC (Model Predictive Control), a systematic, easy-to-use requirement based controller design methodology. The focus of the approach is put on ease. Both Model Based controller Design (MBD) and optimal control are well established in the field of controls engineering but are under-applied in automotive due to the specialized knowledge required to apply them. A dedicated, easy-to-use MPC software package helps to overcome this difficulty and offers the benefits of model based and optimal control design to a larger community of engineers.

The tool approach also offers the possibility to port MPC directly from numeric simulation to embedded software, which can actually run on physical hardware such as Electronic Control Units. Finally, as MPC yields nearly optimal control, tuning is generally of good quality, making it less likely that control tuning choices will mask differences between mechanical hardware.

Methodology

This paper presents a new Model-Based Systems Engineering (MBSE) approach based on the application of existing tools interfaced in order to generate a fast and efficient workflow that allows powertrain hardware selection. The approach is structured around four main stages:

 Development of a non-linear physical plant model for the studied system  Processing of the model in order collect linear models applied for the design of a MPC controller  Validation of the controller using a Model-In-the-Loop environment including the baseline plant model

 Evaluation of the system potential using the plant with controller regarding relevant criteria

The present paper illustrates how this combination of MPC and physical modeling addresses several practical difficulties that typically hinder attempts at MBSE for hardware selection.

The application and benefits of the approach are illustrated on a use case from RENAULT with the benchmarking analysis of gasoline air path systems using a tool chain including LMS Imagine.Lab Amesim combined with the Honeywell OnRAMP Design Suite.

Application of system simulation (Existing tool)

Commercial CAE software is a critical lever toward the management of system complexity and limited development times. This is why it is widely deployed in automotive industry from the early stages of development to the final stages before SOP, where system simulation is applied to validate system integration.

The hardware selection process in the early stages of an innovative product development process can take advantage of the existing deployment of CAE environments and more importantly, its usage for system simulation. The capability to evaluate numerous technical options in a limited time gives a competitive advantage compared to conventional iterative process with real prototypes.

General requirements for system simulation

The application of a simulation based decision process also raises three major challenges in terms of tooling capabilities:

 Integration in a larger MBSE process System hardware selection is located early in the development process. In this so called system predesign phase all system component specifications have to be detailed. The system hardware selection process must be supported by models generated upstream in the V-cycle, which can include an additional level of complexity linked to the poor availability of data. The direct application of high fidelity plant modeling (1D software) is often not appropriate as this can jeopardize the trade-off between prediction capabilities and run times.

 Diversity of hardware selection topics One of the challenges for the system simulation software is to offer the capability a high level of versatility to address different levels of modeling to fit with the variety of systems and sub-systems that must be investigated by car manufacturers. This is why a high level of flexibility and scalability is required from simulation software. For example in the selection of the right air path architecture for a given engine the details of the intake and exhaust flows and their impact on the combustion process have to be known in a precise way while the rest of the vehicle can be modeled in a simple way to study vehicle attributes like fuel consumption or performance. Another example is the development of a hybrid powertrain in which all related powertrain subsystems have to be detailed with high precision but in which the air path system can be described in a simple way.

 Interfacing with control development environments Since hardware models must ultimately be combined with the software model, a common approach is to apply co-simulation technics. Most of the simulation tools offer this kind of interface with Simulink or even with C codes. The problem with co-simulation is that it often has an over-cost in terms of run times. That is also in favor of the application of 0D models compared to high fidelity 1D modeling approaches.

Application with LMS Imagine.Lab Amesim

The 0D models in the LMS Imagine.Lab Amesim (LMS Amesim) software package provide sufficient fidelity to be used in hardware development while remaining simple enough for its use in control design. The 0D lumped parameter approach, which is used in LMS Amesim, offers a good compromise between high level vehicle simulation tools, which are often too simple, and full 1D software that is often too complex. High level software are able to predict the full vehicle attributes over driving cycle but do not allow going deeper into sub-systems details. 1D software is well-suited to engine design, but runs slowly when linked to control environments.

Taking advantage of this flexibility of the LMS Amesim software, RENAULT has progressively adopted it as standard tool for system simulation and plant modeling. It is currently deployed for advanced engineering, vehicle planning phase and supports the full control development cycle from design to MIL to HIL. RENAULT's large plant model database includes detailed engine models, transmissions, actuator models, and a full hybrid vehicle model, and is updated continuously with each new vehicle or powertrain program. RENAULT's controls engineers use these models for system simulation and software validation and calibration LMS Amesim's scripting capabilities and its ability to automatically produce a linearization of full plant models are of special interest for the hardware selection process described in this paper. Both features were used to establish a connection to the MPC control design suite HONEYWELL OnRAMP. On one hand, LMS Amesim is an open software that is delivered with scripting APIs (Matlab, Python…) that ease its integration in processes and application oriented workflows. Hence, scripts were developed in order to pilot remotely the LMS Amesim core, to get the plant model description (definition of I/O, units, ports…), to gather a linear model for a given operating point and to launch non-linear simulations to check the validity of the linear model and validate the designed control function. On the other hand, the 0D software package is able to extract from a physical non-linear model a linear model at a given operating point, which is possible thanks to the use of ordinary differential equations (ODE). Indeed, the numerical methods applied to solve physical equations in CFD1D or 3D software do not support this feature. The automatic creation of linear models from an existing non-linear 0D model is the inbuilt LMS Amesim core capability that made it possible to create the interface with the HONEYWELL MPC design software.

User-friendly MPC control design tool (Existing tool)

MPC (Model Predictive Controls) has been a preferred controls methodology in process industries for several decades thanks to its rapid, systematic approach. Over the past decade, its use has significantly increased in the automotive controls community due to theoretical advances which reduce processing time, and due to the increased computing power of the ECUs themselves. At RENAULT several powertrain control innovation projects are already using MPC in a quasi-industrial context. Other automotive OEMs are also active applying MPC to automotive problems including traction control, vehicle stability control, and exhaust system control.

For the MPC part of this work, RENAULT selected the HONEYWELL OnRAMP software toolset. OnRAMP™ Design Suite is a software tool for modelling and design of advanced control algorithms for a wide range of automotive applications. There are many aspects to consider when evaluating such a tool: certification, compatibility, controller performance, footprint in the ECUs, calibration and documentation. OnRAMP addresses these as a package.

Advanced control toolboxes from academia or the commercial sector often offer significant flexibility and advanced features for MPC design. However they often require advanced knowledge from the user and offer limited support for wide spread deployment within an organization. OnRAMP offers a GUI-driven workflow and automatic tuning algorithms to facilitate the design, tuning and performance evaluation of the MPC controller. The OnRAMP workflow delivers C code that run on more than 15 OEM target environments.

Thus, OnRAMP is targeted at automakers and suppliers who would like to benefit from MPC by making it fast and easy to use for a wide audience including engine calibrators, controls generalists, and MPC controls specialists. The tool proposes a systematic workflow and automates model building and tuning tasks, which are tedious, or that require specialized MPC knowledge.

OnRAMP includes its own modeling libraries. However, the present work seeks to leverage RENAULT's large database of existing physical plant models in LMS Amesim format.

Combining the MPC and system simulation tools into a new MBSE workflow

Since the missing piece of MBSE for powertrain hardware selection is a fast, convenient controller synthesis, it seems natural to link a system simulation tool like LMS Amesim with a MPC control synthesis tool like OnRAMP.

RENAULT engineers have championed this work because they believe that MPC can fill the controls gap in MBSE.

As aforementioned, linking the two tools together is enabled by, first linearization capabilities and secondly by the scripting features of the Siemens PLM Software system simulation platform. The Figure below illustrates the process applied for the design up to the validation of the MPC controller.

The first step of the process is to identify inputs, outputs, disturbances and scheduling variables to the controller. This is determined thanks to a modeling convention in LMS Amesim. In practice, the interface between the two tools is concretized by an interface block included in the plant model which defines the actual control I/O.

The second step of the process is steady-state model analysis and feed-forward design. OnRAMP sends, through the API, actuator positions to the LMS Amesim model with the objective of controlling the plant to satisfactory steady state operating point. LMS Amesim responds with the plant's steady state response, as well as with linear state space models (dx/dt = Ax + Bu, y = Cx + Du) that approximate the behavior of the plant at the proposed actuator positions. OnRAMP then adapts its proposed actuator positions, using information from the A, B, C and D matrices to search in the right direction. Consequently the feedforward actuator positions are found achieving desired set-points while respecting constraints. The non-linear model is then linearized at these feed-forward actuator positions by LMS Amesim. The generated linear models are then processed further in OnRAMP for scaling, order reduction and discretization.

Furthermore the non-linear and linear models step responses are visualized and checked within OnRAMP. This is actually one of the key steps for successful design of MPC feedback control later. Important part of feedback control design within OnRAMP is automatic tuning algorithm to achieve user defined requirements on robustness and bandwidth. This is an essential feature to enable generic design for a wide variety of plants. The user requirements are used to formulate the robust stability condition and OnRAMP then manipulates weighting matrices (e.g. for Q and R in the above) of the MPC cost function such that this condition is satisfied. The controller tuning then consists mainly of adjusting controller bandwidth, relative weights of controlled and manipulated variables and choice of prediction horizon. This leads together with earlier model checks to systematic and intuitive design and tuning of a MPC feedback control.

Finally, the controller designed within OnRAMP is tested in a closed-loop environment, using co-simulation methods. Indeed, the OnRAMP tool can generate the C code for the control and the associated data that can be either exported in Simulink. At this stage, the MiL environment is define as a co-simulation between LMS Amesim and Simulink or can be set in the Siemens platform by importing the Simulink model directly into the LMS Amesim GUI.

Application to hardware selectionengine air path systems

RENAULT engine portfolio already includes gasoline engines that comply with the Euro 6 emissions standard. Engineers are already in the process of upgrading current engines to fit with the upcoming regulations involving new driving cycles like the Worldwide harmonized Light vehicles Test Procedures (WLTP) and Real Driving Emissions (RDE). An update of the control logic and calibration will not be sufficient to fulfill these requirements. As a consequence, new mechanical hardware must be selected.

RENAULT selected a set of candidate suppliers for 10 candidate hardware components, or "technology bricks."

The standard implementation would require to build 10 prototypes and to run test programs to benchmark every component. The resulting costs and time would not be acceptable for RENAULT.

As a consequence, RENAULT decided to apply MBSE and to build 10 virtual prototypes to determine the optimal combination of technology bricks while keeping the associated costs under control. This required building 10 Model-In-the-Loop platforms coupling the plant models with their control models. This can be seen as a very complex, time-consuming task requiring highly skilled engineers for both the plant model and control model development or adaptation. Actually, the process applied by RENAULT does permit to tackle these challenges by taking advantage of:

 LMS Amesim with its well-balanced 0D modeling approach,  OnRAMP with its fast and systematic control design workflow  The coupling of two software one for the design of the other for the controller design and the set-up of the MIL environment

Illustration of the application of the tool chain

The workflow detailed in the paper is illustrated on a simple use case related to the evaluation of hardware for a gasoline air path system. Indeed, downsized turbocharged gasoline engine attributes are strongly dependent on the air path and charging technologies and the associated control. Many different hardware and related architectures are proposed by suppliers, from the common waste-gated turbocharger, to electrical driven compressor and dual-stage devices and the engine manufacturer needs to evaluate their technical versus cost potential.

As a starting point, the tool chain is applied to a standard air path configuration including a turbocharger with a waste-gate and a throttle valve. A physical model for this baseline engine is developed in the LMS Amesim platform, its inputs and outputs are properly set according to the control problem and for the interfacing with OnRAMP. In practice, since the purpose of this engineering task is to define the potential of the system for steady-state and transient operation, models for the actuators were also included in the model as below so as to get an accurate prediction of the real system time response.

The control problem is expressed here in terms of tracked control inputs (sensor signals)boost and charge pressure, outputs (actuator commands)throttle and waste gate and constraintsturbocharger speed and exhaust temperature. The controller is evaluated by applying a given engine speed and boost and charge pressure setpoints and analyzing the results in terms of charging pressure and torque production.

The design of feedback MPC controller consisted of 30 operating points and the controller was designed to heavily prioritize the charge pressure tracking for torque delivery.

Some typical results generated using the MiL environment including the plant model and the MPC controller are given in the followings. The controller performance is good enough to match the charging pressure requirements using a combined control of the waste gate and throttle, and at the end, insures the right shaft torque production even during the highly transient phases of the ARTEMIS cycle.

Since the final goal is the introduction of new technology bricks to address future emissions standards (Euro6c and Euro7), from the baseline configuration presented above, RENAULT introduced the electric supercharger technology (e-boost) to evaluate its potential for increasing the engine performance attributes during transient phase of the cycles in particular. Actually, the combined LMS Amesim-OnRAMP approach permitted to rapidly assess several engineering options:

 Architecture: e-boost can be mounted upstream or downstream the main compressor, can be associated to an additional heat exchanger...  Sizing: power of the e-boost electric machine as a function of the engine performance requirements, eboost can be combined with an upgraded turbocharger...  Pollutants: impact on emissions (NOx, HC, CO) on driving cycle and three-way catalytic converter activation time...

Starting from the baseline air path system, the plant model is upgraded including the e-boost machine.

The e-boost device is added to LMS Amesim engine model upstream of the main turbocharger. The control problem is then changed to take into account this additional actuator and new constraints like the thermal state of the electrical machine or its maximum speed.

The figure below shows some results given by the workflow evaluated on four hardware variantswith and without e-boost device and with two version of the turbocharger. This kind of analysis achieved in a couple of hours demonstrates the efficiency and the potential of the tool chain for the selection and pre-sizing of hardware.

Thanks to this approach where MPC is coupled with the plant models, RENAULT is able to complete these numerical studies in a very short time and select the best settings for each project milestones.

Conclusion

Two complementary software suites, one for physical modeling and the other for MPC control design, were linked into a new workflow that supports automotive mechanical hardware selection. The workflow allows engineers in charge of advanced engineering and pre-design to investigate in a very short time the potential benefit of a new sub-system--a significant step towards "design-right-the-first-time."

Looking ahead, the same systematic calibration approach and scalability that make MPC attractive for predevelopment hardware selection problems, should naturally lead to deployments in production ECUs. RENAULT is currently leading several initiatives in this direction where Renault is evaluating MPC with the use of OnRAMP Design Suite to face scalability and maintainability concerns with production controllers.

Renault is still analyzing how to use MPC on real hardware and industrial application of MPC in powertrain control at Renault remains an open question. Issues yet to be understood better are its impact on ECU metrics and the required changes of the industrial process. Nevertheless, MPC is about to be used for Model based system selection in advanced projects within the company.

  Figure on the right illustrates this. The third step is scheduled linear MPC feedback design. The standard MPC cost function consists of penalties for actuator movement, tracking error and soft constraints. This is outlined in the optimization problem shown in the equation below.


Tewodros Deneke

Lionel Morel

Sébastien Lafond

Johan Lilius

Integration of Dataflow Components Within a Legacy Video Transcoding Framework

Recently the RVC-CAL dataflow language has enabled video codecs to be specified in a more natural way than imperative languages by allowing implicit expression of parallelism and side effect freeness. The tools developed for RVC-CAL have also enabled the automatic generation of parallel C code, among others, from dataflow specifications. This paper introduces a new approach allowing the integration of dataflow components within legacy code. The approach makes use of a generic interface definition that allows seamless interaction between I/O components, which are mostly state operations and are best implemented in imperative languages with data processing components which are mostly stateless dataflow operations and are best implemented in dataflow languages. The advantage of the approach is the ease of development by allowing each language to be used on those parts of the application that it is most appropriate for.

The functionality of the approach is demonstrated by using the generic interface to add a new dataflow based MPEG and HEVC decoder into the legacy video transcoding library FFmpeg.

I. INTRODUCTION

Video is becoming an important medium of communication and accounts for a significant portion of the available bandwidth in the world. Today video is being consumed through various devices and networks. These devices and networks have varying capabilities in terms of screen resolution, storage space, processor speed and bandwidth. Video transcoding is needed to enable seamless exchange of videos among various devices on heterogeneous networks like the internet. Video transcoding is the process of transforming one video format with certain characteristics such as bitrate, framerate and resolution to another video with different characteristics that will better fit the target device. One typical scenario of transcoding would be the delivery of video content by video on demand (VOD) systems, like YouTube, to customers connected trough wireless or wireline using their mobile phone, tablet or IPTV. In order to reach such a wide range of consumer segments, VODs transcode and store videos in several formats that are tailored to each segment.

As presented in figure 1, the components of a video transcoder can be roughly categorized in to two: I/O and video processing components. The I/O components are responsible for reading/writing video/audio in different formats to/from a disk or network (e.g. Muxers and Demuxers). The video processing components are mainly responsible for manipulating the video/audio content (e.g, encoder and decoder). While the I/O components are inherently serial and implemented as state operations, the video processing components are parallelizable and compute intensive and are more naturally expressed with a dataflow programming model. For example a video codec, which is one of the video processing components, is a software that enables compression and decompression of digital multimedia content. The main use of a codec is to compress video so that it uses less resources during transmission and storage and later to decompress it for viewing.

Over the years video codecs have become more efficient in terms of their compression ability while becoming more compute intensive. Until recently the increase in computational complexity of codecs has not been a real concern because computer speed has been doubling every other year following Moore's law. However due to physics constraints like power dissipation and transistor scaling, multi-core architectures have become the solution to allow performance to keep increasing. Programming codecs for multi-cores is currently done using imperative languages such as C/C++ using threads to explicitly express parallelism. This is obviously a challenging, time consuming and error prone process as multi-threading is essentially a way to only prune non-determinism after its introduction [START_REF] Lee | The problem with threads[END_REF]. Research in dataflow programming shows the advantages of using dataflow languages such as RVC-CAL for the purpose of video codec specification and implementation. Among others, the main advantages of using dataflow as the main programming method include ease of use, flexibility, automatic analyzability, automatic parallelizability, visual presentability and above all side effect freeness [START_REF] Orcc | Open RVC-CAL compiler[END_REF], [START_REF] Wipliez | Compilation infrastructure for dataow programs[END_REF], [START_REF] Yviquel | From dataflow-based video coding tools to dedicated embedded[END_REF]. While programming video processing components of a video transcoder (e.g. codecs) entirely in dataflow is an attractive idea, programming the I/O components in dataflow is unnatural on Von-Neumann architecture. I/O operations have been introduced as state operations in the Von-Neumann architecture as a convenient and clean way of reading in data into programs. On the other hand the absence of side effects which is fundamental to dataflow programs does not allow state operations and makes dataflow implementation of I/O operations difficult.

In this paper we propose an approach based on the defini-tion of a generic interface that enables video/audio processing system designers to write dataflow components in a dataflow language and interface them with existing legacy video/audio applications such as a transcoder. The proposed approach allows the splitting of the development of video processing applications in two parts: one part which uses dataflow language and implements parallel sections and another, which uses imperative languages and implements the I/O sections. It combines the advantages of using dataflow languages with the features of legacy imperative code and helps quicker adoption of dataflow languages.

The rest of the paper is organised as follows. Section II introduces the reader with necessary background knowledge. Section III reports on related works. Section IV actually describes the body of work of our proposition. Section V presents experimental results evaluating the benefits in terms of performance and the seemingless integration of a substantial dataflow application into the FFmpeg transcoding platform. Finally, section VI concludes and gives few perspectives to this work.

II. BACKGROUND A. Video Transcoding and FFmpeg

The difference in device resources, network bandwidth and video representation types results in the need for a mechanism enabling video content adoption. This mechanism, called transcoding is currently being used for such purposes as: bitrate reduction in order to meet network bandwidth availability, resolution reduction for display size adoption, temporal transcoding for frame rate reduction and error resilience transcoding for insuring high quality of service (QoS) [START_REF] Chang | Video adaptation: Concepts, technologies, and open issues[END_REF], [START_REF] Xin | Digital video transcoding[END_REF].

As can be noted from figure 2 a generic video transcoder contains five main parts, each having a set of components depicted as blocks. The five main parts include a demuxer, a decoder, spatial and temporal processors, an encoder and a muxer. The demuxer is used to read interleaved streams (e.g. one audio, one video and a subtitle stream) from a network or a file. Usually a set of streams are encapsulated in a container format such as MP4. Packets which are read by the demuxer and contain compressed audio/video frames are then passed to the appropriate decoder to be decompressed. The decompressed audio/video frames are then spatially or temporally processed to adapt the video/audio to a particular framerate and/or resolution. Spatially and temporally processed uncompressed frames are then passed on to the encoder to be compressed via removing temporal, spatial and statistical redundancy that exists inside and among uncompressed frames.

Legacy transcoders such as FFmpeg constitute libraries corresponding to each block. For example FFmpeg contains multiple codec implementations including H264, HVEC, MPEG4. Similarly it contains multiple implementations of muxers and demuxers that can be used to read and write several video and audio file formats such as MP4, MKV, FLV, MP3, WAV, etc.

FFmpeg is an open-source, complete, cross-platform solution to record, convert and stream audio and video [7]. Among others it includes libraries such as libavcodec, libavformat, libavfilter, libavswscale which contain implementations of several video and audio codecs, container formats, filters and spacial processing functions. It is a large project with over 570k lines of code.

The FFmpeg transcoding framework implemented based on FFmpeg libraries provides coarse-grained task (pipeline) and data parallelism. The different components such as the decoder and the encoder can run in a task-based parallel manner. The data parallelism in FFmpeg is currently based on slice-level and/or frame-level threading. In case of slice-level parallelism multiple threads can decode slices (independent parts of a frame) in parallel. In the case of frame-level parallelism multiple threads processes multiple frames. However frames have inter-dependencies and the number of slices in a frame is usually limited to one. Therefore the parallelism implemented in FFmpeg is obviously limited.

B. Dataflow Programming and RVC-CAL

A dataflow program is defined as a directed graph whose vertices are actors (the basic computational units) and edges are unidirectional FIFO channels with unbounded capacity. A stream of data tokens, is processed by actors and passed on to others actors via FIFOs. The advantage of such programming model is its ability to implicitly express concurrency and to enable analyzability. Dataflow graphs we consider here respect the semantics of Dataflow Process Networks (DPNs) [START_REF] Lee | Readings in hardware/software co-design[END_REF], which are related to Kahn Process Networks (KPNs) [START_REF] Kahn | The semantics of a simple language for parallel programming[END_REF]. The main difference between DPN and KPN is that DPN allows actors to check the availability of tokens in the FIFOs. Additionally to the KPN model, DPN introduces the notion of firing. An actor firing, or action, is an indivisible quantum of computation which corresponds to a mapping function of input tokens to output tokens applied repeatedly and sequentially on one or more data streams. This mapping is composed of three steps: input data reading, then computation, and finally output data writing. These functions are guarded by a set of firing rules which specifies when an actor can be fired, i.e. if the number and the values of tokens that are available on the input ports are sufficient.

Dataflow has been used for naturally expressing digital signal processing applications for decades. Currently it has gained particular attention in expressing video processing applications. RVC-CAL is one domain specific dataflow language for video coding based on the DPN model of computation [START_REF] Orcc | Open RVC-CAL compiler[END_REF], [START_REF] Wipliez | Compilation infrastructure for dataow programs[END_REF], [START_REF] Yviquel | From dataflow-based video coding tools to dedicated embedded[END_REF], [START_REF] Eker | Cal language report: Specification of the cal actor language[END_REF]. It was originally developed for specifying video coding standards in the most natural manner. This is due to the fact that video coding is a data-oriented application and can easily be visualized and specified as a dataflow graph where actors correspond to functional units such as discrete cosine transform (DCT) and motion compensation (MC), and video bit stream flows among them via FIFOs. In addition to providing a natural language for specifying video codecs RVC-CAL provides a compilation framework called ORCC 1 . ORCC is implemented as an eclipse plug-in and has backends for C, VHDL and others and comes with various example applications including MPEG4 and HEVC decoders.

An RVC-CAL dataflow program is described as a set of interconnected actors via unidirectional FIFOs. Actors are composed of a set of actions, I/O ports (FIFOs) and internal state variables. Actors perform computation by firing actions depending on the state of their I/O ports and their internal state variables.

A multicore platform ideally runs an RVC-CAL program by executing each actor in parallel as long as they are fireable. However because the number of actors in a typical dataflow application is usually much greater than the number of processors, several actors are usually executed on the same processor. The RVC-CAL runtime therefore maps and schedules actors using different approaches. Among others mapping is done in a round-robin or in a weighted load balancing shame, and scheduling is usually implemented using a round-robin or a data driven algorithm [START_REF] Yviquel | Towards run-time actor mapping of dynamic dataflow programs onto multi-core platforms[END_REF], [START_REF] Yviquel | Efficient multicore scheduling of dataflow process networks[END_REF].

III. RELATED WORK

RVC-CAL [START_REF] Wipliez | Compilation infrastructure for dataow programs[END_REF], [START_REF] Yviquel | From dataflow-based video coding tools to dedicated embedded[END_REF] uses CAL procedures to interface with legacy code and more specifically to call the I/O imperative functions such as file readers and display functions. CAL procedures are included in the RVC-CAL language for this particular purpose and can modify state variables and parameters passed to them by reference. Even though CAL procedures defeat some purposes of the dataflow programming approach such as side effect freeness they are, however practical considering the Von-Neumann architecture. In our work we propose an interface which can be used to call (use) dataflow components from the legacy code. This is important for legacy applications to tap in to dataflow components when appropriate.

In [START_REF] Green | Interfacing java with haskell[END_REF], Mark Green et al. have shown two ways of interfacing haskell which is a functional programming language and java an imperative language. Their motivation for their work was to provide each language an access to certain missing language features from the other. As a use case they showed how Java's I/O and UI related libraries which require imperative features that violate the referential transparency of functional programs can be used in haskell. On the other side they have also shown how the haskell features such as lazy evaluation and higher-order functions can be used in Java programs. In our work we explore a similar but more generic interfacing approach on a dataflow language (RVC-CAL) and the C imperative language using a more realistic application.

In [START_REF] Chatterjee | Integrating asynchronous task parallelism with mpi[END_REF], Chatterjee et al. presented the HCMPI programming model and runtime for programming distributed systems, which unifies asynchronous task parallelism at intra-node level with MPIs message passing model at the inter-node level. With HCMPI's task parallel model, users can benefit from MPI integration with structured task parallelism and dataflow programming. Similarly, in [START_REF] Rabenseifner | Hybrid mpi/openmp parallel programming on clusters of multi-core smp nodes[END_REF] and other research works around hybrid programming approaches, combining programming models is proven to be useful in dealing with hierarchical and various hardware designs.

In this paper, we propose the use of dataflow programming to express the available concurrency in video transcoding applications in a more natural way. At the same time, we allow to keep the effective legacy components of current transcoding applications. We enable this through the use of a generic interface defined between components implemented using different programming paradigms.

IV. INTERFACING DATAFLOW WITH LEGACY CODE

Ideally we would need to find the most transparent way to accomplish the interfacing. It should be done in such a way that enables code in both languages to remain natural. There are two possible approaches to interface imperative code with dataflow code. One possible approach, adopted by current RVC-CAL application developments, is based on the interface driven by the dataflow code. The I/O imperative functions are called from the dataflow program using CAL procedures which are ad-hoc mechanisms put in place to accomodate legacy code in to dataflow components.

The approach proposed in this paper is also based on the use of imperative code for I/O. However in this approach it is the imperative code that calls the dataflow code with an input data to be processed. The main advantage of this approach is the ease of development. Each language is used to implement those parts of code for which the language is most appropriate for, without the need to accommodate the languages to each other (as in RVC-CAL procedures). This enables independent prototyping and reuse of code already written. It also permits legacy libraries to access dataflow components and helps adoption of dataflow programing.

A. Interface Definition

The interface is designed to be generic enough such that any new dataflow component, like a decoder, encoder or filter, can be added to any legacy video processing library. The proposed interface consists of three functions and a data structure which are explained and implemented as follows.

1) init component: This interfacing function is used to launch the dataflow component. Launching a dataflow component can be conceived as starting a conveyor belt system in a factory. Once a conveyor belt along with the processing units connected to it are started, a factory is ready to receive a stream of items to be processed. In our case, the initialization function starts the dataflow runtime system. The RVC-CAL runtime mainly consists of mapping, scheduling and other utility routines. The runtime takes in a set of actors, their network and user-supplied parameters such as an input data and maps, schedules and executes actors on a number of processing units. Mapping is either done by simply assigning actors to available processing units in a round-robin manner or via more complex post-profiling weight-based methods [START_REF] Yviquel | Towards run-time actor mapping of dynamic dataflow programs onto multi-core platforms[END_REF]. Once actors are mapped to a processing unit they are scheduled using round-robin or more advanced data-driven methods. [START_REF] Yviquel | Efficient multicore scheduling of dataflow process networks[END_REF]. The pseudo-code in Algorithm 1 shows the implementation of the init component function. context is any information that is needed to start the component. It contains the runtime options of the dataflow component such as mapping policy, scheduling policy, number of cores to be used and the dataflow network itself. Once the desired context of the component is set, the dataflow component is launched with a new thread which ensures the init component function returns control to the legacy code immediately. This allows the caller to continue its execution by sending row data and receiving processed data from the dataflow component.

2) process: This function is responsible for feeding the already initialized dataflow component with input data and grubbing the output data if available. Every call to this function from the legacy transcoder might fill the input FIFO of the dataflow component or get tokens from the output FIFO of the component, or both.

As shown in Algorithm 2, the process function takes in the context of the dataflow component which contains the network information and the data to be processed in ipkt. It then returns any result that might be available from the dataflow component in opkt. Note that this function also returns the size of the data consumed by the component through the variable sent. As can be noted from Algorithm 4, the component definition allows multiple dataflow components to be identified by a name or id. It also contains pointer to the three functions that are used to initiate, use and close a given dataflow component.

B. Generating Interface and Dataflow Component

In order to have an automated workflow for integrating dataflow components into a transcoding framework, we propose generating the interface automatically from the ORCC backend.

As shown in figure 3 we have modified the ORCC C backend to generate the dataflow components as a library along with a header file instead of stand alone executable. In addition we have added generation of package configuration files so that the library can be installed and be used easily.

C. Using the Interface

In order to demonstrate the functionality of the approach we have generated MPEG and HEVC video decoders form the corresponding dataflow descriptions, written in RVC-CAL, using our modified C backend. Figure 4 shows the structure of the HEVC dataflow decoder. The generated dataflow component libraries are then compiled and installed in our system. Following that we have included the header files of the generated dataflow components in our legacy video transcoding framework and linked to the installed component libraries during its compilation.

Algorithm 5 shows the use of a dataflow decoder component by the legacy video transcoder FFmpeg. Three main points can be noted from the pseudo-code. The first is the register_all function that is used to register the available formats, codec and filters in a given system. This function is from FFmpeg libraries and we have also used it to register our new dataflow decoder component. The second point to note is the use of our interface which constitutes the three functions, init component, process and close component. This interface can be used to abstract the various types of dataflow components that can be implemented and integrated to FFmpeg or any other legacy transcoder. Finally one can note that the FFmpeg libraries supply the I/O (read/write) functions which are capable of parsing almost any known video container format efficiently. In addition to the I/O functions, data processing functionalities such as video scalers and encoders that are yet to be implemented by dataflow approach can also be used.

Using our interface we were therefore able to provide FFmpeg, a legacy video transcoding framework with dataflow components that implement fine-grained parallelism. Note from figure 4

V. EXPERIMENTS

In order to check the proper operation and benefits of our integration approach2 we have made two evaluations which include quality and scalability measurements. In all of our experiments we used four different input videos from different categories with resolution of 1080p, bitrate of 1200kbps and frame rate of either 24 or 30 or 50. The transcoding operation performed in the experiments are resolution reduction transcoding. See result tables I and II for details on the input video characteristics and the transcoding parameters.

A. Quality

First, we made a quality difference measure between video transcoding operations which use the dataflow decoder and the original legacy decoders that comes with the FFmpeg transcoder. The original videos were of 1080p resolution and were transcoded to 240p, 480p or 720p. The quality difference were measured using three metrics, psnr,ssim, msssim [START_REF] Wang | Image quality assessment: from error visibility to structural similarity[END_REF], [START_REF] Vqmt | vqmt[END_REF]. The visual similarity matrices ssim, msssim are close to 1.00 and the structural similarity metrics psnr is inf for all tests indicating that the resulting videos are similar. This means our proposed interfacing works properly.

B. Evaluation of Scalability

Besides showing the proposed interface works correctly we here present scalability measurements on a 6 core Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz Machine. Table I shows the speed up of 62-80% when using 4 cores for a dataflow component (MPEG4 part 2 simple profile decoder) against a single core. Similarly Table II shows the speed up of 48-65% when using 4 cores for the dataflow component (HEVC simple profile decoder) against using a single core. The remaining 2 cores were assigned to the rest of the transcoding framework components such as the encoder. The speed up is calculated as (f ps4-f ps1) f ps1 * 100 where f ps 4 is the transcoding speed in frames per second (fps) when using 4 cores for the dataflow component and f ps 1 is the transcoding speed in fps when using 1 core for the dataflow component. Each mesurment is repeated 10 times to calculate the confidence intervals.

VI. CONCLUSION

In this paper we have proposed the use of a generic interface for integrating dataflow components such as decoders, encoders and filters with legacy transcoding libraries. The interface enables seamless interaction between dataflow and legacy imperative code allowing each programming approach to implement components for which it is appropriate for.

We have also tested and shown the proper functionality of our approach. Scalability evaluations also show the gain that can be obtained from using dataflow components via the proposed interface.

In the future we would like to further explore the effect of the dataflow component runtime and transcoding framework runtime on each other.

Fig. 1 .

 1 Fig. 1. Basic description of video transcoding. Demuxer and Muxer are I/O components used to read/write compressed video and the rest are video processing components which encode, decode and scale video.

1 :

 1 Data: context Result: success 1 set component context(context); 2 success = thread create(launcher, launch, context, tid); Algorithm Dataflow component initialization

3 : 2 const 9 }

 329 2 success = free component context(context); Algorithm Dataflow component termination Algorithm 3 shows the close component function. 4) Component Structure: This structure definition enables the use of multiple dataflow components and ensures the generic nature of the interface. 1 typedef struct component { init component)(context *); 7 int (*process)(context *, opkt*, ipkt*, *got result); 8 int (*close component)(context *); Algorithm 4: Component Structure

Fig. 3 .

 3 Fig. 3. Generation of the dataflow component library and its interface

Fig. 4 .

 4 Fig. 4. Dataflow Decoder Implementation from ORCC

  Complete picture of video transcoding showing the main blocks it constitutes and the different options of doing transcoding depending on the required type of conversion operation, acceptable quality and complexity

		Decoder
	Demux	Entropy Decoder	Inverse Quant.	Inverse Transform	+	Prediction	Frame Buffer
							Spacial and
							Temporal
							Processing
	stream	Prediction		Spatial/	Spatial/
	copy	Refinment		Temporal Processing	Temporal processing
						Prediction
						(ME)
	Mux	Entropy Coding	Quant.	Transform	-	Prediction (MC)	Frame Buffer
		Encoder	Inverse Quant.	Inverse Transform	+
	Fig. 2.

1 see http://orcc.sourceforge.net/

  Any unconsumed data on the input FIFO of the dataflow component should be re-supplied to this function. got result is used to tell a calling function if the dataflow component resulted in a valid output via opkt. This interface function is used to end the already running dataflow component. More specifically it ends the runtime of the component by joining all created threads that were responsible for executing the component's actors.

		Data: context, ipkt
		Result: sent, opkt, got result
	1 tosend=ipkt.size;
	2 sent = 0;
		/ * send input to dataflow	* /
	3 sent += send(context,ipkt, tosend);
	4 last processed=processed;
		/ * recive processed data	* /
	5 processed += recv(context,opkt);
	6 if last processed < processed then
		/ * we have got data	* /
	7	got result = 1;
	8 else
	9	got result = 0;
	10 end
	11 ipkt.size -= sent;
	12 ipkt.data += sent;
		Algorithm 2: Dataflow processing
		3) close component: Data: context
		Result: success
	1 thread join(launcher);

  that the dataflow decoder component provides a Data: v s (source video), context Result: v p (processed video)

	1 register all();
	2 init component(context) ;	// initialize
		component e.g. decoder/filter/encoder
	3 while read(context, v s , ipkt) do
	4	process(context, opkt, ipkt, got result) ; // e.g.
		decode/filter/encoder
	5	if got result then
	6	rescale frame (context, fpkt, opkt, got result);
	7	end
	8	if got result then
	9	encode frame (context, opkt, fpkt, got result);
	10	end
	11	if got result then
	12	write(context, opkt);
	13	end
	14	ipkt.data += ret;
	15	ipkt.size -= ret;
	16 end
	17 flush();
	18 close component(context) ; // close component
		e.g. decoder/filter/encoder
		Algorithm 5: FFmpeg overview
	fine grained parallelism by implementing functional blocks as
	separate actors.

TABLE I

 I

	.	EXPERIMENTAL RESULTS SHOWING THE FRAME RATE (FPS)
	OF THE DIFFERENT VIDEO STREAMS AND THE CONFIDENCE INTERVAL FOR
	A 97.5% CONFIDENCE LEVEL. THE SPEEDUP FROM RUNNING THE
	DATAFLOW COMPONENT OF THE TRANSCODER WITH A SINGLE CORE TO
	RUNNING IT IN FOUR CORES IS GIVEN FOR EACH RESOLUTION AND VIDEO
			SEQUENCE.
		240p	480p	720p
	Cartoon -Elephant Dreams -24fps
	1	3.456 ± 0.036	3.319 ± 0.027	3.075 ± 0.039
	2	4.808 ± 0.011	4.529 ± 0.051	4.092 ± 0.027
	4	6.085 ± 0.172	5.422 ± 0.537	4.995 ± 0.074
	speedup	76.07%	63.36%	62.44%
	Consumer Video -Old Town Cross -50fps
	1	3.432 ± 0.030	3.302 ± 0.029	3.065 ± 0.036
	2	4.805 ± 0.012	4.535 ± 0.017	4.106 ± 0.011
	4	6.180 ± 0.066	5.742 ± 0.050	4.996 ± 0.172
	speedup	80.07%	73.89%	63.00%
	Documentary -Snow mountain -30 fps
	1	3.487 ± 0.010	3.386 ± 0.040	3.191 ± 0.025
	2	4.822 ± 0.028	4.607 ± 0.047	4.223 ± 0.020
	G4	6.064 ± 0.380	5.863 ± 0.071	5.171 ± 0.292
	speedup	73.90%	73.15%	62.05%
	Sport -Touchdown Pass -30 fps
	1	3.373 ± 0.040	3.241 ± 0.031	2.975 ± 0.033
	2	4.528 ± 0.382	4.383 ± 0.073	3.917 ± 0.022
	4	5.508 ± 0.764	5.478 ± 0.250	4.519 ± 0.583
	speedup	63.3%	69.02%	51.9%

TABLE II .

 II EXPERIMENTAL RESULTS SHOWING THE FRAME RATE (FPS) OF THE DIFFERENT VIDEO STREAMS AND THE CONFIDENCE INTERVAL FOR A 97.5% CONFIDENCE LEVEL. THE SPEEDUP FROM RUNNING THE DATAFLOW COMPONENT OF THE TRANSCODER WITH A SINGLE CORE TO RUNNING IT IN FOUR CORES IS GIVEN FOR EACH RESOLUTION AND VIDEO SEQUENCE.

		240p	480p	720p
	Cartoon -Elephant Dreams -24fps
	1	1.474 ± 0.001	1.473 ± 0.001	1.467 ± 0.002
	2	1.800 ± 0.001	1.797 ± 0.003	1.791 ± 0.003
	4	2.193 ± 0.089	2.185 ± 0.144	2.233 ± 0.033
	speedup	48.78%	48.34%	52.22%
	Consumer Video -Old Town Cross -50fps
	1	1.396 ± 0.001	1.394 ± 0.002	1.391 ± 0.001
	2	1.729 ± 0.003	1.729 ± 0.004	1.723 ± 0.003
	4	2.176 ± 0.025	2.136 ± 0.119	2.144 ± 0.084
	speedup	55.87%	53.23%	54.13%
	Documentary -Snow mountain -30fps
	1	1.493 ± 0.002	1.489 ± 0.003	1.482 ± 0.002
	2	1.916 ± 0.011	1.916 ± 0.006	1.905 ± 0.001
	4	2.461 ± 0.065	2.460 ± 0.044	2.273 ± 0.164
	speedup	64.84%	65.21%	53.37%
	Sport -Touchdown Pass -30fps
	1	1.307 ± 0.001	1.304 ± 0.001	1.299 ± 0.001
	2	1.356 ± 0.673	1.595 ± 0.002	1.588 ± 0.003
	4	1.993 ± 0.054	2.009 ± 0.023	2.004 ± 0.009
	speedup	52.49%	54.06%	54.27%

https://github.com/tdeneke/ffmpeg-2.5.3, https://github.com/tdeneke/orcc and https://github.com/tdeneke/orc-apps