Vincent Aranega

Anne Etien

Sebastien Mosser
email: sebastien.mosser@sintef.no

Using Feature Model to build Model Transformation Chains

Model transformations are intrinsically related to modeldriven engineering. According to the increasing size of standardised metamodel, large transformations need to be developed to cover them. Several approaches promote separation of concerns in this context, that is, the definition of small transformations in order to master the overall complexity. Unfortunately, the decomposition of transformations into smaller ones raises new issues: organising the increasing number of transformations and ensuring their composition (i.e. the chaining). In this paper, we propose to use feature models to classify model transformations dedicated to a given business domain. Based on this feature models, automated techniques are used to support the designer, according to two axis: (i) the definition of a valid set of model transformations and (ii) the generation of an executable chain of model transformation that accurately implement designer's intention. This approach is validated on Gaspard2, a tool dedicated to the design of embedded system.

Introduction

Model-Driven Engineering (MDE) advocates the principle of separation of concerns, through the extensive use of models in all the steps of the software development cycle [START_REF] Miller | MDA Guide Version 1.0.1[END_REF][START_REF] Selic | The Pragmatics of Model-Driven Development[END_REF]. In this context, model transformations are used to achieve integration of concerns [START_REF] Ossher | Software engineering tools and environments: a roadmap[END_REF][START_REF] Douglas | Guest Editor's Introduction: Model-Driven Engineering[END_REF][START_REF] Bézivin | Towards a Precise Definition of the OMG/MDA Framework[END_REF]. Considering the intrinsic complexity of the meta-models in use (e.g., UML 2.x and its profiles), large model transformations (up to ten thousands lines of code) are developed. Such transformations have substantial drawbacks [START_REF] Pilgrim | Constructing and Visualizing Transformation Chains[END_REF], including limited reusability, reduced scalability, poor separation of concerns, limited learnability, and undesirable sensitivity to changes. The separation of concerns paradigm advocates the decomposition of a complex system (e.g., architectures, object-oriented models) into smaller artefacts. Thus, exactly as other artefacts, it is desirable to decompose transformations [START_REF] Vanhooff | A Framework for Transformation Chain Development Processes[END_REF]. Other researches have also argued that focusing on such an engineering of transformations improves the uptake of MDE [START_REF] Wagelaar | Module Superimposition: a Composition Technique for Rule-based Model Transformation Languages[END_REF]. It is then essential to support the systematic definition of small model transformations with a unique intention [START_REF] Cordy | Eating our own Dog Food: DSLs for Generative and Transformational Engineering[END_REF], to improve scalability, maintainability and reusability of transformations. Such an approach leads to the definition of a family of transformations associated to a given domain that jointly enable to generate systems from a business domain.

The existence of small transformations raises two new issues. First, the chain designer (called end user in the remainder of the paper) is in presence of a family of model transformations, which needs to be organised. Secondly, the reification of the dependencies that exist between elements of this family becomes critical. As model transformations cannot be chained anyhow, dependencies that lead to valid transformation chains must be captured. One way to automate this development process is to use a Software Product Line (SPL) approach. In a SPL, multiple products are derived by combining a set of different core assets. One of the most important challenges of SPL engineering concerns variability management, i.e., how to describe, manage and implement the commonalities and variabilities existing among the members of the same family of products. A well-known approach to variability modelling is by means of Feature Diagrams (FD) introduced as part of Feature Oriented Domain Analysis [START_REF] Kyo | Feature-Oriented Domain Analysis[END_REF] back in 1990.

Our contribution is to accurately combine model transformations and SPL to support the end user while developing transformation-based applications. Business experts' knowledge is reified in a FD to accurately organise the different transformations according to their intentions. Then, automated code analysis techniques are used to accurately generate constraints between these transformations 1 , reified in the feature model as requirements between features. Thus, it is possible for end users to use the FD to accurately define their own products, that is, a valid subset of transformations that matches their intentions. Product derivation mechanisms are then used to automatically generate the model transformation chain that implements what the end user asked for. The approach is validated using Gaspard2, a transformation-based tool that supports the modelling of embedded systems.

The remainder of this paper is organised as follows. In Section 2, we motivate this work by exposing the different challenges that need to be addressed in this domain. Then, Section 3 describes the approach we propose to tackle these challenges. Section 4 validates the approach by applying it to the Gaspard2 case study. Finally, Section 5 discusses the related works and Section 6 concludes this paper by exposing some research perspectives.

Motivation

In order to enhance reusability, variability, flexibility and verifications, Gas-pard2 [START_REF] Gamatié | A Model Driven Design Framework for Massively Parallel Embedded Systems[END_REF], a co-design environment dedicated to high performance embedded systems based on massively regular parallelism has been designed using Model Driven Engineering (MDE) technologies. Thus it enables the generation of VHDL, SystemC, OpenMP or Lustre code from a UML model enhanced with the Modelling and Analysis of Real Time Embedded systems (MARTE) profile. Each language is targeted using a chain composed of three to five dedicated transformations. These large transformations (up to 1500 lines of codes) were not reusable and hardly maintainable even by their own developers.

To introduce flexibility and reusability, the Gaspard2 environment has been re-engineered to rely on smaller transformations. Each transformation has a single intention such as memory management or scheduling and corresponds to 150 lines of code in average. Finally, 19 transformations including 4 model to text (M2T) transformations, and thus 15 model to model (M2M) transformations were defined. The number of chains that can be constructed from them is humongous. Let T = {τ 1 , . . . , τ n } a set of model to model transformations, and M = {µ 1 , . . . , µ m } a set of model to text transformations. We denote as N T ∪M the number of chains available in this context. The number of potential model to model chains is equal to the number of sequences without repetition that involve elements defined in T (denoted as P (k, n)). Secondly, There is (m + 1) potential targets for the previously defined sub-chain (as a transformation chain may not generate text). Finally, it is also possible to only generate text without involving other model transformation (thus, m chains).

N T ∪M = m + (m + 1) n k=1 P (k, n), P (k, n) = n! (n -k)!
N T ∪P is hardly computable generically. Nevertheless, a sub-optimal approximation is to consider N T ∪P bigger than (m + 1) times the highest term of the sum P (k, n) (i.e., P (n, n), that in our case is equals to 5 × 15!).

N T ∪P (m + 1) × P (n, n) = (m + 1) × n!, n = 15, m = 4, N T ∪P 6, 5 × 10 12
But only a few chains make sense! It becomes crucial to help the designer to built such chains. Thus, the definition of transformation libraries raises new issues such as (i) the representation of the transformations highlighting their purpose and the relationships between them; (ii) their appropriate selection according to the characteristics of the expected targeted system and (iii) their composition in a valid order. Traditionally, transformations are represented in chains or with their metamodels. Such representations are not adapted to the description of transformation libraries. In preparation for chaining the transformations, it seems indispensable to specify their purpose (i.e., what they handle), in addition to their associated metamodels. To generate systems with their own characteristics (e.g., management of distributed versus shared memory, optimised vs simple scheduling), transformations have to be consequently selected. Thus the end user has to select the transformations not only according to the characteristics of the resulting system she would like, but also to the relationships between the transformations. Manually performed, this selection may be tedious and error prone. From the selected transformations, several chains can be built. Transformations cannot be chained arbitrarily, some constraints must be fulfilled [START_REF] Etien | Combining Independent Model Transformations[END_REF][START_REF] Mens | Detecting Structural Refactoring Conflicts Using Critical Pair Analysis[END_REF]. If it is often simple to identify the first transformation of the chain (depending on the input metamodels) and the last one (that is a model to text transformation, if code has to be generated), establishing a valid order between the other selected transformations may be difficult. Indeed, existing approaches check if the proposed order is valid, but do not automatically provide a valid one.

In order to support the end user in the design of transformation chains, the following challenges have to be addressed: C 1 Propose to the end user a library in which each transformation can be easily identified according to the characteristics of the expected final system (Section 3.1). C 2 Help the end user to select transformations while automatically taking into account the relationships between transformations (Section 3.2). C 3 Automatically derive the transformation chain from the characteristics selected by the end user (Section 3.3).

Solution

To tackle the aforementioned challenges, we propose a feature-oriented approach and the associated too set to automatically generate accurate model transformation chains as depicted in Figure 1.

Feature Diagram This approach relies on three pillars: (i) the classification of the available transformations as a Feature Diagram (FD) produced by the business expert, (ii) the reification of requirement relationships between transformation (directly generate from the Transformations set by the Extraction Tool ) and (iii) the automated generation of transformation chains for a given product (using our Derivation Tool ) from features selected by the end user.

f 1 f 3 Fd f 2 f 4 f 7 f 5 f 6 f 6 f 3 f 2 f 5
The FD is designed once for all by the business expert as a prerequisite. It is nevertheless possible to modify it when new transformations and thus new features become available. The requirement relationships are expressed between the features and automatically computed from the transformation codes by the Extraction Tool we provide. The extracted relations enable to derive other features (and then the associated transformation) from the ones selected by the end user using a Configuration Tool (e.g., FeatureIDE2 ). The requirement relationships are also used by our Derivation Tool to order the selected features in order to design valid chains.

Structuring the Transformation Set as a Feature Diagram

As a transformation is used to support a given intention according to a business domain, a set of transformations implicitly model the variability of the different intentions associated to a domain. FD were defined to model such a variability, so its use is natural. We represent in Figure 2 an excerpt of the complete FD associated to Gaspard2. Using FD, features (represented as nodes) are classified among others according to constraints such as exclusiveness or optionality. Model transformations are bound to features as assets. A feature f holds a link to the actual model transformation to be used to implement the intention captured by f at run-time. Normally, each feature corresponds to a single transformation and vice versa. However, it occurs in practise that a single transformation may catch many intentions and thus corresponds to many features.

For example, in Figure 2(a), the FD models that a given product must contain a Scheduling feature, and may contain a Synchronisation feature. The features Graph and Polyhedron are exclusive, i.e., the use of one in a given product implies that the other cannot be used in this particular product. We call a product a set of features that respects the constraints modelled in the FD. For example, Figures 2(b) and 2(c) represent two products among the eight valid w.r.t. the modelled FD. The first one (Figure 2(b)) considers a system synchronised using a BlocByBloc method, and scheduled with a simple Graph. The second product (Figure 2(c)) considers a system synchronised with a Barrier method, and scheduled with a Polyhedron approach. In our context, features reify model transformations: the actual implementation of the transformation is bound as an asset of the associated feature node. Thus, considering a given product, it is possible to automatically infer the set of transformations involved in the transformation chain that supports it. Key Points. The role of the FD is to capture the business knowledge associated to a given set of transformations. It actually transforms a flat set of transformations into an organised family of products.This classification is done by the business expert, that is, someone who deeply knows the different transformations, their underlying intentions, as well as the artifact they are handling. The key idea here is that this work is done once by the business expert, and capitalised in the FD. Without the use of a FD to support such a classification, it would be up to the end users to guess how the different transformations cope with each others before assembling them.

Recovering Require Relationship from Transformations

On top of constraints expressing the mandatory/optional character of the features as well as the and/or relationships between them, "require" relationships can also be captured in FD. They enable to automatically deduced other features from the selected ones, independently of the tree structure of the FD. Require relationships can be determined manually by the business expert. However, when the number of features is huge, omission can happen leading to erroneous products determination. Therefore, we provide an automatic analysis of the transformations to recover the require relationships (bijection) between the features associated to them. A requirement between two features f and f (denoted as a logical implication, i.e., f ⇒ f ) means that the transformation bound to f requires the transformation bound to f . The following question is raised: "When does a require relationship between two transformations exists? ". In fact, it relies on the element type production and consumption. For two transformations τ and τ , if τ consumes types created by τ , then it implies that a require relationship exists between τ and τ , denoted as τ → τ (for τ requires τ ). For each transformation, it is thus mandatory to automatically determine the element types it produces and it consumes to provide an automatic require relationships determination.

This automatic analysis relies on the different actions performed on element types by a transformation. Four actions are classically performed by transformations: reading, creating, deleting and modifying. This analysis does not rely on transformation execution but on static code analysis. Thus an element of the input or the output metamodel of a transformation is considered read if the presence of one on its instance enables the application of a transformation rule. An element is considered created, if at least one of its instance can be created by the transformation, and so on. Thus, τ requires τ if τ reads some elements created by τ [START_REF] Etien | Chaining Model Transformations[END_REF]. As we stated in the previous section, for a feature f from the FD, it exists at most one transformation τ . So, considering two features f , f in the FD and two transformations τ , τ mapped to f , respectively f , if τ → τ , it implies that f ⇒ f . Key Points. The proposed generation of the require relationships relies on a static analysis of the transformation codes. Once the FD designed and the constraints generated, the end user can use a Configuration Tool to select the features she wants for her transformation chain. The Configuration Tool is parametrised by the feature diagram and the generated constraints. Thus, by taking into account the generated constraints, during the feature selection, the Configuration Tool can either invalidate features or add required features according to the ones already selected by the end user. The automatic characteristic of the generation enables a certain evolutivity of the FD.

Generating Transformation Chains

Based on the two previous parts of the contribution, it is now possible to (i) consider a set of model transformations as a product family and (ii) automatically infer the requirement relationships that exist inside the product family. These two contributions act at the level of the FD. According to the global process, the selected features are then passed to a Derivation Tool, which uses the generated constraints to propose transformation chains from the selected features.

We consider now a given product p = {f 1 , . . . , f n }, i.e., a subset of features that satisfies the constraints modelled in the FD. As stated in Section 3.1, model transformations are bound to features. It is then possible to obtain the set of model transformations associated to p (denoted as T p ) by mapping each feature to its associated transformation: T p = {τ 1 , . . . , τ m }. As some features are only used to structure the FD and are not related to any concrete transformation, it should be noted that the cardinality of T p may be lesser than the cardinality of p. But this set of transformations is not sufficient to properly derive a concrete transformation chain from a given product. The requirement constraints identified in Section 3.2 must be taken into account. Considering two features f and f , if the requirement f ⇒ f exists, then the transformation τ mapped to f must be executed before the transformation τ mapped to f . As a consequence, the analysis of the set of requirement constraints leads to the identification of sequences of model transformations. Two situations can be encountered. If the requirement constraints implement a total order on the set of transformations, only one sequence will be identified, i.e., the proper transformation chain to be executed to support the intentions captured by this product. But if the requirement constraints implement a partial order, only partial sequences can be identified automatically. But as there is no requirement between these different sub-sequences, their order is not important. Consequently several valid chains are generated. This "subchains approach" is useful to support the business expert while assessing the FM consistency. It also helps the non-expert end user to construct a valid chain: any of the chains built upon these sub-chains, will by essence respect the dependencies captured by the FM. Key Points. A concrete chain of model transformations is automatically derived, through the FD, from the transformation set selected by the end user. First, the knowledge of a business expert is captured in the FD, and then an automatic static analysis is used to properly extract technical constraints from the implementation of the transformations. Finally, it is possible to automatically derive the chain, through the systematic exploration of the identified constraints. As a consequence,the generation of the concrete model transformation chain is automated, and the end user does not require any knowledge of model transformation from a technical point of view.

Validation: The Gaspard2 Case Study

Gaspard2 is a co-design environment dedicated to high performance embedded systems based on massively regular parallelism. From high level specifications, it automatically generates code for high performance computing, hardwaresoftware co-simulation, functional verification or hardware synthesis using model transformations. Such generations are complex and require intermediary steps, e.g., the explicit mapping of application tasks onto processing units, the mapping of the data onto memories or the scheduling of the tasks. Each transformation has a specific intention and deals with few concepts. Nineteen transformations have been implemented for now but the framework may support even more of them in the upcoming months. It is difficult for a non expert user to easily understand the purpose of each transformation, to select the ones useful to reach the desired platform and finally to order them in order to compose a chain.

In this paper, we used the Familiar tool suite [START_REF] Acher | Reverse engineering architectural feature models[END_REF] to manipulate feature diagrams. This tool allows us to model FD, and is well integrated in the Eclipse platform. Thus, standard Configuration Tools (e.g., FeatureIDE) can be used to allow the end user to configure products. But it should be noted that the approach is not bound to this tool nor to this case study from a theoretical point of view, as described in the previous section.

4.1

Step #1: Capturing Business Expert Knowledge in a FD Embedded systems designers usually do not master model transformation paradigm and underlying technologies. It is then essential to support them while designing the transformation chains used to generate code from high level specifications. The design of these chains consists in the selection of relevant transformations available in a library and in the computation of a valid order. Selecting a transformation requires to easily distinguish one transformation from another and to quickly identify its intention. In order to help the embedded systems designers, we have classified the available transformations based on embedded characteristics using feature model. It is up to the business expert to find the most appropriate classification method to be used to support the end user. During the implementation of the case study, we applied an incremental definition of the FD. We produced 12 successive versions of the FD. Excepting from one deep refactoring to better handle business requirement constraint, the implementation of the FD by the business expert was straightforward.

Most transformations of Gaspard2 have a unique intention representing a specific characteristics of the produced systems such as memory management. The transformations and their associated intentions are listed in Table 1. The Gaspard2 transformation library counts 19 intentions through 15 M2M and 4 M2T transformations. For example, the scheduling transformation has the following intention: it manages a simple scheduling of application tasks on computing units. As a consequence of the non mandatory bijection between features and transformations, the barrier and the openMP features are implemented by a single transformation. This many-to-one (surjection) relationship corresponds to a lack of modularisation of the transformation.

From these intentions, the business expert builds the feature diagram by associating a feature to each intention. Moreover, some features are added in the hierarchy in order to specify the relationship AND/OR/XOR between features. Indeed, as stated in Section 3.1, each feature represents at most one transformation. The resulting FD, depicted in Figure 3, gathers, in an non exhaustive way, some characteristics that an embedded system produced by Gaspard2 may possess. For example, the OpenCL and OpenMP features, introduce a scientific computation intention. However, only one of these two features can be selected. Indeed, the target language is either OpenCL, or OpenMP. In the FD, this choice is designed by the introduction of an intermediary abstract node ScientificComputation and an alternative between the two features.

The associated tooling provided by the Familiar platform can be used to query the model, as shown in Figure 4. This FD models up to 200 different available configurations (obtained by the Familiar counting algorithm). The configs command computes all the available products, returning the set of valid products defined by this FD.

Step #2: Extracting Constraints from the Implementation

This feature model enables the classification and the distinction of the transformations the one from the others. However, in this primary form, it does not gather enough information to build the chains: some others may be required and the selection of one transformation may require the selection of others. Such dependencies between transformations have to be captured and the feature model tools enable to take them into account for the product configuration. Thanks to the Extraction Tool, the implementation of the available transformations is automatically analysed. The result of this analysis is a set of "require" constraints between the features modelled in the FD. We represent in Listing 1.1 the set of constraints obtained after the execution of the tool. These constraints are generated using the syntax of the Familiar tool, and thus can be automatically integrated in the FD. Contrarily to the initial FD that captures the knowledge of the business expert, these relations reify the implementation constraints that exist between the transformations, from a technical point of view. It then ensures that the products configured w.r.t. this FD will be valid at both level: (i) business domain and (ii) technical implementation. Considering this set of constraints, the Configuration Tool now proposes 37 available products to the end user (from 200 at the beginning). This highlights the fact that working with the implementation of the transformation is critical. The technical implementation of the transformations dramatically reduces the initial variability of the domain as it was designed by the business expert.

Taking into account the "real" features implementations in the FD (i.e, the transformations code in our context) through this set of automatically computed constraints also leads to interesting situations that help the business expert. We consider here the feature Repetition, defined as optional by the business expert (see Figure 3). The generated set of constraints identifies a requirement between the feature AbsoluteComputation and the feature Develop (line 1 in Listing 1.1). However, AbsoluteComputation is mandatory, and selecting Develop implies to select Repetition. Thus, the Repetition feature is automatically identified by the tool suite as a false optional feature, that is, a feature modelled as optional but enforced as mandatory by a requirement constraints. In this case, it helped the business expert to identify a missing artifact in the FD: it should also contain an alternative implementation for Repetition instead of only defining the Develop approach.

Step #3: Deriving Transformation Chains

Based on the FD enhanced with the implementation constraints, we can now ensure that the products configured by the end user through the configuration tool are valid. The final step is to use a derivation tool that properly builds the transformation chains associated to a given product. We consider here one of the 37 products available according to this FD, denoted as p corresponding for example to the set of the features selected by the end user. The first step is to translate p into T p , that is, the set of transformations involved by this product. It should be noted that |p| > |T p |, as several features are only used to structure the FD and consequently are not bound to concrete transformations. For example, for the following product, corresponds the associated T p : The second step is to map the constraints between features as a partial order among the transformations. The requirements involved in p are the following: Based on this partial order, it is possible to compute3 the following sets of "independent" sub-chains involved in this product, as a chain template, that is, a partition of the transformation set taking into account the partial order:

tplp = [ [openM P ], [explicitAllocation], [tilerM apping] (1)
[memM apping, [shape2loop, poly loop, tiler2task] ] ] (2)

Among the computed sub-chains, the openMP transformation is a "model to text" transformation and will always be the last one executed in the chain. In line 2, the partial order indicates that the memMapping transformation must be preceded by the 3 transformations listed, without specifying any order between them. Thus, there is up to 6 ways to combine these transformations according to this constraint. As the explicitAllocation and tilerMapping transformations can be executed independently of these sub-chains, they can be executed before, after or inside the previously described sub-chains. As a consequence, up to 180 chains can be obtained from this product. Following the sub-chains computed by our derivation tool, a valid transformation chain could be:

explicitAllocatlion → tiler2task → tilerM apping → • • • • • • poly loop → shape2loop → memM apping → openM P
Without any lead, the end user has only one constraint: the model to text transformation must be the last of the chain. From the product p and its associated set of transformations T p , it means that the end user has the choice to organise 6 transformations. Thus, she has P (6, 6) = 720 choices to organise the model to model transformations. Among the 720 chaining possibilities, many are not valid because the require relationships are not considered. So, without any indication, the end user has to choose from 720, potentially non valid, chains, whereas with our methodology, the choice is reduced to 180 valid chains only.

To sum up, our methodology and the associated tool have allowed the end user (without any knowledge about transformations) to easily build chains. She has selected transformations based on embedded system features i.e. using terms she is familiar with. Finally, she has to choose among 180 valid chains whereas initially she was confronted to a huge number of possible chains that she has to build by scrutinizing the transformation code.

State of the Art

In order to enhance the reusability of transformations, several authors promote the decomposition of transformations into smaller ones. However transformations have then to be chained. Vanhooff et al. proposed an approach based on the explicit and manual identification of the required and provided concepts by the chain developer for example using a profile in order to later build the chain [START_REF] Vanhooff | Breaking Up the Transformation Chain[END_REF]. Our approach relies on the feature model to compose the transformations.

Several approaches have been proposed to build chains. Transformations are considered as functions to compose if their domains are compliant [START_REF] Oldevik | Transformation Composition Modelling Framework[END_REF] or UML activity that can be chained using different operators: composition, conditional composition, parallel composition and loop [START_REF] José | Orchestrating ATL Model Transformations[END_REF]. However, in both cases, the transformation chain has to be manually specify by the designer, without any specific help. In the latter case, they are executed using the provided model transformation orchestration tool. Our approach could be used upstream to identify the useful transformations and to compose them.

Transformation chaining relies on constraints that can be automatically identified, e.g. using the distinction between concepts copied and those mutated [START_REF] Chenouard | Automatically Discovering Hidden Transformation Chaining Constraints[END_REF]. This approach only deals with endogenous transformations (even if a possible extension to heterogeneous transformations is suggested). With the "require" constraints, we have extended this approach to heterogenous transformations.

Several approaches propose to deal with the complexity of large systems with a feature-based approach. For example, feature models were accurately used to model the intrinsic variability of the Linux Kernel [START_REF] Lotufo | Evolution of the linux kernel variability model[END_REF], and support end-user during the kernel configuration task. The approach proposed in this paper follow the same idea, that is, the use of feature modelling to leverage a highly variable systems into an entity configurable by the end-user.

Being able to extract the features from the implementation is a challenge [START_REF] Acher | On extracting feature models from product descriptions[END_REF]. The most difficult part is the extraction of the feature hierarchy from the "flattened" implementation [START_REF] She | Reverse engineering feature models[END_REF]. Inferring such a hierarchy relies on domain heuristics that rank the possible hierarchy, and the final assessment of these ranks by a domain expert. In this paper, we do not consider the automatic extraction of the features from the transformation set, and only rely on the business expert to properly model the feature diagram. Being able to support the business expert during this task is an interesting perspective of this work.

Feature models are also used to support the reverse engineering of large scale systems [START_REF] Acher | Reverse engineering architectural feature models[END_REF]. For example, the FraSCAti platform (an open source implementation of the SCA standard) was accurately reverse-engineered to support its assessment. Based on a dedicated tool that extracts the architecture from the implementation, the authors confront the automatically extracted feature model with the one defined by the business expert. This approach complements ours, as we also rely on a tool that automatically infers feature information from the actual implementation of the system (in our case requirements between features). But instead of assessing the model defined by the business expert, we focused on its enrichment, by merging the set of automatically identified information in this feature model. We were able to identify several situations where the actual system was not "as variable" as the business expert thought.

Conclusions & Perspectives

In order to be reusable and maintainable, model transformations are written according to a single intention and complex transformations are built as the chaining of smallest ones. In this paper, we proposed an approach based on FD to support the design of model transformation chains. Based on a classification of the transformations made by a business expert, This approach allows an end user to build such chains, without any prior knowledge of model transformation technologies. The implementation of the transformations is also automatically taken into account to ensure that the built chains are valid from a run-time point of view. From an implementation point of view, the approach is independent of any tools and can be easily coupled to existing approaches (e.g., FeatureIDE, Familiar). The approach was validated on the Gaspard2 case study, and we are currently pursuing another validation study in the domain of website engineering.

The resulting chains are valid according to a type based approach [START_REF] Etien | Combining Independent Model Transformations[END_REF]. However, two transformations that can be chained into both orders from a syntactic perspective are not obviously commutable from a business point of view: the execution of the two successive transformations on whatever models may not always lead to the same result. A perspective of this work is to enhance the expressiveness of the requirement detection mechanisms to address this issue. Another perspective concerns the FD refinement. Indeed, the FD being manually designed by the business expert, some constraints between features may have been omitted. The automatic requirement relationships extraction could be a first help to highlight a badly / incompletely designed FD. To help the business expert in the definition or the refinement of the FD, we plan to automatically extract features from the documentation written by the transformation developers.

Fig. 1 .

 1 Fig. 1. Approach Process Overview

Fig. 2 .

 2 Fig. 2. Gaspard2: Feature diagram and associated products (using FeatureIDE)

Fig. 3 . 1 .Fig. 4 .

 314 Fig. 3. Feature model associated to Gaspard2.

1 AbsoluteComputationListing 1 . 1 .

 111 Set of requirement constraints.

p

  = {Gaspard, M emoryT ype, P olyhedron, Data, Barrier, M appingM gmt, KeepHierarchy, Hierarchy, T iler, Develop, StaticScheduling, AbsoluteComputation, T ask, Explicit, Scientif icComputation, Scheduling, Objective, Repetition, Synchronisation, OpenM P } Tp = {explicitAllocation, memM apping, openM P, poly loop, shape2loop, tilerM apping, tiler2task}

Informally, a transformation τ requires a transformation τ if the model elements handled by τ are produced by τ .

http://wwwiti.cs.uni-magdeburg.de/iti db/research/featureide/

We used a set of logical predicates implemented using he Prolog language to implement the Derivation Tool.


Jean-Pierre Briot
email: jean-pierre.briot@lip6.fr

François Pachet
email: francois@spotify.com

Keywords: convolutional networks, recurrent networks, autoencoders, restricted Boltzmann machines, etc. [GBC16]

In addition to traditional tasks such as prediction, classification and translation, deep learning is receiving growing attention as an approach for music generation, as witnessed by recent research groups such as Magenta at Google and CTRL (Creator Technology Research Lab) at Spotify. The motivation is in using the capacity of deep learning architectures and training techniques to automatically learn musical styles from arbitrary musical corpora and then to generate samples from the estimated distribution. However, a direct application of deep learning to generate content rapidly reaches limits as the generated content tends to mimic the training set without exhibiting true creativity. Moreover, deep learning architectures do not offer direct ways for controlling generation (e.g., imposing some tonality or other arbitrary constraints). Furthermore, deep learning architectures alone are autistic automata which generate music autonomously without human user interaction, far from the objective of interactively assisting musicians to compose and refine music. Issues such as: control, structure, creativity and interactivity are the focus of our analysis. In this paper, we select some limitations of a direct application of deep learning to music generation, analyze why the issues are not fulfilled and how to address them by possible approaches. Various examples of recent systems are cited as examples of promising directions.

Music Generation by Deep Learning

-Challenges and Directions * 1 Introduction

Deep Learning

Deep learning has become a fast growing domain and is now used routinely for classification and prediction tasks, such as image and voice recognition, as well as translation. It emerged about 10 years ago, when a deep learning architecture significantly outperformed standard techniques using handcrafted features on an image classification task [START_REF] Hinton | A fast learning algorithm for deep belief nets[END_REF]. We may explain this success and reemergence of artificial neural networks architectures and techniques by the combination of:

1. technical progress, such as: convolutions, which provide motif translation invariance [START_REF] Le | Convolutional networks for images, speech, and time-series[END_REF], and LSTM (Long Short-Term Memory), which resolved inefficient training of recurrent neural networks [START_REF] Hochreiter | Long short-term memory[END_REF];

2. availability of multiple data sets;

3. availability of efficient and cheap computing power, e.g., offered by graphics processing units (GPU).

There is no consensual definition of deep learning. It is a repertoire of machine learning (ML) techniques, based on artificial neural networks 1 . The common ground is the term deep, which means that there are multiple layers processing multiple levels of abstractions, which are automatically extracted from data, as a way to express complex representations in terms of simpler representations.

Main applications of deep learning are within the two traditional machine learning tasks of classification and prediction, as a testimony of the initial DNA of neural networks: logistic regression and linear regression. But a growing area of application of deep learning techniques is the generation of content: text, images, and music, the focus of this article.

Deep Learning for Music Generation

The motivation for using deep learning, and more generally machine learning techniques, to generate musical content is its generality. As opposed to handcrafted models for, e.g., grammar-based [START_REF] Steedman | A generative grammar for Jazz chord sequences[END_REF] or rule-based music generation systems [START_REF] Ebcioglu | An expert system for harmonizing four-part chorales[END_REF], a machine-learning-based generation system can automatically learn a model, a style, from an arbitrary corpus of music. Generation can then take place by using prediction (e.g., to predict the pitch of the next note of a melody) or classification (e.g., to recognize the chord corresponding to a melody), based on the distribution and correlations learnt by the deep model which represent the style of the corpus.

As stated by Fiebrink and Caramiaux in [START_REF] Fiebrink | The machine learning algorithm as creative musical tool[END_REF], benefits are: 1) it can make creation feasible when the desired application is too complex to be described by analytical formulations or manual brute force design; 2) learning algorithms are often less brittle than manually-designed rule sets and learned rules are more likely to generalize accurately to new contexts in which inputs may change.

Challenges

A direct application of deep learning architectures and techniques to generation, although it could produce impressing results2 , suffers from some limitations. We consider here3 :

• Control, e.g., tonality conformance, maximum number of repeated notes, rhythm, etc.;

• Structure, versus wandering music without a sense of direction;

• Creativity, versus imitation and risk of plagiarism;

• Interactivity, versus automated single-step generation.

Related Work

A comprehensive survey and analysis by Briot et al. of deep learning techniques to generate musical content is available in a book [START_REF] Briot | Deep Learning Techniques for Music Generation[END_REF]. In [START_REF] Herremans | A functional taxonomy of music generation systems[END_REF], Herremans et al. propose a function-oriented taxonomy for various kinds of music generation systems. Examples of surveys about of AI-based methods for algorithmic music composition are by Papadopoulos and Wiggins [START_REF] Papadopoulos | AI methods for algorithmic composition: A survey, a critical view and future prospects[END_REF] and by Fernández and Vico [START_REF] David | AI methods in algorithmic composition: A comprehensive survey[END_REF], as well as books by Cope [START_REF] Cope | The Algorithmic Composer[END_REF] and by Nierhaus [START_REF] Nierhaus | Algorithmic Composition: Paradigms of Automated Music Generation[END_REF]. In [START_REF] Graves | Generating sequences with recurrent neural networks[END_REF], Graves analyses the application of recurrent neural networks architectures to generate sequences (text and music). In [START_REF] Fiebrink | The machine learning algorithm as creative musical tool[END_REF], Fiebrink and Caramiaux address the issue of using machine learning to generate creative music. We are not aware of a comprehensive analysis dedicated to deep learning (and artificial neural networks techniques) that systematically analyzes limitations and challenges, solutions and directions, in other words that is problem-oriented and not just application-oriented.

Organization

The article is organized as follows. Section 1 (this section) introduces the general context of deep learning-based music generation and lists some important challenges. It also includes a comparison to some related work. The following sections analyze each challenge and some solutions, while illustrating through examples of actual systems: control/section 2, structure/section 3, creativity/section 4 and interactivity/section 5.

Control

Musicians usually want to adapt ideas and patterns borrowed from other contexts to their own objective, e.g., transposition to another key, minimizing the number of notes. In practice this means the ability to control generation by a deep learning architecture.

Dimensions of control strategies

Such arbitrary control is actually a difficult issue for current deep learning architectures and techniques, because standard neural networks are not designed to be controlled. As opposed to Markov models which have an operational model where one can attach constraints onto their internal operational structure in order to control the generation4 , neural networks do not offer such an operational entry point. Moreover, the distributed nature of their representation does not provide a direct correspondence to the structure of the content generated. As a result, strategies for controlling deep learning generation that we will analyze have to rely on some external intervention at various entry points (hooks), such as:

• Input;

• Output;

• Encapsulation/reformulation.

Sampling

Sampling a model5 to generate content may be an entry point for control if we introduce constraints on the output generation (this is called constraint sampling). This is usually implemented by a generate-and-test approach, where valid solutions are picked from a set of generated random samples from the model6 . As we will see, a key issue is how to guide the sampling process in order to fulfill the objectives (constraints), thus sampling will be often combined with other strategies.

Conditioning

The strategy of conditioning (sometimes also named conditional architecture) is to condition the architecture on some extra conditioning information, which could be arbitrary, e.g., a class label or data from other modalities. Examples are:

• a bass line or a beat structure, in the rhythm generation system [MKPKK17];

• a chord progression, in the MidiNet architecture [START_REF] Yang | MidiNet: A convolutional generative adversarial network for symbolic-domain music generation[END_REF];

• a musical genre or an instrument, in the WaveNet architecture [vdODZ + 16];

• a set of positional constraints, in the Anticipation-RNN architecture [START_REF] Hadjeres | Interactive music generation with positional constraints using Anticipation-RNN[END_REF].

In practice, the conditioning information is usually fed into the architecture as an additional input layer. Conditioning is a way to have some degree of parameterized control over the generation process.

Example 1: WaveNet Audio Speech and Music Generation

The WaveNet architecture by van der Oord et al. [vdODZ + 16] is aimed at generating raw audio waveforms. The architecture is based on a convolutional feedforward network without pooling layer7 . It has been experimented on generation for three audio domains: multi-speaker, text-to-speech (TTS) and music.

The WaveNet architecture uses conditioning as a way to guide the generation, by adding an additional tag as a conditioning input. Two options are considered: global conditioning or local conditioning, depending if the conditioning input is shared for all time steps or is specific to each time step.

An example of application of conditioning WaveNet for a text-to-speech application domain is to feed linguistic features (e.g., North American English or Mandarin Chinese speakers) in order to generate speech with a better prosody. The authors also report preliminary experiments on conditioning music models to generate music given a set of tags specifying, e.g., genre or instruments.

Example 2: Anticipation-RNN Bach Melody Generation

Hadjeres and Nielsen propose a system named Anticipation-RNN [START_REF] Hadjeres | Interactive music generation with positional constraints using Anticipation-RNN[END_REF] for generating melodies with unary constraints on notes (to enforce a given note at a given time position to have a given value). The limitation when using a standard note-to-note iterative strategy for generation by a recurrent network is that enforcing the constraint at a certain time step may retrospectively invalidate the distribution of the previously generated items, as shown in [START_REF] Pachet | Finite-length markov processes with constraints[END_REF]. The idea is to condition the recurrent network (RNN) on some information summarizing the set of further (in time) constraints as a way to anticipate oncoming constraints, in order to generate notes with a correct distribution.

Therefore, a second RNN architecture8 , named Constraint-RNN, is used and it functions backward in time and ts outputs are used as additional inputs of the main RNN (named Token-RNN), resulting in the architecture shown at Figure 1, with:

• c i is a positional constraint;
• o i is the output at index i (after i iterations) of Constraint-RNN -it summarizes constraint informations from step i to final step (end of the sequence) N . It will be concatenated (⊕) to input s i-1 of Token-RNN in order to predict next item s i .

The architecture has been tested on a corpus of melodies taken from J. S. Bach chorales. Three examples of melodies generated with the same set of positional constraints (indicated with notes in green within a rectangle) are shown at Figure 2. The model is indeed able to anticipate each positional constraint by adjusting its direction towards the target (lower-pitched or higher-pitched note).

Input Manipulation

The strategy of input manipulation has been pioneered for images by DeepDream [START_REF] Mordvintsev | Deep Dream[END_REF]. The idea is that the initial input content, or a brand new (randomly generated) input content, is incrementally manipulated in order to match a target property. Note that control of the generation is indirect, as it is not being applied to the output but to the input, before generation. Examples are:

• maximizing the activation of a specific unit, to exaggerate some visual element specific to this unit, in DeepDream [START_REF] Mordvintsev | Deep Dream[END_REF];

• maximizing the similarity to a given target, to create a consonant melody, in DeepHear [Sun17];

• maximizing both the content similarity to some initial image and the style similarity to a reference style image, to perform style transfer [GEB15];

• maximizing the similarity of structure to some reference music, to perform style imposition [START_REF] Lattner | Imposing higher-level structure in polyphonic music generation using convolutional restricted Boltzmann machines and constraints[END_REF].

Interestingly, this is done by reusing standard training mechanisms, namely back-propagation to compute the gradients, as well as gradient descent to minimize the cost. The DeepHear architecture by Sun [START_REF] Sun | DeepHear -Composing and harmonizing music with neural networks[END_REF] is aimed at generating ragtime jazz melodies. The architecture is a 4-layer stacked autoencoders (that is 4 hierarchically nested autoencoders), with a decreasing number of hidden units, down to 16 units.

At first, the model is trained9 on a corpus of 600 measures of Scott Joplin's ragtime music, split into 4measure long segments. Generation is performed by inputing random data as the seed into the 16 bottleneck hidden layer units and then by feedforwarding it into the chain of decoders to produce an output (in the same 4-measure long format of the training examples), as shown at Figure 3.

In addition to the generation of new melodies, DeepHear is used with a different objective: to harmonize a melody, while using the same architecture as well as what has already been learnt10 . The idea is to find a label instance of the set of features i.e. a set of values for the 16 units of the bottleneck hidden layer of the stacked autoencoders which will result in some decoded output matching as much as possible a given melody. A simple distance function is defined to represent the dissimilarity between two melodies (in practice, the number of not matched notes). Then a gradient descent is conducted onto the variables of the embedding, guided by the gradients corresponding to the distance function until finding a sufficiently similar decoded melody. Although this is not a real counterpoint but rather the generation of a similar (consonant) melody, the results do produce some naive counterpoint with a ragtime flavor.

Example 2: VRAE Video Game Melody Generation

Note that input manipulation of the hidden layer units of an autoencoder (or stacked autoencoders) bears some analogy with variational autoencoders11 , such as for instance the VRAE (Variational Recurrent Auto-Encoder) architecture of Fabius and van Amersfoort [START_REF] Fabius | Variational Recurrent Auto-Encoders[END_REF]. Indeed in both cases, there is some exploration of possible values for the hidden units (latent variables) in order to generate variations of musical content by the decoder (or the chain of decoders). The important difference is that in the case of variational autoencoders, the exploration of values is user-directed, although it could be guided by some principle, for example an interpolation to create a medley of two songs, or the addition or subtraction of an attribute vector capturing a given characteristic (e.g., high density of notes as in Figure 4). In the case of input manipulation, the exploration of values is automatically guided by the gradient following mechanism, the user having priorly specified a cost function to be minimized or an objective to be maximized.

Example 3: Image and Audio Style Transfer

Style transfer has been pioneered by Gatys et al. [START_REF] Gatys | A neural algorithm of artistic style[END_REF] for images. The idea, summarized at Figure 5, is to use a deep learning architecture to independently capture:

• the features of a first image (named the content), • and then, to use gradient following to guide the incremental modification of an initially random third image, with the double objective of matching both the content and the style descriptions12 .

Transposing this style transfer technique to music was a natural direction and it has been experimented independently for audio, e.g., in [START_REF] Ulyanov | Audio texture synthesis and style transfer[END_REF] and [START_REF] Foote | Audio style transfer -Do androids dream of electric beats?[END_REF], both using a spectrogram (and not a direct wave signal) as input. The result is effective, but not as interesting as in the case of painting style transfer, being somehow more similar to a sound merging of the style and of the content. We believe that this is because of the anisotropy13 of global music content representation.

Example 4: C-RBM Mozart Sonata Generation

The C-RBM architecture proposed by Lattner et al. [START_REF] Lattner | Imposing higher-level structure in polyphonic music generation using convolutional restricted Boltzmann machines and constraints[END_REF] uses a restricted Boltzmann machine (RBM) to learn the local structure, seen as the musical texture, of a corpus of musical pieces (in practice, Mozart sonatas). The architecture is convolutional (only) on the time dimension, in order to model temporally invariant motives, but not pitch invariant motives which would break the notion of tonality. The main idea is in imposing by constraints onto the generated piece some more global structure (form, e.g., AABA, as well as tonality), seen as a structural template inspired from the reference of an existing musical piece. This is called structure imposition14 , also coined as templagiarism (short for template plagiarism) by Hofstadter [START_REF] Hofstadter | Staring Emmy straight in the eye-and doing my best not to flinch[END_REF].

Generation is done by sampling from the RBM with three types of constraints:

• Self-similarity, to specify a global structure (e.g., AABA) in the generated music piece. This is modeled by minimizing the distance between the self-similarity matrices of the reference target and of the intermediate solution;

• Tonality constraint, to specify a key (tonality). To estimate the key in a given temporal window, the distribution of pitch classes is compared with the key profiles of the reference;

• Meter constraint, to impose a specific meter (also named a time signature, e.g., 4/4) and its related rhythmic pattern (e.g., accent on the third beat). The relative occurrence of note onsets within a measure is constrained to follow that of the reference. Generation is performed via constrained sampling, a mechanism to restrict the set of possible solutions in the sampling process according to some pre-defined constraints. The principle of the process (illustrated at Figure 6) is as follows. At first, a sample is randomly initialized, following the standard uniform distribution. A step of constrained sampling is composed of n runs of gradient descent to impose the high-level structure, followed by p runs of selective Gibbs sampling to selectively realign the sample onto the learnt distribution. A simulated annealing algorithm is applied in order to decrease exploration in relation to a decrease of variance over solutions.

Results are quite convincing. However, as discussed by the authors, their approach is not exact, as for instance by the Markov constraints approach proposed in [START_REF] Pachet | Finite-length markov processes with constraints[END_REF].

Reinforcement

The strategy of reinforcement is to reformulate the generation of musical content as a reinforcement learning problem, while using the output of a trained recurrent network as an objective and adding user defined constraints, e.g., some tonality rules according to music theory, as an additional objective.

Let us at first quickly remind the basic concepts of reinforcement learning, illustrated at Figure 7:

• An agent sequentially selects and performs actions within an environment;

• Each action performed brings it to a new state,

• with the feedback (by the environment) of a reward (reinforcement signal), which represents some adequation of the action to the environment (the situation).

• The objective of reinforcement learning is for the agent to learn a near optimal policy (sequence of actions) in order to maximize its cumulated rewards (named its gain).

Generation of a melody may be formulated as follows (as in Figure 8): the state s represents the musical content (a partial melody) generated so far and the action a represents the selection of next note to be generated.

Example: RL-Tuner Melody Generation

The reinforcement strategy has been pioneered by the RL-Tuner architecture by Jaques et al. [START_REF] Jaques | Tuning recurrent neural networks with reinforcement learning[END_REF]. The architecture, illustrated at Figure 8, consists in two reinforcement learning architectures, named Q Network and After training Note RNN on the corpus, a fixed copy named Reward RNN is used as a reference for the reinforcement learning architecture. The reward r of Q Network is defined as a combination of two objectives:

• Adherence to what has been learnt, by measuring the similarity of the action selected (next note to be generated) to the note predicted by Reward RNN in a similar state (partial melody generated so far);

• Adherence to user-defined constraints (e.g., consistency with current tonality, avoidance of excessive repetitions. . . ), by measuring how well they are fulfilled.

Although preliminary, results are convincing. Note that this strategy has the potential for adaptive generation by incorporating feedback from the user.

Unit Selection

The unit selection strategy relies in querying successive musical units (e.g., a melody within a measure) from a data base and in concatenating them in order to generate some sequence according to some user characteristics.

Example: Unit Selection and Concatenation Melody Generation

This strategy has been pioneered by Bretan et al. [START_REF] Bretan | A unit selection methodology for music generation using deep neural networks[END_REF] and is actually inspired by a technique commonly used in text-to-speech (TTS) systems and adapted in order to generate melodies (the corpus used is diverse and includes jazz, folk and rock). The key process here is unit selection (in general each unit is one measure long), based on two criteria: semantic relevance and concatenation cost. The architecture includes one autoencoder and two LSTM recurrent networks.

The first preparation phase is feature extraction of musical units. 10 manually handcrafted features are considered, following a bag-of-words (BOW) approach (e.g., counts of a certain pitch class, counts of a certain pitch class rhythm tuple, if first note is tied to previous measure, etc.), resulting in 9,675 actual features.

The key of the generation is the process of selection of a best (or at least, very good) successor candidate to a given musical unit. Two criteria are considered:

• Successor semantic relevance -It is based on a model of transition between units, as learnt by a LSTM recurrent network. In other words, that relevance is based on the distance to the (ideal) next unit as predicted by the model;

• Concatenation cost -It is based on another model of transition16 , this time between the last note of the unit and the first note of the next unit, as learnt by another LSTM recurrent network.

The combination of the two criteria (illustrated at Figure 9) is handled by a heuristic-based dynamic ranking process. As for a recurrent network, generation is iterated in order to create, unit by unit (measure by measure), an arbitrary length melody.

Structure

Another challenge is that most existing systems have a tendency to generate music with "no sense of direction". In other words, although the style of the generated music corresponds to the corpus learnt, the music lacks some structure and appears to wander without some higher organization, as opposed to human composed music which usually exhibits some global organization (usually named a form) and identified components, such as:

• Overture, Allegro, Adagio or Finale for classical music;

• AABA or AAB in Jazz;

• Refrain, Verse or Bridge for songs. Note that there are various possible levels of structure. For instance, an example of finer grain structure is at the level of melodic patterns that can be repeated, often transposed in order to adapt to a new harmonic structure.

Reinforcement (as used by RL-Tuner at Section 2.5.1) and structure imposition (as used by C-RBM at Section 2.4.4) are approaches to enforce some constraints, possibly high-level, onto the generation. An alternative top-down approach is followed by the unit selection strategy (see Section 2.6), by incrementally generating an abstract sequence structure and filling it with musical units, although the structure is currently flat. Therefore, a natural direction is to explicitly consider and process different levels (hierarchies) of temporality and of structure.

Example: MusicVAE Multivoice Generation

Roberts et al. propose a hierarchical architecture named MusicVAE [RER + 18b] following the principles of a variational autoencoder encapsulating recurrent networks (RNNs, in practice LSTMs) such as VRAE introduced at Section 2.4.2, with two differences:

• the encoder is a bidirectional RNN;

• the decoder is a hierarchical 2-level RNN composed of: a bottom-layer RNN using each embedding as an initial state17 and also as an additional input concatenated to its previously generated token to produce each subsequence.

The resulting architecture is illustrated at Figure 10. The authors report that an equivalent "flat" (without hierarchy) architecture, although accurate in modeling the style in the case of 2-measure long examples, turned out inaccurate in the case of 16-measure long examples, with a 27% error increase for the autoencoder reconstruction. Some preliminary evaluation has also been conducted with a comparison by listeners of three versions: flat architecture, hierarchical architecture and real music for three types of music: melody, trio and drums, showing a very significant gain with the hierarchical architecture.

Creativity

The issue of the creativity of the music generated is not only an artistic issue but also an economic one, because it raises a copyright issue 18 .

One approach is a posteriori, by ensuring that the generated music is not too similar (e.g., in not having recopied a significant amount of notes of a melody) to an existing piece of music. To this aim, existing tools to detect similarities in texts may be used.

Another approach, more systematic but more challenging, is a priori, by ensuring that the music generated will not recopy a given portion of music from the training corpus19 . A solution for music generation from Markov chains has been proposed [START_REF] Papadopoulos | Avoiding plagiarism in Markov sequence generation[END_REF]. It is based on a variable order Markov model and constraints over the order of the generation through some min order and max order constraints, in order to attain some sweet spot between junk and plagiarism. However, there is none yet equivalent solution for deep learning architectures. • by decreasing the values of the two control parameters of feature matching regularization, in order to less enforce the distributions of real and generated data to be close.

These experiments are interesting although the approach remains at the level of some ad hoc tuning of some hyper-parameters of the architecture.

Creative Adversarial Networks

Another more systematic and conceptual direction is the concept of creative adversarial networks (CAN) proposed by El Gammal et al. [START_REF] Elgammal | CAN: Creative adversarial networks generating "art" by learning about styles and deviating from style norms[END_REF], as an extension of generative adversarial networks (GAN) architecture, by Goodfellow et al. [GPAM + 14] which trains simultaneously two networks:

• a Generative model (or generator) G, whose objective is to transform random noise vectors into faked samples, which resemble real samples drawn from a distribution of real images; and

• a Discriminative model (or discriminator) D, that estimates the probability that a sample came from the training data rather than from G.

The generator is then able to produce user-appealing synthetic samples (e.g., images or music) from noise vectors. The discriminator may then be discarded. Elgammal et al. propose in [START_REF] Elgammal | CAN: Creative adversarial networks generating "art" by learning about styles and deviating from style norms[END_REF] to extend a GAN architecture into a creative adversarial networks (CAN) architecture, shown at Figure 11, where the generator receives from the discriminator not just one but two signals:

• the first signal, analog to the case of the standard GAN, specifies how the discriminator believes that the generated item comes from the training dataset of real art pieces;

• the second signal is about how easily the discriminator can classify the generated item into established styles. If there is some strong ambiguity (i.e., the various classes are equiprobable), this means that the generated item is difficult to fit within the existing art styles.

These two signals are thus contradictory forces and push the generator to explore the space for generating items that are at the same time close to the distribution of existing art pieces and with some style originality. Note that this approach assumes the existence of a prior style classification and it also reduces the idea of creativity to exploring new styles (which indeed has some grounding in the art history). In most of existing systems, the generation is automated, with little or no interactivity. As a result, local modification and regeneration of a musical content is usually not supported, the only available option being a whole regeneration (and the loss of previous attempt). This is in contrast to the way a musician works, with successive partial refinement and adaptation of a composition20 . Therefore, some requisites for interactivity are the incrementality and the locality of the generation, i.e. the way the variables of the musical content are instantiated.

Instantiation Strategies

Let us consider the example of the generation of a melody. The two most common strategies (illustrated at Figure 12)21 for instantiating the notes of the melody are:

• Single-step/Global -A global representation including all time steps is generated in a single step by a feedforward architecture. An example is DeepHear [Sun17] at Section 2.4.1.

• Iterative/Time-slice -A time slice representation corresponding to a single time step is iteratively generated by a recurrent architecture (RNN). An example is Anticipation-RNN [HN17] at Section 2.3.2.

Let us now consider an alternative strategy, incremental variable instantiation. It relies on a global representation including all time steps. But, as opposed to single-step/global generation, generation is done incrementally by progressively instantiating and refining values of variables (notes), in a non deterministic order. Thus, it is possible to generate or to regenerate only an arbitrary part of the musical content, for a specific time interval and/or for a specific subset of voices (shown as selective regeneration in Figure 12), without regenerating the whole content.

Example: DeepBach Chorale Generation

This incremental instantiation strategy has been used by Hadjeres et al. in the DeepBach architecture [START_REF] Hadjeres | DeepBach: a steerable model for Bach chorales generation[END_REF] for generation of Bach chorales22 . The architecture, shown at Figure 13, combines two recurrent and two feedforward networks. As opposed to standard use of recurrent networks, where a single time direction is considered, DeepBach architecture considers the two directions forward in time and backwards in time. Therefore, two recurrent networks (more precisely, LSTM) are used, one summing up past information and another summing up information coming from the future, together with a non recurrent network for notes occurring at the same Figure 13: DeepBach architecture Create four lists V = (V 1 ; V 2 ; V 3 ; V 4 ) of length L; Initialize them with random notes drawn from the ranges of the corresponding voices for m from 1 to max number of iterations do Choose voice i uniformly between 1 and 4; Choose time t uniformly between 1 and L; Re-sample V t i from p i (V t i |V \i,t , θ i ) end for Training, as well as generation, is not done in the conventional way for neural networks. The objective is to predict the value of current note for a a given voice (shown with a red ? on top center of Figure 13), using as information surrounding contextual notes. The training set is formed on-line by repeatedly randomly selecting a note in a voice from an example of the corpus and its surrounding context. Generation is done by sampling, using a pseudo-Gibbs sampling incremental and iterative algorithm (shown in Figure 14, see details in [START_REF] Hadjeres | DeepBach: a steerable model for Bach chorales generation[END_REF]) to produce a set of values (each note) of a polyphony, following the distribution that the network has learnt.

The advantage of this method is that generation may be tailored. For example, if the user changes only one or two measures of the soprano voice, he can resample only the corresponding counterpoint voices for these measures.

The user interface of DeepBach, shown at Figure 15, allows the user to interactively select and control global or partial (re)generation of chorales. It opens up new ways of composing Bach-like chorales for non experts in an interactive manner, similarly to what is proposed by FlowComposer for lead sheets [START_REF] Papadopoulos | Assisted lead sheet composition using FlowComposer[END_REF]. It is implemented as a plugin for the MuseScore music editor.

Conclusion

The use of deep learning architectures and techniques for the generation of music (as well as other artistic content) is a growing area of research. However, there remain open challenges such as control, structure, creativity and interactivity, that standard techniques do not directly address. In this article, we have discussed a list of challenges, introduced some strategies to address them and have illustrated them through examples of actual architectures24 . We hope that the analysis presented in this article will help at a better understanding of issues and possible solutions and therefore may contribute to the general research agenda of deep learning-based music generation.

Figure 1 :

 1 Figure 1: Anticipation-RNN architecture. Reproduced from [HN17] with permission of the authors

Figure 3 :

 3 Figure 3: Generation in DeepHear. Extension of a figure reproduced from [Sun17] with permission of the author

Figure 4 :

 4 Figure 4: Example of melody generated (bottom) by MusicVAE by adding a "high note density" attribute vector to the latent space of an existing melody (top). Reproduced from [RER + 18a] with permission of the authors

Figure 5 :

 5 Figure 5: Style transfer full architecture/process. Reproduced with permission of the authors

Figure 6 :

 6 Figure 6: C-RBM Architecture

Figure 8 :

 8 Figure 8: RL-Tuner architecture

Figure 9 :

 9 Figure 9: Unit selection based on semantic cost

Figure 10 :

 10 Figure 10: MusicVAE architecture. Reproduced from [RER + 18b] with permission of the authors

Figure 11 :

 11 Figure 11: Creative adversarial networks (CAN) architecture

Figure 12 :

 12 Figure 12: Strategies for instantiating notes during generation

Figure 14 :

 14 Figure 14: DeepBach incremental generation/sampling algorithm

Figure 15 :

 15 Figure 15: DeepBach user interface

Music difficult to distinguish from the original corpus.

Additional challenges are analyzed in[START_REF] Briot | Deep Learning Techniques for Music Generation[END_REF].

Two examples are Markov constraints[START_REF] Pachet | Finite-length markov processes with constraints[END_REF] and factor graphs[START_REF] Pachet | Sampling variations of sequences for structured music generation[END_REF].

The model can be stochastic, such as a restricted Boltzmann machine (RBM)[START_REF] Goodfellow | Deep Learning[END_REF], or deterministic, such as a feedforward or a recurrent network. In that latter case, it is common practice to sample from the softmax output in order to introduce variability for the generated content[START_REF] Briot | Deep Learning Techniques for Music Generation[END_REF].

Note that this may be a very costly process and moreover with no guarantee to succeed.

An important specificity of the architecture (not discussed here) is the notion of dilated convolution, where convolution filters are incrementally dilated in order to provide very large receptive fields with just a few layers, while preserving input resolution and computational efficiency [vdODZ + 16].

Both are 2-layer LSTMs[START_REF] Hochreiter | Long short-term memory[END_REF].

Autoencoders are trained with the same data as input and output and therefore have to discover significative features in order to be able to reconstruct the compressed data.

Note that this is a simple example of transfer learning[START_REF] Goodfellow | Deep Learning[END_REF], with a same domain and a same training, but for a different task.

A variational autoencoder (VAE)[START_REF] Diederik | Auto-encoding variational Bayes[END_REF] is an autoencoder with the added constraint that the encoded representation (its latent variables) follows some prior probability distribution (usually a Gaussian distribution). Therefore, a variational autoencoder is able to learn a "smooth" latent space mapping to realistic examples.

Note that one may balance between content and style objectives through some α and β parameters in the L total combined loss function shown at top of Figure5.

In the case of an image, the correlations between visual elements (pixels) are equivalent whatever the direction (horizontal axis, vertical axis, diagonal axis or any arbitrary direction), in other words correlations are isotropic. In the case of a global representation of musical content (see, e.g., Figure12), where the horizontal dimension represents time and the vertical dimension represents the notes, horizontal correlations represent temporal correlations and vertical correlations represent harmonic correlations, which have very different nature.

Note that this also some kind of style transfer[START_REF] Dai | Music style transfer issues: A position paper[END_REF], although of a high-level structure and not a low-level timbre as in Section 2.4.3.

They use a deep learning implementation of the Q-learning algorithm. Q Network is trained in parallel to Target Q Network which estimates the value of the gain)[START_REF] Hado Van Hasselt | Deep reinforcement learning with double Q-learning[END_REF].

At a more fine-grained level, note-to-note level, than the previous one.

In order to prioritize the Conductor RNN over the bottom layer RNN, its initial state is reinitialized with the decoder generated embedding for each new subsequence.

On this issue, see a recent paper[START_REF] Deltorn | Deep creations: Intellectual property and the automata[END_REF].

Note that this addresses the issue of avoiding a significant recopy from the training corpus, but it does not prevent to reinvent an existing music outside of the training corpus.

An example of interactive composition environment is FlowComposer[START_REF] Papadopoulos | Assisted lead sheet composition using FlowComposer[END_REF]. It is based on various techniques such as Markov models, constraint solving and rules.

The representation shown is of type piano roll with two simultaneous voices (tracks). Parts already processed are in light grey; parts being currently processed have a thick line and are pointed as "current"; notes to be played are in blue.

J. S. Bach chose various given melodies for a soprano and composed the three additional ones (for alto, tenor and bass) in a counterpoint manner.

The two bottom lines correspond to metadata (fermata and beat information), not detailed here.

A more complete survey and analysis is[START_REF] Briot | Deep Learning Techniques for Music Generation[END_REF].

Acknowledgements

We thank Gaëtan Hadjeres and Pierre Roy for related discussions. This research was partly conducted within the Flow Machines project which received funding from the European Research Council under the European Union Seventh Framework Programme (FP/2007-2013) / ERC Grant Agreement n. 291156.