Nicolas Barroso

Amélie Fondevilla

David Vanderhaeghe

Stroke based Painterly Inbetweening

come

Stroke based Painterly Inbetweening

Nicolas Barroso 1 , Amélie Fondevilla 2 , David Vanderhaeghe 1 1. IRIT, Université de Toulouse, CNRS, INP, UT3 , Toulouse, France. 2. Les fées spéciales, Montpellier, France.

Traditional paint-on-glass animation use a technique called under the camera. The artist creates a first frame and captures it on film. The next frame is obtained by slight modifications of the first one, and so on for the whole sequence. Each frame of the animation only exists when it is captured and is then destroyed to produce the next frame. The artist has to anticipate the entire sequence, there is no option to go back. On the other hand, cell animation artists usually think the animation in term of keyframes and then refine the animation with intermediate frames. Each frame can be drawn and modified independently. Our goal is to provide an approach with the look and feel of the under the camera technique and the level of control of the keyframe approach.

RESULTS

In these examples, key frames are marked with arrows on the timeline. Intermediate frames convey the styles given by the key frames.

The time needed to render a frame depends on the number of advected strokes and the number of pixel covered by the generated strokes. These two examples took less than one second per frame to render.

On the example on the right the motion flow captures the motion of the shape. The specular highlight should follow another motion. In this example we can see this motion inconsistency creates two highlights for the red and blue spheres instead of one. This limitation could be addressed by providing multiple motion flows and set which strokes follow which flow. We select a subset of strokes from each strokes set. The ratio of strokes is defined such that at least one strokes set is fully selected for every intermediate frame. This assumption helps conveying the keyframe information during the animation.

RELATED WORK

The ratio is for the set corresponding to the first keyframe. Where i is the intermediate frame indice, and n the total frame count between the two keyframes. The ratio for the second set is

  13] BASSETT K., BARAN I., SCHMID J., GROSS M., SUMNER R. W.: Authoring and animating painterly characters.ACM Trans.Graph. 32, 5 (Oct. 2013). [DBB * 17] DVOROŽŇÁK M., BÉNARD P., BARLA P., WANG O., SÝKORA D.: Example-based expressive animation of 2d rigid bodies.ACM Trans. Graph. 36, 4 (July 2017). [SSGS11] SCHMID J., SENN M. S., GROSS M., SUMNER R. W.: Over-coat: An implicit canvas for 3d painting. ACM Trans. Graph. 30, 4 (July 2011). [WNS * 10] WHITED B., NORIS G., SIMMONS M., SUMNER R. W.,GROSS M., ROSSIGNAC J.: Betweenit: An interactive tool for tight in-betweening. CGF 29, 2 (2010) Step 1: Propagation • Strokes from the first keyframe are advected forward • Strokes from the second keyframe are advected backward Each intermediate frame has two sets of advected strokes. To advect strokes we compute an extended motion flow from the input motion flow following these steps: • Triangulate image space using a Delaunay triangulation with max edge length to evenly distribute motion samples • Extrapolate valid motion samples using bi-harmonique diffusion to each triangulation vertices • Interpolate motion vectors from vertices to each pixel of the motion flow. OVERVIEW From a set of key frames and a motion flow, our methods automatically generates intermediate frames. The set key frames defines the style of intermediate frames. The motion flow captures the motion of the animation and drive the stroke generation process.

  2D automatic inbetween methods focus on contour and silhouettes sketches [DBB*17, WNS*10]. Previous work on filled region by stroke based rendering focus on temporal coherence, and attach painting strokes to 3D surfaces [SSGS11, BBS*13].We propose to generate all the strokes in 2D, without specific temporal coherence, taking inspiration from traditional paint-on-glass style, and giving controle with key frame authoring. is generated from the two strokes sets computed in step 1.


INTRODUCTION

Despite the emphasis put on institutions permanence and their continuity through selfreproduction (Scott 2003;Zilber 2006), the theoretical underpinning for the concept of institutional work stressed that an active effort is yet required to maintain institutions [START_REF] Lawrence | Institutions and Institutional Work[END_REF]. According to [START_REF] Lawrence | Institutions and Institutional Work[END_REF] "institutional work aimed at maintaining institutions involves supporting, repairing or recreating the social mechanisms that ensure compliance (p 230). A key question in institutional work literature is "who" can perform maintenance work (Lawrence et al. 2011). It is widely recognized in the literature that field position of actors acts as an enabling/constraining condition for performing institutional work. Based on this assumption, a distinction is made between two extreme field positions: central or incumbent players and challengers or peripheral players (Batilana 2006).

Extant studies acknowledge that incumbents and central players have a higher propensity to perform institutional work of maintenance since they are those who are the most favoured by the status quo [START_REF] Hampel | Institutional work : Taking stock and making it matter[END_REF]). An additional argument that supports this thesis consist in considering incumbents and central players in the field as more socialized and endowed with adequate resources to ensure compliance and policing of existing institutional setting.

However, these assertions leave much untold about how peripheral actors can maintain institutions. Yet, several studies highlighted how peripheral players may play the role of "gatekeepers" [START_REF] Carroll | Why the microbrewery movement ? Organizational dynamics of resource partitioning in the US brewing industry[END_REF] of the field and work on maintaining and repairing institutions. Such situation is more likely to happen when elite actors in the field deviate from institutional settings and the core values of the field [START_REF] Rao | Institutional Change in Toque Ville : Nouvelle Cuisine as an Identity Movement in French Gastronomy[END_REF]. The hyper concentration of power among few elite actors as well as their exposition to increasing conflicting institutional pressures and imperatives are more and more common phenomena that push central players to discard institutionalized practices and betray its core values.

Therefore, institutional maintenance by peripheral players lends itself as a worthy phenomenon to study.

This paper aims at addressing this theoretical gap through the exploration of institutional work of maintenance performed by peripheral players in their field. We draw on a qualitative study based on an extreme single case study [START_REF] Yin | Design and methods[END_REF] of the work of institutional maintenance performed in the field of press in France by Mediapart. Mediapart is an online newspaper founded in 2008 with the aim of defending the independence of press from financial and politic hegemony as well as mainlining field practices such as investigations and the involvement of journalists in the governance of newspapers.

As a new entrant, Mediapart has low resources and no recognition from the professional networks and inter-organization networks in the field. Its peripheral position in the field is therefore a transparent and compelling phenomenon.

Our findings show that the work of maintenance performed by Mediapart is threefold. First an extensive use of historical narrativization and memory work intended to invoke an affiliation with reformist and figures of resistance from the past and to recall obscure periods of history of the field of press in France such as the autocratic era of the restoration of the second empire. Second, since peripheral players are not well endowed with resources, Mediapart constructed an alternative model of gathering resources. Mediapart worked on delegitimizing existing modes of access to resources and theorizing new resources such as the monetization of news online. The third finding derives from the insularity of Mediapart as a peripheral actor and consist in a work of rebellion and opposition toward established power in the field.

Mediapart had fuelled controversy and scandals targeting political and economic elite in France and engaged in illegal actions geared toward gaining acceptance in the field. This article makes several contributions to institutional work of maintenance and to institutional theory more broadly. First it contributes to challenging the structural determinism in institutional theory that restrict maintenance work to elite actors. Therefore, we build a theoretical underpinning of when and how institutions peripheral players maintain institutions.

We show how peripheral actors may substitute resources and power that they are deprived of by other means in order to perform maintenance work.

Second, we contribute to the recent turn to history and memory in institutional work since we show how maintenance work connect the past with the present and build social meaning by invoking reputation trajectory of figures from the past [START_REF] Jansen | Resurrection and appropriation : Reputational trajectories, memory work, and the political use of historical figures[END_REF]. The analogy and the multiple comparisons made with obscure and dark periods in the French history of autocratic political power and corruption sheds light on the role of memory work through triggering emotions such as fear and anxiety in maintaining institutions. We therefore expand the use of history and memories in institutional maintenance beyond the work of mythologisation [START_REF] Lawrence | Institutions and Institutional Work[END_REF]. We locate this contribution in historical institutionalism, and we extend the use of memory as a strategic asset [START_REF] Foster | History as social memory assets : The example of Tim Hortons[END_REF]) to institutional maintenance.

Institutional work of maintenance:

Institutional work is defined as the daily efforts of actors in the creation, maintenance and destabilization of institutions [START_REF] Lawrence | Institutions and Institutional Work[END_REF]. It is focused on the daily, repeated, routine, decided or unexpected interactions of a wide range of actors, without distinction of their status or position in the organizational field [START_REF] Hampel | Institutional work : Taking stock and making it matter[END_REF]. Unexpectedly, the turn to institutional work revived the interest in the process of institutional continuity and maintenance while institutional scholars' attention was geared toward explaining institutional change and disruption.

Whereas early explanations of institutional continuity and maintenance relied on selfreproduction mechanisms [START_REF] Jepperson | Institutions, Institutional Effects and Institutionnalism[END_REF], institutional work brought to the fore the vital role of continuous and intentional efforts of actors to insure institutional maintenance.

Therefore, institutional maintenance relies on agency and needs actors' willingness and purposeful actions to insure institutional reproduction even for highly institutionalized social arrangements.

According to [START_REF] Lawrence | Institutions and Institutional Work[END_REF] "institutional work aimed at maintaining institutions involves supporting, repairing or recreating the social mechanisms that ensure compliance (p 230). Assuming that institutional maintenance requires agency and purposeful action has as a corollary which is that institutionalized practices and institutions may vanish and erode as they face outsider driven attacks [START_REF] Maguire | Discourse and deinstitutionalization : The decline of DDT[END_REF] or when they are slowly left in oblivion (Douglas 1985).

In their study of the hight table dinners in Cambridge, [START_REF] Dacin | Formal dining at Cambridge colleges : Linking ritual performance and institutional maintenance[END_REF] shows the importance of rituals in maintaining institutions against oblivion. When institutions are called into questions or face legitimacy attacks, actors maintain their institution through work of policing, that consists of using sanctions and inducement, deterring aimed at inculcating obedience through the fear of sanctions and through valourizing and demonizing behaviours [START_REF] Lawrence | Institutions and Institutional Work[END_REF]. Building on the recent development on the role of emotions and institutional dynamics, Gill and Burrow (2017) uncovered the role of being frightened in maintaining practices in the French haute cuisine.

Institutional work of maintenance can take several forms. [START_REF] Micelotta | Institutions and maintenance : The repair work of Italian professions[END_REF] introduced institutional repair to refer to work of maintaining broken institutions. [START_REF] Maguire | Discourse and deinstitutionalization : The decline of DDT[END_REF] pinpointed the defensive nature of institutional work when the institutions at place are attacked by outsiders such as activists or social movement. Key to institutional work concept is addressing the question of "who" can engage in efforts of institutional change and maintenance. Such question echoes the enabling conditions [START_REF] Hardy | Institutional entrepreneurship[END_REF] which facilitate and constrain the propensity and the willingness of actors to engage in institutional work of maintenance. Since the organizational field is the arena where actors live in/with, struggle over the meaning and fight over resources [START_REF] Fligstein | Social skill and institutional theory[END_REF], actors' position in the field is crucial to the understanding of who may engage in which kind institutional work.

Field position as an enabling condition of institutional work:

Institutional theory owes much to the sociology of Pierre Bourdieu from which it borrowed the concept of organizational field which became one of its iconic concepts. According to [START_REF] Dimaggio | The Iron Cage Revisited : Institutional Isomorphism and Collective Rationality in Organizational Fields[END_REF] the organizational field refers to "those organizations that, in the aggregate, constitute a recognized area of institutional life: key suppliers, resource and product consumers, regulatory agencies, and other organizations that produce similar services or products" (p 148). Organizational fields have a high degree of autonomy from macro political and social phenomena [START_REF] Naidoo | Fields and institutional strategy : Bourdieu on the relationship between higher education, inequality and society[END_REF]) and generate their own rules, social meaning, values, and behavioural imperatives. Bourdieu used his theory of the field to acknowledge the existence of particular institutional life in many fields such as education and the field of press. Having the field as level of analysis allowed Bourdieu and his followers to uncover how power relations manifest and how they shape social interactions and practices [START_REF] Bourdieu | L'emprise du journalisme[END_REF]. The translation of the concept of field and the sociology of Pierre Bourdieu broadly speaking in institutional theory got rid from power and domination (Greenwood and Meyer 2008) in favour to more cultural and cognitive mechanisms. Institutional theory acknowledges that actor's positions in the field reflects their degree of socialization to the institutional setting and their propensity to comply to rules and avoid sanctions for deviant behaviour [START_REF] Battilana | Agency and institutions : The enabling role of individuals' social position[END_REF]). Field position is therefore an important lens from which institutional work and agency in institutional theory should be grasped.

Institutional scholars often distinguish between two extreme positions: central players and peripheral players (Leblibici et al. 1991;[START_REF] Rao | Institutional Change in Toque Ville : Nouvelle Cuisine as an Identity Movement in French Gastronomy[END_REF][START_REF] Suddaby | Rhetorical Strategies of Legitimacy[END_REF].

Taking a strategic lens, Batillana (2006) uses a different terminology: incumbent and challenger to designate the two extreme positions.

Incumbents and central players are deemed the more socialized actors to the institutional setting and the actors who profit the most from the status quo [START_REF] Lawrence | Institutions and Institutional Work[END_REF].

Peripheral players or challengers are actors who hold less powerful positions in the field and whose access to resources is made difficult by their insularity [START_REF] Hardy | Institutional entrepreneurship[END_REF]). Since they are less favoured by the established institutions and are less exposed to the risk of being sanctioned because of deviance. They are often portrayed as the source of institutional disruption and novelty of their field (Leblibici et al. 1991). According to [START_REF] Croidieu | Complex field-positions and non-imitation : Pioneers, strangers, and insulars in Australian fine-wine[END_REF] peripheral players are also those who may afford discarding institutionalized practices the most.

Even though institutional theory had put much effort to discard some of its early "unfortunate intellectual baggage" [START_REF] Scott | Approaching Adulthood : The Maturing of Institutional Theory[END_REF]) such considering actors as "cultural dopes", it got trapped' again, in an unfortunate structural determinism regarding actors' position in the field and the institutional work that they can perform.

Indeed, incumbents and central players are expected to maintain institutions and to defend the status quo as they are favoured by the institutional setting at place and peripheral players are condemned to disruption (Batilana 2006;Leblibici et al. 1991). Few works tried to challenge this determinism by showing how incumbents and central players may bring about institutional change. For instance, [START_REF] Rao | Institutional Change in Toque Ville : Nouvelle Cuisine as an Identity Movement in French Gastronomy[END_REF] show that a major chance in the field of French cuisine had been triggered by elite chefs with multiple Michelin stars. The explanation lies in the connections that high status actors in the field may have with other high-status players outside their field and how such interactions is conducive to challenging extant values of the field and social meaning. Chefs like Bocus or Troisgros visits in Japan contributed heavily to changing how they perceive and view the art of haute cuisine in France. In a similar vein [START_REF] Suddaby | Rhetorical Strategies of Legitimacy[END_REF] show that incumbents are nested in different organisational field and may import practices from one field to another and bring about institutional change. Incumbents can therefore easily challenge the status quo if they preserve and sustain their advantages (Ben Slimane 2012). However, institutional theory still struggles with the conundrum of peripheral players performing maintenance work.

Institutional work of maintenance by peripheral players

Because they are less socialized, they are disadvantaged by status quo and are less resourceful, institutional theory falls short in explaining institutional work of maintenance by peripheral players. However, literature is replete with stories and cases of peripheral players defending the status quo and performing institutional work of maintenance.

In this section we will discuss the counterexamples and the situations where peripheral players may work on maintaining institutions rather than being restricted to the straitjacket of institutional deviance and disruption. In their study of the emergence and the rise of microbreweries movement in the late nineties in US Caroll and Swaminathan (2000) show that the dissatisfaction with the quality of beer and the critics of incumbents' practices who turned to mass production, pushed a wide variety of challengers to start a movement of restoration and defense of the old taste as well as old practices in the beer industries. The challengers have been dissatisfied with the commoditization of beer and the deterioration of its taste. Their aim was to bring back craftsmanship practices and aesthetic values to the field.

In the music field, peripheral players reacting to the hegemony of big Majors record label constituted an association of independent labels [START_REF] Williamson | Rethinking the music industry[END_REF] to defend what they've advocated was the core values of the field such as plurality of tastes and raising stars. Peripheral players appear therefore as gatekeepers (Caroll and Swaminithan 2000) of the institutional order that is threaten by the concentration of power of few actors and their deviation from the core values of the field.

Scholars showed that the work of institutional maintenance is driven by the interests of incumbents which position and advantages in the field are favored by existing institutional settings [START_REF] Dacin | Formal dining at Cambridge colleges : Linking ritual performance and institutional maintenance[END_REF]. This raises the question of the motivation of peripheral players to defend and maintain the institutional order. Insights from the beer industry (Caroll and Swaminathan 2000) and the music industry (Anand and Petterson 2000) show that the concentration of power of incumbents and their deviation from core values of the field [START_REF] Rao | Institutional Change in Toque Ville : Nouvelle Cuisine as an Identity Movement in French Gastronomy[END_REF]) act as a motivation for peripheral players and challengers to perform institutional work of maintenance.

Institutional work of maintenance by peripheral players differs from the maintenance work of incumbents in two aspects. First, the motivation of maintaining institutional order doesn't hinge on favoring interests and advantage but is rather driven by symbolic and cultural dimensions of the field such as cores values and collective identity.

Second, institutional work literature is unanimous on the necessity of power and resourcefulness to perform institutional work of maintenance and defensive institutional work [START_REF] Micelotta | Institutions and maintenance : The repair work of Italian professions[END_REF]. Examples of institutional work of maintenance such as valourizing, demonizing or policing can only be performed by resourceful and powerful actors [START_REF] Lawrence | Institutions and Institutional Work[END_REF]. All these developments raise unanswered questions yet on how peripheral players who lack resources and who are more insular than central players maintain institutions in their field. Accordingly, we formulate the following research question:

How peripheral players maintain institutions of the field?

Case description, data collection, methods and data analysis

Case selection

This research is based on an exploratory case study whose aim is to build theory through revealing the richness and trustworthiness of extreme cases [START_REF] Gioia | Seeking qualitative rigor in inductive research : Notes on the Gioia methodology[END_REF][START_REF] Langley | Templates and turns in qualitative studies of strategy and management[END_REF][START_REF] Yin | Design and methods[END_REF]. We've selected the case of the newspaper Mediapart in the French press field for two reasons. First, Mediapart position in the field of press in France satisfies with the theoretical conditions of peripherality: (1) lack of resources and (2) lower socialization as concerns about Mediapart membership to the press unions and the acceptance of its journalist as professional had been raised in the aftermath of its creation and continued thereafter. Second, the field of press in French is the locus of criticisms for the high concentration of incumbents and the continuous attempts by pollical and economic power to control press. Pierre Bourdieu who dedicated many of his works to the study of the field of French journalism raised the concern of the high degree of control held by market logic over journalism [START_REF] Bourdieu | L'emprise du journalisme[END_REF].

Mediapart is an independent and participative online newspaper, founded in 2008. Its founders: François Bonnet, Gérard Desportes, Laurent Mauduit and Edwy Plenel, aimed at creating an alternative, independent newspaper that responds to the crisis of independence of the press in France. Challenging what they call "the old press", the four journalists had the ambition to create the conditions for their independence through the establishment a new newspaper aimed at embodying and maintaining the values of the field.

Mediapart is an independent newspaper that refuses to be owned by industrial groups or wealthy businessmen, rejects public subsidies and advertising revenues.

The newspaper started with limited resources: "We really worked on the bone, that is to say we didn't spend anything. We set up, each person set up their own desk with their own screwdriver, table, chair... We were careful with all our expenses" (MH Smiejan).

Data collection

Data collection was conducted between 2014 and 2015. We've conducted 15 semi-structured interviews that lasted between 1.5 and 2.15 hours. All interviews were recorded and transcribed. We interviewed 14 respondents and met with the co-founder, president and director of Mediapart, Edwy Plenel twice. All of the interviewees were involved in the creation of the Mediapart and are hold key positions in Mediapart.

We triangulated primary data with secondary data for more reliability. We analyzed 50 articles written by the founders about the launch of Mediapart. We've also analyzed 4 books written by two of the co-founders on the press and the independence of the press and collected 10 videos of TV shows and conferences.

Data analysis

In order to enhance the rigor of our data analysis process that is often called into question for qualitative research [START_REF] Pratt | From the editors : For the lack of a boilerplate : Tips on writing up (and reviewing) qualitative research[END_REF], we adopted the Gioia method that offers a "boilerplate" for qualitative data analysis. Gioia method is a methodological framework that guarantee a systematic conceptual and analytical discipline that allows the achievement of rigorous and relevant interpretations of data [START_REF] Gioia | Seeking qualitative rigor in inductive research : Notes on the Gioia methodology[END_REF]. Our analysis followed multiple steps.

First, we have conducted an open coding of our data looking for what are the major issues in the field and how they are framed and viewed by the different constituencies of the field [START_REF] Hoffman | Institutional evolution and change : Environmentalism and the US chemical industry[END_REF]. We have revealed numbers of themes such as press independence, economic power, political power, advertisement, public subsidies, competition, business model, democracy, freedom of speech, the profession of journalist, journalism practices etc.

In the second step and consisting with Gioia method, we've initiated a process if iteration between open codes, which reflects how actors perceive and talk about the object of the study, and theory, institutional theory more particularly. This second step is close to what [START_REF] Strauss | Qualitative analysis for social scientists[END_REF] designates as an axial coding and resulted in the identification of templates of second order codes. For instance we've coded the concerns and critics voiced by Mediapart against advertisement and public subsidies as "strategies of delegitimating existing resources". From an institutional prescriptive discursive struggle about legitimacy [START_REF] Hardy | Institutional entrepreneurship[END_REF]) is an important process in the dynamic of institutional continuity and change.

The third step of Gioia method consists of lumping second order themes in aggregate dimensions as portrayed by Figure (1). At this stage we are firmly located in the theoretical realm looking for labels that resonate with theory and that could answer to our research question. As an illustration our label "memory work and invocation of the past" refers to how peripheral players through invoking affiliation with reputed figures of the past in their field and through recalling on obscures periods of the field.

1-Historical narrativization and memory work

This work of institutional maintenance is based on the use of field memory and historical narrative. We have identified two sub-strategies for this maintenance work. The first strategy consists of invoking a filiation with historical figures of the press field known for their reformism and resistance for a free and independent press. In the second strategy, Mediapart drew a parallel between the current situation of the field and dark episodes when the press totally lost its independence.

Invoking an affiliation with reformist and resistant figures

In interviews, books and articles published, as well as in blogs and public conferences, In the same vein, the interviewees often refer to the committed British writer and journalist George Orwell . Coming from a bourgeois background, Orwell never stopped opposing British imperialism. He was always scandalized by a press that fell victim to the noise of communication, which relays "false" and unverified information and worked to make known the "truth". He writes to denounce lies and draws attention to issues as valuable as

Mediapart as the verbatim below attests.

"Orwell, whom I cherish, said about the "industrial press" that they are in the hands of people who have good reasons to have bad These figures of the past symbolize a committed, militant, critical, non-conformist and revolutionary journalism, which opposes the powers that be, the totalitarian regimes and ideologies. They embody thoughts on the fringe of the dominant currents.

Invoking dark periods of history

Mediapart's use of history has also served to recall the dark episodes in French history when the press was enslaved by the authorities and prevented from playing its role as an essential cog in democracy.

The speeches of the founders call for great vigilance, and recall the dark times of the Second Empire, the Third Republic and the interwar period. Periods marked by censorship, "systems of corruption", and media-financial and political scandals that destroyed, among others, the Mediapart warns against a return of "the business press" as in the interwar period. A conniving, complacent, conformist press, which gives a smooth image and serves the interests of those who have power and money. They draw a parallel between the press of today, which is highly concentrated and inbred with the business world, and the press of the time, which was controlled and financed by the employers' organization, the Comité des Forges, which on the eve of the First World War worked to promote the interests of the steel industry in France.

2-Constructing an alternative model of gathering field resources

The second institutional maintenance work performed by Mediapart relates to resources. Our results show that Mediapart has built an alternative revenue model, and therefore of resource appropriation. Mediapart has invested in a work of delegitimization of the existing modes of resource appropriation responsible for the drift of the field. Secondly, through a process of bricolage that we have labelled financial bootstrapping, the co-founders of Mediapart have succeeded in gathering the necessary funding to launch their activity. Third, Mediapart theorized the paying nature of online information and thus a new revenue model.

Deligitimizing existing sources of revenues

The founders question the legitimacy of the dominant revenue and financing model of the print press, which relies on three resources: capital participation by industrialists or investors, advertising and public subsidies. According to Mediapart, these three resources are responsible for the drift of practices in the press field.

In France, the capital of newspapers is most often held by industrial groups and rich French investors whose business is not journalism. As the following excerpt illustrates, this type of financial control necessarily leads to a control of the editorial line and reduces the independence of the press and deviates it from its role as a cog in democracy. Mediapart castigates the risk of censorship.

"We can see very well, for example, what happened, we wrote about it in the newspaper a few days ago, how Bolloré censored a documentary on Canal+ that had been validated by the program management and the legal department, which was even announced in the newspapers. He censored it a few days before it was broadcast because it was a very disturbing investigative documentary on a French bank, Crédit Mutuel, which happens to be one of Mr. Bolloré's main financial partners" (Fabrice Arfi).

The founders refuse to finance the newspaper through subsidies, a key source of income in the press field. They consider that direct state aid is detrimental to the independence of a newspaper. Subsidies mean owing something to the government in power.

"In addition, Mediapart says no to the question of subsidies, we do not want it because we want to remain completely independent of power" (Laurent Chemla).

In addition to undermining the independence of newspapers and journalists, subsidies create a bias in the press economy. The founders speak of a sector and companies under perfusion.

Many newspapers cannot find a financial balance without direct subsidies, which ultimately undermines their independence.

"When we see that the State gives 17 million euros to Le Monde, 10 million euros to Libération, etc., it is unhealthy for the economic balance of the newspaper. And it's unhealthy because a press company, like any other, must find a financial balance, otherwise it will never be independent" (Marie-Hélène Smiejan, co-founder of Mediapart).

The founders also denounced the subsidies, which they described as a democratic scandal.

Indeed, there is a strong opacity around the grants and how they are distributed.

This work of delegitimization also touches advertising revenues, which are strongly criticized for several reasons. The advertising models require a strong consultation, a race to the audience, a race to the click, thus dictating a certain type of content. "You have to do people, sex and sports. There you can hope on this trio" (Marie-Hélène Smiejan).

The editorial choices are thus strongly determined by the logic of the massive click.

According to the founders, the editorial model is increasingly being pulled down.

The contingency of the advertising model is also caused by censorship imposed by the advertising companies and/or their clients. The founders point out that the most important advertisers are close to the economic and political power and that this implies editorial complications. Independence is challenged when some newspapers show more freedom in their editorial choices and publish articles that displease the advertisers and their clients, with the risk of losing advertising and advertising revenue.

"Libération paid the price with Bernard Arnaud, because one day they made a headline "Casse-toi riche con" [f*** off rich jerk],

which he found insulting" (Laurent Mauduit).

Financial bootstrapping

Mediapart's revenue model reflects the founders' commitment to editorial independence, which they believe is intimately linked to the economic and financial independence of the newspaper. The financial aspects of the project, the sources and flow of income (s) have been thought out in this sense, starting with the constitution of the capital. For a year the founders "groped to find the right system".

"Our ambition, both in financial and legal terms is to resume the path, which was interrupted by the loss of economic independence of "Le Monde", "Libération" and other newspapers, of a press of journalists...So what I want to show is that we can arrive serenely, step by step, to build in a loyal, honest, transparent way, without selling our soul," (Edwy Plenel).

To start the newspaper and ensure its viability for three years, five million euros had to be raised. The founders did not want the newspaper to be under the control of a "Boss", a rich

French businessman or industrialist who would put the five million euros on the table. This led the founders to look for financing in increments of 500,000 euros. Finally, two independent entrepreneurs, actors in the computer and new technologies sector, Jean-Louis Bouchard and Thierry Willem, embarked on the adventure convinced by the project and by their support for the cause and values of the founders. They contributed with 1,110,000 euros.

Marie-Hélène Smiejan and Edwy Plenel, the co-founders, also had to go into debt to raise the first Million Euros. Together with the other co-founders, they contributed with 1,325,000 euros to Médiapart's capital.

"Today the guarantee of independence is ... the capital belongs to the journalists" (Marie-Hélène Smiejan).

The founders then set up a simplified joint stock company, the "Société des amis", which

initially brought together about forty people. According to the president of "Société des amis", Michel Broué, a university mathematician close to the founders, it is a question of "friends" who brought for some 1500 euros for others a few thousands or tens of thousands euros by solidarity.

Theorizing new resources: the monetization of online news

The independence of the newspaper is determined by the way its capital is constituted, but also the sources of income. Mediapart is an online newspaper, whose capital is majority owned by its founders and editors, which gives access to different editorial contents via a monthly subscription of nine euros. It is a newspaper whose revenue model goes against the doxa of the time.

In the 2000s, pure players did not exist, the Internet meant free, a freeness that eventually affected the paper press, where revenues are generated by advertising. Mediapart arrived with a model that was different from the dominant model, an online paying newspaper and long articles of 35,000 to 40,000 characters against an average of one thousand to one thousand five hundred characters. In the field nobody believed in it, the model provoked a lot of criticism.

According to the founders, the model that supports the economic and editorial independence, which are very intertwined, is the paying model.

"The best way, even if it is unorthodox, to have economic independence is to depend only on the trust of its readers and nothing else. (Fabrice Arfi).

The founders' challenge was to find fifty thousand readers willing to pay for independent, quality information. Since the revenues come only from the readers, no influence can prevent the newspaper and the journalists from publishing what they want to publish, hence their slogan "only our readers can buy us".

The founders worked to establish an economic universe around the production of information, independent information that has value, public interest, and a price. The interviewees talked about the behind-the-scenes production of information, the long time it takes, the resources it consumes, the cost of information and independence, and normalized the payment of information by the readers. The discourse was that by accepting "this pact", readers guarantee the independence of the newspaper, build it and maintain it.

3-Rebellion and opposition

Mediapart also had to invest in an institutional work of rebellion and disobedience against the rules of the field that constrained the work of maintenance especially coming from an actor at the periphery of the field. By bringing political and financial scandals to light, Mediapart opposed the powers and proved its independence. Secondly, Mediapart has entered into resistance against the rules that do not recognize it as a full member of the field and that call into question its legitimacy.

Fueling controversy and scandal

The starting point of Mediapart was the will of its founders to get out of the agendas of those who have the power of speech and communication, of storytelling, of media noise, of the "train-train" of AFP (Agence France-Presse), to be apart. The objective was to create information that is not a willing victim of communication, information that is democratic and of public interest, information that is often hidden, that sometimes disturbs and upsets. The best way to do this, according to the founders, is to put investigative journalism back at the heart of the editorial activity, to be independent and to have intellectual autonomy.

The founders and their collaborators have set up an offensive, critical, rebellious journalism that "does not bend. They have made this known through their articles, their speeches in various TV and radio shows, the videos they publish, etc.

At its creation, the newspaper was engaged in the fight against "Sarkozysm", to reveal a shadowy part of the French power at that time (2007)(2008)(2009)(2010)(2011)(2012).

"We were anti-Sarkozy, it was Sarkozy who was President. Every day, we were saying bad things about Sarkozy, so the government didn't like it.

There was only hostility... It wasn't fun every day" (Gérard Desportes, cofounder of Mediapart).

The independence of Mediapart and the investigative journalism that embodies this independence, have allowed to reveal a first major political and financial scandal under the "We managed to completely disrupt the national agenda by putting out information that was so much in the public interest that it earned us several trials in France... we had a very strong attack from the State. We were watched, we were robbed, we were insulted"

(Fabrice Arfi).

The investigative journalism on political and economic powers in a "locked information system" has allowed to unpack other cases such as the Libyan financing of the presidential campaign of Nicolas Sarkozy, in 2007.

The person in charge of investigations for the newspaper, Fabrice Arfi, stresses that the spectacular and scandalous nature of these cases should not be overlooked. The scandal and the noise of the scandal are effective means of raising public awareness, learning lessons about corruption, tax evasion, power, etc. and to mark the independence of Mediapart.

In the name of press independence, the founders and journalists have brought to light information, files, scoops, scandals, however they have faced retaliation from attacked actors lying in intimidation, lawsuits and indictments.

Mediapart is also Arfi [journalist]. With what he did on Bettencourt etc... he's smart, he's tenacious, he's brave... the other day he was on a bike when he turned the block a car hit him brushed. I told him not to take a bike, a bicycle accident happened so quickly! (Michel Broue).

Some investigations on French businessmen, French industrial groups and media group have Mediapart's opposition continued after the Sarkozy era, in 2012 François Hollande (socialist party) was elected president of the Republic, six months later, Mediapart released the Cahuzac affair, the budget minister who had been defrauding the taxman for 20 years. Revelations that the newspaper has "almost paid dearly". Since a few months later, the newspaper was subject to a tax audit, followed by a tax adjustment of several million euros.

The founders pointed to a "vengeful attitude" of the administration and the government that the scandal Cahuzac, which led the budget minister to resign.

Performing illegal actions

At the beginning, and this lasted a few years, the public authorities did not consider Mediapart as a newspaper and therefore as a legitimate actor in the field of the written press. As a consequence, the VAT rate applied to Mediapart was 19.6% and not the super reduced rate of 2.1% applied to the press. The question of the super reduced VAT rate, which allows to lower the financial balance rate and increases the independence of a newspaper, especially during its first years of existence, has been transformed later into a political fight led essentially by

Mediapart.

Mediapart has posed as a reformer of the press field. In order to make the law evolve, Mediapart stood up against the law and refused to pay the 19.6% VAT and to pay it at 2.1%.

While informing the tax authorities, of its creation and until 2014, Mediapart had applied the super reduced VAT reserved for the paper press.

For the tax authorities, in order to benefit from the 2.1% VAT, you must be a press company As soon as Mediapart was created, the directors asked for press cards for their journalists and a CPPAP number, but to get it, they needed a printer's number. The CPPAP refused Mediapart's request.

"So we had to fight to get a number from a joint press commission" (Laurent Mauduit).

Mediapart reiterated its request in the CPPAP appeal commission, the administration blocked the agreement because behind this status of press company there is the question of the super reduced VAT of 2.1% of the press or the VAT of 19.6%. At that time the Council of State is seized.

An arm wrestling match between the newspaper and the public authorities, actors/institutions in the field of the press, the tax authorities.

Mediapart is engaged in a legal battle supported by law firms to benefit from the VAT at 2.1%.

The General Direction of Media and Cultural Information has encouraged Mediapart to have a representative, a structure that will be present at the CPPAP, in the commissions for the attribution of subsidies and that will carry the specificities of online press companies. Thus the Union of the Independent Press of online information (SPIIL) was created.

Mediapart launches a media, political and legal battle to benefit from the 2.1% VAT and to obtain a CPPAP number. The SPIIL has played a lobbying role in particular with the General

States of the Press and public authorities, parliamentarians and also in Brussels.

The battle, which according to the founders is a battle for the profession, was won in 2014.

now Mediapart as well as all online press is taxable at 2.1 %.

Meanwhile, in 2013, Mediapart was subject to a tax reassessment of 3.3 million euros, plus 1.3 million in late penalties, while the newspaper acted in full transparency vis-à-vis the tax authorities.

Discussion

In this paper we addressed the question of how peripheral players maintain institutions in their field. Based on an exploratory single case study of the work of Mediapart in the French field of press we've revealed three main findings that answer to our research gap. In the following section we discuss our two contributions to institutional work of maintenance and to institutional theory more broadly.

Our first contribution challenges the structural determinism of institutional theory which restrict institutional work of maintenance to central players. Actors position in the field is an important explanation of their agency in the field [START_REF] Battilana | Agency and institutions : The enabling role of individuals' social position[END_REF] Second, we contribute to exhibit stances of illegality and rebellion in institutional work.

Extant works have portrayed that institutional work is mainly geared toward finding our agreements and seeking cooperation between multiple actors with different interests [START_REF] Hampel | Institutional work : Taking stock and making it matter[END_REF]. They have also pointed to the importance of targeting through discursive and symbolic work, collective cognitive schemas that allow actors to define collectively what is accepted, proper and appropriate in their field [START_REF] Ben-Slimane | The legitimation strategies of early stage disruptive innovation[END_REF]). However, few works explored the use of violence, rebellion in institutional work and how they may be used in defying established institutional orders. Recent works showed that peripheral players and powerless actors when they attack highly guarded institutions don't speak up and act covertly while hiding their actions and intentions (Claus and Tracey 2022). The threat of retaliation has been pointed as an important endeavor for peripheral actors who attack powerful and central players [START_REF] Vaccaro | Values against violence : Institutional change in societies dominated by organized crime[END_REF]. Our results provide a balanced view of the perspective of how powerless actors engage in institutional work geared toward dislodging powerful players and guarded institutions. First, we show that powerful players retaliation strategies may be mitigated by peripheral players and doesn't necessarily hinder them from acting against the established institutions. Mediapart showed resilience after the several attacks by the government and the attempts to unravel their organization. Second, we assume that when peripheral players engage in aggressive institutional work and when they directly attack powerful incumbents, they may enhance their chance of shaping institution in directions they favor.

Recently [START_REF] Roulet | The power of being divisive : Understanding negative social evaluations[END_REF] acknowledges how actors may capitalize on public hostility and negative evaluations in order to harvest positive effects and to achieve their institutional projects. Such strategies have been documented for activists when they use extreme actions often illegal to raise the awareness of audience on a given issue such as Greenpeace and PET (People for Ethical Treatment of Animals) (Roulet 2020).

Our findings show that extreme action such as rebellion, outlaw activities and fueling political scandals are useful to maintain institutions by peripheral players. Such actions are even more successful when they target powerful actors and reveals their misconduct and misbehavior.

Power is indeed ambivalent in that attacks of powerful actors may attract wider audience and receive positive feedback and support as shown by [START_REF] Maguire | Institutional Entrepreneurship in Emerging Fields : Hiv/Aids Treatment Advocacy in Canada[END_REF] on the attacks of big pharma by AIDS patients. We advocate that such extreme actions because they target powerful actors and raise concerns about their hegemony are crucial to the work of maintenance by peripheral and low power players to maintain institutions.

We locate our third contribution in the realm of historical institutionalism [START_REF] Foster | History as social memory assets : The example of Tim Hortons[END_REF].

The historical turn in neo institutionalism stresses how memories may be used as a strategic asset in institutional work. Our findings portray two ways of using memories and historical narratives with the aim of maintaining institutions by peripheral players. The first relates to managing and using reputational trajectories of key figures from history of the field [START_REF] Jansen | Resurrection and appropriation : Reputational trajectories, memory work, and the political use of historical figures[END_REF]. By so doing, peripheral players enhance the meaning of affiliation with key figures which may convey the meaning of them playing role of gatekeepers. Of great importance is how the choice of the figures from the past is made. Based on their reputational trajectories and on the values, they embody, peripheral players pick figures that resonate with the audience memories and convey the meaning they are looking for. This strategy shows how the mechanism of association with figure from the past is conducive to the creation of social meaning in the field.

The use of memory work and historical rhetoric can also be used to convey negative meaning.

The second use of memory work aims at triggering negative emotions such as fear through recalling dark episode form the past. The analogy and the multiple comparisons made with obscure and dark periods in the French history of autocratic political power and corruption sheds light on the role of memory work through triggering emotions such as fear and anxiety in maintaining institutions. We therefore expand the use of history and memories in institutional maintenance beyond the work of mythologisation [START_REF] Lawrence | Institutions and Institutional Work[END_REF] Emotions play an important role institutional work and emotions such as fear had been shown

to play a role in maintaining the power of chef in French cuisine. However, the fear that we relate in this strategy resembles more to strategies of dramatization or constructing scenarios of the worst [START_REF] Vaara | Justification, Legitimization and Naturalization of Mergers and Acquisitions : A Critical Discourse Analysis of Media Texts[END_REF]. The idea is that peripheral players tell the audience that history may repeat itself and the field may slip into dark era.

Following

  [START_REF] Gioia | Seeking qualitative rigor in inductive research : Notes on the Gioia methodology[END_REF] we support the transparency of our data analysis with a figure of the data structure (Fig 1) with an illustration of coding schemes Findings Our analysis shows three kinds of institutional work performed by peripheral players to maintain the institution of their field: historical narrativization and memory work; constructing an alternative model of gathering resources and work of rebellion and opposition.

  figures are well present in the memory of the field and embody the values of freedom and

  interests for the press" (Fabrice Arfi, Journalist and Reporter at Mediapart) Other figures are invoked, each one referring to a particular memory and embodying the values that Médiapart claims. For example, the journalist Robert Ezra Park (1864-1944) is cited for having placed inquiry and investigation at the heart of journalism. While to insist on the values of freedom, independence, opposition, resistance, the founders of Mediapart evoke in their speeches Charles Delescluze one of the figures of the press, the political landscape during the French Revolution. Critical and rebellious, Delescluze opposed the Second Empire, denounced its tools, men and institutions. His publications and positions earned him several trials, fines, prison sentences and exile. He fought for the Republic and democracy for forty years and until his death on a barricade.

  (1852)(1853)(1854)(1855)(1856)(1857)(1858)(1859)(1860)(1861)(1862)(1863)(1864)(1865)(1866)(1867)(1868)(1869)(1870) represents one of the darkest moments of democratic life in France. During this return of the monarchy, the field of the written press was the scene of a mixture of genres where famous businessmen invested "at all costs in the press to better consolidate their influence and maintain relations of connivance with the power". Those close to Napoleon III, the new monarch, bought and put under supervision almost all the newspapers. The press was thus muzzled and the right to information was scorned by the powers of money. According to Laurent Maduit, co-founder of Mediapart, the today state of the French press reminds us of the dark days of the Second Empire. "In a way, we were reliving the fate of the press under the Second Empire. Obliged persons of the Palace acquiring newspapers to please Nicolas Sarkozy [President of France 2007-2012], and buying themselves at the same time a power of influence, in the manner of the Duke of Morny, the half-brother of Napoleon, and of the speculator Jules Mirès, buying up the press in turn to reinforce their speculations, notably around the railroads, and to put it at the service of the Empire " (Laurent Mauduit, co-founder of Mediapart).A second reference to history by Mediapart and which makes a parallel with the current situation of the written press in France is that of the political-financial scandals of the Third Republic. This period was marked by the Panama scandal and the Raffalovitch scandal. The Panama scandal is a case of corruption related to the construction of the Panama Canal. The construction project, which was much riskier and more expensive than competing projects, was selected because of bribes. The raising of public funds, blocked twice by parliamentarians, was made possible by bribing parliamentarians and, above all, the press, which promotes the project and the company that is carrying it to public opinion. The company that led the Panama project went into receivership in 1889, ruining hundreds of thousands of subscribers. At the same time, another financial scandal of the same kind was taking shape: the Arthur Raffalovich affair of Russian loans. A financial affair of corruption of the press, which made it possible to promote the placement of Russian loans, in France at the beginning of the XXth century. The French press, corrupted by more than 25 million Euros, praises the strength of the Russian economy. A "misleading press campaign" promotes the Russian sovereign bonds placed and sold by French banks to constitute a third of French savings. "One of the biggest financial scandals of the time: if nearly one and a half million French people were swindled by these loans, it is because the press was stipendiated to entice them" (Laurent Mauduit).

"

  It was in those years that all the paper titles built low-cost websites that essentially did flow and dispatch, fed by editorial teams of young journalists on precarious contracts, paid by the slingshot, and that's how the Internet sites were run. That's what is used at the time" (François Bonnet, co-founder of Mediapart).

  Sarkozy era: the Bettencourt affair. The Bettencourt affair is the "signal" of independence for Mediapart. It was published by the newspaper in 2010 and refers to links between the Minister of the Budget Eric Woerth and the businesswoman Liliane Bettencourt, the largest shareholder of the L'Oreal group and one of the first fortunes of France. The newspaper published authenticated recordings that shed light on conflicts of interest between Liliane Bettencourt and the Minister of the Budget and a suspicion of illegal financing of Nicolas Sarkozy's presidential campaign (2007).

  earned the newspaper several legal attacks. For instance they have been suited after revealing the Caisses d'Epargne (private bank) affair, as Mediapart pointe to "an illegal taking of interest" and questions decisions taken at the top of the State in favour of the private bank. «We still live in a universe that is quite hostile or sometimes even criminalized. Me in my investigations on the Caisse d'Epargne in 2010, I was the subject of 12 defamation complaints and I was under examination 12 times. Eventually I'll win the case and we'll have the bank convicted of malicious prosecution. Plenel, as he is director of publications, it is even more frequent" (Laurent Mauduit)." Mediapart, then barely a year old and financially fragile, launched an appeal for financial support to deal with the 12 libel suits filed against it by the Caisses d'Epargne. « Our determination is intact, but we need you. Pot of clay against pot of iron, the disproportion of financial means is obvious here… you can therefore support Mediapart financially…" (Edwy Plenel)"

  and have a number of the Commission Paritaire des Publications et Agences de Presse (CPPAP). The CPPAP is a body composed of equal representation of the State administration and press professionals. It was in charge of : Issuing an opinion relating to the benefit of the economic regime of the press (privileged postal and tax rates) of the publications; proposing the registration on the list of the companies having the status of press 2 agency.

  . Scholars have shown how different roles in bringing about institutional change or in insuring institution continuity are played by actors according to their central or rather peripheral position in the field.This structural determinism builds on an early assumption within institutional theory that emphasize that degree of socialization of actors in the field and their adherence to institution is high for central players and incumbents and low for challengers and peripheral players(Leblibici et al. 1991). Strategic arguments had also been brought to support his theoretical underpinning arguing that central players maintain institutions because their interest are favoured by the status quo(Vaccaro and Palazzio 2015) and because they enjoy an easy access to resources.Peripheral players are therefore deemed less socialized[START_REF] Croidieu | Complex field-positions and non-imitation : Pioneers, strangers, and insulars in Australian fine-wine[END_REF]) and lack resources to maintain institutions. Few works tried to challenge this structural determinism by showing how central players may bring about changes in their field and deviate from established norms[START_REF] Rao | Institutional Change in Toque Ville : Nouvelle Cuisine as an Identity Movement in French Gastronomy[END_REF]. However, extant literature is still puzzled by considering how peripheral players could maintain institutions. A key question deriving from this assertion is how peripheral players may substitute and subvert resources and power in their institutional work of maintenance.DiMaggio (1988) in his early conceptualisation of strategic action that aims at shaping institutions highlighted the importance of being a "resourceful actor". However, despite the prominence of this concept there was few efforts to study how actors in the field define what resources are and how their substitution by other resources may unfold in the field[START_REF] Bourdieu | Raisons pratiques[END_REF].Our findings show that rather than mobilizing and using extant resources of the field to perform strategic action and institutional work of maintenance, peripheral actors commit themselves into an undocumented work of redefining critical resources of the field. They achieve that in two ways. First, they infuse exiting resources with a negative meaning and work on their delegitimization. In their rhetoric they theorize a causality relationship between the vey kind of resources used in the field and the institutional disorder and entropy that characterises the field. Resources appear therefore as a core component of institution and institutional order rather than a generic dimension and a mere instrument of action. While studying institutions and institutional order scholars should pay attention to define what are the resources that allow institution to work and how players shape their meaning and their definition.Pierre[START_REF] Bourdieu | Raisons pratiques[END_REF] argues that social actors have multiple forms of capital (social, economic and cultural) and these different kinds of capital may be substituted by others depending on the social context. For instance, economic capital may be a substitute to social capital. Our findings are consistent with this theory of the substitutability of capital, here resources, as maintenance work by peripheral players brings new kind of resources. Our data shows how Mediapart substituted the extant resources in the filed of press in France (advertisement, public subsidies and private investments) by new resources (crowdfunding and the monetization of online information). This institutional work of substitution is vital for the institutional work of maintenance by peripheral players. Their lack of access to extant resources should lead them to bring new kind of resources. The dynamic of resource substitution should attract more attention from scholars interested in studying the dynamics of institutional continuity and change.

http://www.cppap.fr/presentation-de-la-cppap/


Cédérick Mouliets
email: cederick.mouliets@univ-tlse3.fr

Isabelle Ferrané
email: isabelle.ferrane@irit.fr

Heriberto Cuayáhuitl
email: hcuayahuitl@lincoln.ac.uk

Reward-Based Environment States for Robot Manipulation Policy Learning

Training robot manipulation policies is a challenging and open problem in robotics and artificial intelligence. In this paper we propose a novel and compact state representation based on the rewards predicted from an image-based task success classifier. Our experiments-using the Pepper robot in simulation with two deep reinforcement learning algorithms on a grab-and-lift task-reveal that our proposed state representation can achieve up to 97% task success using our best policies.

Introduction

Teaching a robot new skills easily, quickly and reliably is one of the main goals of robotics and AI. In that way, robots could be of great assistance at home (for example carrying out cleaning or cooking tasks), in factories (carrying out assembly or quality check tasks during production of products), or in public spaces (for instance serving food/drinks or playing games), among other scenarios [START_REF] Akalin | Reinforcement learning approaches in social robotics[END_REF][START_REF] Cuayáhuitl | A data-efficient deep learning approach for deployable multimodal social robots[END_REF][START_REF] Kroemer | A review of robot learning for manipulation: Challenges, representations, and algorithms[END_REF]. Some of the requirements for such an ambitious scientific and engineering goal include understanding robot commands from a human instructor, understanding of the environment and task, acquisition and processing of visual (and audio) perceptions, characterisation of the learning task, ability to select actions for achieving the task, and ability to measure performance, among others. Those requirements and the fact that the world situations are always different, make the problem of skill acquisition of a tall order. The fields of Deep Learning and Reinforcement Learning have been suggested as a promising framework to enable the requirements above. This implies the creation of an agent with a concrete set of states, actions, rewards, and a simulator or logged data. Training a robot means finding the policy that maximises the long-term reward signal via trial-and-error search [START_REF] Liu | Deep reinforcement learning for the control of robotic manipulation: A focussed mini-review[END_REF][START_REF] Zhang | Reinforcement learning for robot research: A comprehensive review and open issues[END_REF].

The learning scenario of focus in this paper is illustrated in Figure 1, where the Humanoid robot Pepper has to carry out a manipulation task in a simulated environment. We focused particularly on a classic task in robotics: grab-and lift. The robot has to grab an object and lift it without dropping it by using reinforcement learning. The novel aspect in this architecture is the use of compact environment states. In contrast to previous works that use states with learnt features from visual images [START_REF] Singh | End-to-end robotic reinforcement learning without reward engineering[END_REF][START_REF] Mohtasib | A study on dense and sparse (visual) rewards in robot policy learning[END_REF] (not as compact as ours), we use the last N =15 rewards to represent our proposed states. To our knowledge, this approach has not been studied before. Intuitively, those reward-based states can be seen as describing a sequence of task success scores-where the scores grow as the task progresses from a time step to the next.

NeurIPS Workshop on Deployable Decision Making in Embodied Systems, 2021 Visual reward learning has been considered as a promising solution for equipping robots with learnable manipulation abilities [START_REF] Singh | End-to-end robotic reinforcement learning without reward engineering[END_REF][START_REF] Mohtasib | A study on dense and sparse (visual) rewards in robot policy learning[END_REF][START_REF] Hundt | Good Robot!": Efficient reinforcement learning for multi-step visual tasks with sim to real transfer[END_REF]. This approach measures the success of experiences via reward in robotic tasks by estimating the probability of success from realistic input images [START_REF] Kumra | Learning robotic manipulation tasks through visual planning[END_REF]. While [START_REF] Singh | End-to-end robotic reinforcement learning without reward engineering[END_REF] focus on standard Convolutional Neural Networks (CNNs), [START_REF] Mohtasib | A study on dense and sparse (visual) rewards in robot policy learning[END_REF] extend CNN-based architectures with additional paths on the neural network to add more information for better predictions and therefore calculating visual rewards more reliably. These works tested DRL (Deep Reinforcement Learning) agents in classic tasks using a variety of reinforcement algorithms such as DDPG, TD3 or SAC. To do so, different dense and sparse rewards have been investigated, where dense rewards have been found as a more promising alternative than sparse rewards.

Previous works in DRL for robot manipulation such as [START_REF] Finn | Learning visual feature spaces for robotic manipulation with deep spatial autoencoders[END_REF][START_REF] Lillicrap | Continuous control with deep reinforcement learning[END_REF][START_REF] Fujimoto | Addressing function approximation error in actor-critic methods[END_REF][START_REF] Singh | End-to-end robotic reinforcement learning without reward engineering[END_REF] represent environment states mostly based on image-based features, where raw images are digested by CNNs in order to learn compact representations that can be used for policy learning. But the idea of using rewards as environment states or part the environment states has not really been studied. This highlights the need for a comparison of different state representations, which we identify as a research gap in this paper. In our case, we focus our study on comparing image-based against reward-based state representations.

Research Methods and Results

Methodology

We study the overall problem of how to train robot manipulation policies using reinforcement learning in a simulated environment. Given a state-of-the-art algorithm and a defined set of continuous actions, we adopted a reward function based on the success classifier approach as in [START_REF] Singh | End-to-end robotic reinforcement learning without reward engineering[END_REF][START_REF] Mohtasib | A study on dense and sparse (visual) rewards in robot policy learning[END_REF]. To study our proposed state representation, we relied on the following methodology.

1. Create an environment for the grab-and-lift task.

2. Create a dataset of success and non-success images.

Train a probabilistic task success classifier (to be used as a reward function).

4. Define reward-based states and action representations. 5. Determine the policy learning algorithm with support for continuous actions.

6. Train policies using the states, actions and rewards specified above.

7. Test the learnt policies using evaluation metrics such as avg. reward and avg. task success. We used the qiBullet robot simulation environment [START_REF] Busy | qiBullet, a bullet-based simulator for the pepper and NAO robots[END_REF], which is powered by the Bullet physics engine and the PyBullet real-time physics simulation framework. To create our simulation environment, we used Unified Robot Description Format (URDF) files for creating cube and table objects as illustrated in Figure 1-the task environment component. To make the robot interact with the cube objects, we specified friction, dampening and stiffness coefficients in the URDF files and put low weights (100g) so the cubes can be lifted. The images for visual perceptions are generated by the 2D camera on the mouth of the robot with resolution K QV GA =(320,240).

PCA and image embeddings enable the significant reduction of amount of information contained in an image. While with PCA we reduced the amount of features to 50 components, with image embeddings we obtained vectors of 2048 dimensions.

Our learnt policies take on average 4:45 hrs to train and Table 1 shows their performance out of five runs. While at first sight it looks like TD3 outperforms DDPG in terms of average rewards, that is not the case in terms of average task success. The best DDPG policy obtained a task success of 97.04% and the best TD3 policy obtained 96.78% of task success as shown in Figure 2. Thus, both learning algorithms can achieve similar performance-but their best policy must be found from multiple runs. The policies were trained and tested for 10000 time steps. If the robot could not grab the cube after 50 actions, we considered the task as a failure and started over another episode as shown in Fig 2 .  Manual inspection showed that when an agent finishes episodes sooner than later, it obtains less rewards. Therefore, Average Task Success is a more reliable metric in our case. Our best policy uses DDPG with states including the last 15 rewards-a reward-based representation and the most successful in our comparative study. See episode examples in Figures 3 and4 with successive steps.

Figure 1 :

 1 Figure 1: Cycle of the proposed learning agent

Figure 2 :Figure 3 :Figure 4 :

 234 Figure 2: Training and testing of TD3 with 160*120 images and 50 actions a step

Visual Reward Classifier and Policy Learning

Instead of manually engineering a reward function, we use learnt rewards from visual observations. To do so, we employ an image-based binary classifier to predict task success (CNN-based). The goal is to generalise situations on the simulation environment in a data efficient manner. From the front camera, the robot only sees two cubes on a table. 'Success' images show the robot's hand with the cube grabbed, from different angles, in front of the camera, while 'Non-Success' images show the cube outside the robot's hand. We randomised the arm positions (on or away from the cube) and generated a total of 6400 labelled images. This dataset of images was created from 10 balanced capture sessions, each session consisting of 200 success images and 440 non-success ones. From one capture session to another, we modified the robot's right arm to introduce variability in its work space. Non-success images are more present because they are more likely to be encountered. The neural architecture of our reward predictor is based on Tensorflow's image classifier 1 .

To learn manipulation policies, we experimented two actor-critic algorithms for decision-making with continuous actions: DDPG [START_REF] Lillicrap | Continuous control with deep reinforcement learning[END_REF], and TD3 [START_REF] Fujimoto | Addressing function approximation error in actor-critic methods[END_REF]. We used the Stable-Baselines3 framework 2 with default hyperparameters to implement our policies and experimented different state representations. Regarding actions and since the proposed policies only use the right arm of the robot, we focused on four joints: RShoulderPitch, RElbowRoll, RHand, and RShoulderRoll. Those actions are generated by rotating at a constant speed the angle of each corresponding joint between 0°and 180°. A typical learning episode from Figure 1 is as follows. (A) The robot captures an image of the current situation from the simulated environment. (B) This image is sent to the pre-trained classifier which outputs a confidence score from its last Softmax layer. This score will be considered both as the reward as well as a part of the state representation. (C) Finally and via a deep reinforcement learning algorithm, the agent will choose an action that combines the four joints previously mentioned.

Reward Classification and Manipulation Results

We split our dataset of success/non-success images (6400 in 10 demos) into three subsets: 8 demos for training, 1 demo for validation and 1 demo for testing. We experimented different configurations in terms of number of epochs and batch size. Our results report that all of our 6 classifiers achieve high performance in terms of classification accuracy (98% average accuracy on test data-set) and we selected our best model for predicting visual rewards. Each classifier provides a confidence score of the success of the task which implies that the closer the arm is to the cube, the greater the confidence. Each classifier has been trained and tested using the same dataset split. The high performance is presumably due to the much simpler simulation environment than the real world. The classifier prediction time per image varied between 5.2 and 5.6 milliseconds 3 .

We trained DDPG policies with six different state representations and compared against their corresponding TD3 policies. These state representations include three different image resolutions (320x240, 160x120, 80x60), Imagenet-based pre-trained embeddings 4 and PCA-based (Principal Component Analysis) images with 50 components [START_REF] Pedregosa | Scikit-learn: Machine learning in Python[END_REF]. Our goal was to improve them using our proposed compact representation based on the last 15 visual rewards (predicted from 160x120 images).


Salama Shady
email: shady.salama@kaede.cs.kobe-u.ac.jp

Toshiya Kaihara

Nobutada Fujii

Daisuke Kokuryo

Automatic Design of Dispatching Rules with Genetic Programming for Dynamic Job Shop Scheduling

Keywords: Dynamic job shop, Scheduling, Dispatching rules, Hyper-heuristic, Genetic programming

Traditionally, scheduling experts rely on their knowledge and experience to develop problem-specific heuristics that require a considerable amount of time, experience, and code effort. Through this tedious process, experts must follow a trial-and-error cycle by evaluating the generated rules in a simulation model for the problem under consideration until achieving satisfactory results. Recently, hyper-heuristic approach has emerged as a powerful technique that uses artificial intelligence to automatically design efficient heuristics for various optimization problems. Genetic programming (GP) is the most popular hyperheuristic approach to automate the design of production scheduling heuristics. In this paper, a genetic programming framework is proposed to generate efficient dispatching rules in a dynamic job shop. The proposed framework integrates the reasoning mechanism of GP with the ability of discrete event simulation in analyzing the performance of generated rules under dynamic conditions. Afterward, the evolved heuristics are compared to human-tailored literature rules under different dynamic settings using mean flow time and mean tardiness as performance measures. The achieved results prove the ability of the proposed approach in generating superior scheduling rules rapidly, within a few hours, compared to the conventional literature rules commonly adopted in the industry.

Introduction

Job Shop Scheduling Problem (JSSP) is a well-studied problem in combinatorial optimization literature that has proven to be NP-hard. JSSP can be defined as there are a number of jobs and each job has a predefined set of operations (tasks) that need to be processed for specific processing time on a limited number of machines (resources) while optimizing some performance criteria [START_REF] Pinedo | Scheduling: theory, algorithms and systems development[END_REF]. Due to the complex nature of the JSSP, several solution approaches have been developed to provide satisfactory solutions for both static and dynamic scenarios. Although many solution approaches are proposed for solving JSSP, most of these approaches mainly focus on relatively small problem instances or moderate to large instances with over-simplified assumptions. Dispatching Rules (DRs) have been widely employed for solving Dynamic Job Shop Scheduling Problem (DJSSP) in a practical time due to its flexibility, scalability, and fast response to real-time events. There are several comparative studies in the literature for analyzing the performance of traditional scheduling rules under different conditions, for example, reference [START_REF] Rajendran | Comparative study of dispatching rules in dynamic flowshops and jobshops[END_REF] and [START_REF] Sels | A comparison of priority rules for the job shop scheduling problem under different flow time-and tardiness-related objective functions[END_REF]. The common conclusion in these studies is that human-designed rules are developed to address specific job shop settings using a certain objective function, and usually perform poorly under other system configurations or performance measures. Moreover, literature rules experience a myopic nature by considering only local and current shop conditions that lead to poor quality solutions compared to global optimization methods.

In order to tackle these limitations, many scholars have used Artificial Intelligence (AI) techniques to automatically develop competitive scheduling rules that are trained under different scenarios. In recent years, Genetic Programming (GP) has proven its superiority compared to other AI techniques to generate DRs for different production scheduling problems [START_REF] Nguyen | Genetic programming for production scheduling: a survey with a unified framework[END_REF]. Miyashita [5] is perhaps the first study that proposed GP approach to evolve DRs for job shop scheduling problems using a multi-agent model. Geiger et al. [START_REF] Geiger | Rapid modeling and discovery of priority dispatching rules: An autonomous learning approach[END_REF] employed the GP approach to obtain high-quality DRs for a singlemachine scheduling problem. After evaluating the generated rules against 6 DRs from the literature, the generated rules revealed competitive results in minimizing completion time, total tardiness, and maximum tardiness. In the same context, the authors in [START_REF] Jakobović | Dynamic scheduling with genetic programming[END_REF] employed a decision tree to classify the available machines in a job shop as bottlenecks or not, then the GP approach is used to generate scheduling heuristic on a given machine based on its classification. The evolved rules provided better performance compared to seven literature rules. In [START_REF] Jakobović | Evolving priority scheduling heuristics with genetic programming[END_REF], single-machine and static job-shop scheduling problems are considered. For the single machine problem, the authors examined the ability of the GP approach under both static and dynamic conditions as well as with or without setup times. While for JSSP, the GP approach used to minimize weighted job tardiness and total makespan. Their findings are consistent with previous studies in supporting the superiority of GP rules against literature rules. A twostage hyper-heuristic is suggested in [START_REF] Pickardt | Evolutionary generation of dispatching rule sets for complex dynamic scheduling problems[END_REF] by adopting the GP approach to generate DRs and an evolutionary algorithm to assign the developed rules to different work centers. In [START_REF] Hunt | Evolving machine-specific dispatching rules for a twomachine job shop using genetic programming[END_REF], the authors investigated the use of GP hyper-heuristic approach to evolve DRs for a two-machines job shop in both static and dynamic conditions. For the dynamic case, two GP representations were examined, including a single rule in both machines and evolving a specific rule for each machine. The GP approach achieved the optimal makespan in the static problem and good results in the dynamic problem.

The aim of this paper is to propose a GP framework for generating competitive DRs for DJSSP under different system scenarios. There are different sources of randomness considered in this study, such as job arrival, operations processing time, the number of operations per job, operation-machine compatibility, and due dates. To verify the effectiveness of the proposed framework, the GP evolved heuristics are compared to 12 human-tailored rules in minimizing mean flow time and mean tardiness under various dynamic settings. The remainder of this paper is organized as follows. The proposed framework is presented in Section 2. Also, the experimental details are given in Section 3. Section 4 provides a summary of the obtained results. Conclusions and some suggestions for future work are addressed in Section 5.

Genetic Programming Framework

The proposed framework consists of two main modules, a GP-based reasoning module, and a performance evaluation module to automatically generate dispatching rules for DJSSP. As shown in Figure 1, the GP module starts by generating an initial population of candidate heuristics using predefined functions, terminals (attributes), and a specific representation. In this study, we adopted a tree-based representation which is the most popular data structure to represent any priority function. Moreover, it is worth noting that there are three random techniques widely used to create the initial population, namely grow, full, and ramped-half-and-half. The grow method generates trees in variable sizes, while all trees have the same shape and size using the full method. Ramped-half-and-half combines both the full and the grow methods to produce a diverse population of trees with various shapes and sizes.

Afterward, the solution quality of generated heuristics will be assessed in the performance evaluation module using a Discrete Event Simulation (DES) model. The DES model includes a pre-defined set of training instances and a meta-algorithm. The meta-algorithm encapsulates the GP evolved rule and possible system constraints to produce valid schedules for a specific scheduling environment. The Giffler & Thompson (GT) meta-algorithm is widely used to create active and non-delay schedules [START_REF] Pinedo | Scheduling: theory, algorithms and systems development[END_REF]. The employed non-delay GT algorithm can be briefly described as follows. While there are unprocessed operations and there is an idle machine, the GT algorithm calculates the priority value for queued operations and schedules the operation with the highest priority. Then, the evolutionary process starts by selecting fit individuals from the population. The selection process determines which individuals can survive (copy the fittest heuristics to the new population) and which individuals are allowed to reproduce (apply genetic operations). After selecting the mating pool, genetic operators are employed to combine existing individuals (crossover operator) or introduce diversity (mutation operator) in order to form offspring for the next generation. The population evolves over a specified number of generations until the stopping criteria are satisfied. Finally, the algorithm terminates, and the best dispatching rule is obtained. Numerical Experiments

The Genetic Programming Module

For the sake of evaluating the effectiveness of the proposed GP framework, it is used to generate competitive dispatching rules in minimizing mean flow time and mean tardiness for DJSSP. Then, the fittest rule for each objective is compared with 12 human-made rules under different system configurations. Tables 1 and2 demonstrate the set of terminals and functions used in the GP module to create priority functions. The first part of the terminal set represents the local job shop attributes, while the second part refers to the global attributes. The ramped-half-and-half method is employed to generate the initial population with a maximum tree depth of 12. After executing multiple pilot experiments in order to fine-tune the GP parameters, the parameters presented in Table 3 are approved. The maximum number of generations is set to 50 generations each with a population size of 1000 dispatching rules. Using a double tournament selection technique, the GP individuals must pass two layers of tournaments, one by fitness and the other by size in order to be chosen. This selection procedure helps in mitigating the bloating effect which usually slows down the evolutionary search process, consumes extra memory, and restricts effective breeding [START_REF] Luke | Fighting bloat with nonparametric parsimony pressure[END_REF]. Afterward, genetic operations are employed to create offspring. One-point leaf biased crossover is used to create children by randomly selecting a crossover point (with 10% probability to choose terminal nodes) in each individual and the parts below crossover points are exchanged. Also, uniform mutation operator randomly selects a point in an individual and replaces the subtree at this point with a randomly generated tree (with tree depth equals three). The best 5% of individuals are copied across generations to prevent the fittest heuristics from getting lost during the evolution process.

The Performance Evaluation Module

A DES model for a commonly used symmetrical job shop was developed to assess the steady-state performance of the evolved heuristics [START_REF] Nguyen | Genetic programming for production scheduling: a survey with a unified framework[END_REF]. The job shop has 10 machines and each machine can process only one operation at a time. The interarrival times between jobs are generated using an exponential distribution, and each job consists of a random number of operations between 2 and 14 operations. The processing time for each operation is randomly distributed U [START_REF] Pinedo | Scheduling: theory, algorithms and systems development[END_REF]49] and U [START_REF] Pinedo | Scheduling: theory, algorithms and systems development[END_REF]99]. The job due date is assigned using the total work content method, i.e. job due date = job release date + (allowance factor × total job processing time). The model is analyzed under three levels of utilization, including 70%, 80%, and 90%. In each simulation replication, the shop starts empty. Then, the obtained data between the beginning of the simulation until the arrival of the 1500th job is discarded, and the statistics are collected from the 1500th job to the next completed 5000 jobs. The triplet (p, u, c) represents the system settings by defining the average processing time, the job utilization level, and the tightness factor (to determine the tightness of job due date). Changing the values of "p" and "u" significantly affects the mean flow time, while varying the "c" values greatly affects the mean tardiness. The training and testing scenarios are illustrated in Table 4. Two simulation scenarios are employed to train the evolved rules in minimizing mean flow time and mean tardiness. Also, the 6 testing scenarios for each objective are used to compare the performance of the GP evolved rules with 12 rules from the literature. The benchmark rules are described as follows; SPT: Shortest Processing Time, LPT: Longest Processing Time, FIFO: First In First Out, LIFO: Last In First Out, WINQ: Work In Next Queue, SL: Slack, NSL: Negative Slack, SPT + WINQ + SL, MWKR: Most Work Remaining, COVERT: Cost Over Time, EDD: Earliest Due Date, and MOD: Modified Due Date, for more details please refer to [START_REF] Sels | A comparison of priority rules for the job shop scheduling problem under different flow time-and tardiness-related objective functions[END_REF]. Table 4. Simulation configurations for the training and testing sets.

Parameters

Training Testing Processing time U [START_REF] Pinedo | Scheduling: theory, algorithms and systems development[END_REF]49] and U [START_REF] Pinedo | Scheduling: theory, algorithms and systems development[END_REF]99] U [START_REF] Pinedo | Scheduling: theory, algorithms and systems development[END_REF]49] and U [1, 99] Shop utilization 70% and 90% 70%, 80%, and 90% Tightness factor 4

Results and Discussion

The proposed framework was executed on a PC with 2.7GHz CPU and 16 GB RAM. The GP module was trained on the training set for minimizing the mean flow time and the mean tardiness. Then, the fittest GP rule was compared with the 12 human-tailored rules on the 6 testing scenarios for each performance indicator. For the sake of convenience, we referred to the best GP rule for the mean flow time as FGP, and the best GP rule for mean tardiness as TGP. Tables 5 and6 show the mean flow time and mean tardiness for each rule on different problem instances. Regarding the mean flow time, it is obvious that the evolved GP rule outperforms the literature rules for all the test instances. The relative change was calculated using the following equation:

𝑅𝑒𝑙𝑎𝑡𝑖𝑣𝑒 𝑐ℎ𝑎𝑛𝑔𝑒 = 𝑃 𝑟𝑒𝑓. -𝑃 𝐺𝑃 𝑃 𝑟𝑒𝑓. * 100
Where 𝑃 𝑟𝑒𝑓. denotes the performance value of the best literature rule in a specific problem instance, and 𝑃 𝐺𝑃 denotes the performance value of the GP rule for the same instance. As shown in Table 5, the performance of the FGP dominates the best litera-ture rule, and the average relative change is 5.85%. Moreover, the gap between the performance of FGP and the best literature rules in each instance widens by increasing the system usage and processing time interval i.e. when the problem instance becomes more complex and dynamic. The FGP rule was analyzed to understand how this role works and what parameters have the highest impact on the mean flow time.

We noticed that all the global features are used in the evolved rule, while only some local features are considered. This finding manifests the importance of global attributes in mitigating the myopic nature of scheduling rules. Also, the top 4 features used in descending order based on the number of occurrences are the operation processing time, the work in the next queue, the processing time of the next operation, and the number of remaining operations within the job. This observation is consistent with the superiority of SPT and WINQ in being the best literature rules which proves the reasoning ability of GP to choose the most important job shop attributes for a specific objective. Excluded features include job release date, due date, and operation waiting time. Using these results, we concluded that the excluded features do not have a significant impact on average job flow time, which explains the disappointing performance of LPT, FIFO, and LIFO rules in reducing the mean flow time.

Regarding the mean tardiness, the TGP showed substantially better performance compared to the best literature rule with an average relative change of 45.56%. Also, the superiority of the GP approach was revealed as the job shop becomes more random, and due dates become tighter, as shown in case 1 compare to case 6. After investigating the features employed in the TGP rule and sorting them in descending order of importance, the following results were obtained. The most significant attributes for minimizing the mean tardiness are operation processing time, work in the next queue, number of operations within the job, and job due date. Furthermore, we gained a solid understanding of why the TGP achieved the best results, as well as why MOD, SPT+WINQ+SL, and SPT are the top literature rules in minimizing the mean tardiness. Similar to the FGP rule, all the global features are included in the TGP rule which demonstrates its great impact on enhancing the performance of DRs. This paper proposed a GP framework to automatically generate scheduling policies in dynamic job shops. The main building blocks of this approach were presented with a detailed explanation of the internal mechanism. Also, the proposed approach was employed to minimize mean flow time and mean tardiness under different dynamic scenarios. The experimental results demonstrated the ability of the proposed framework to automatically design effective dispatching rules with respect to the considered performance measures. Moreover, the GP reasoning mechanism was able to select the most significant attributes to include them in evolved heuristics and exclude insignificant attributes. In future research, the proposed approach will be extended to generate scheduling rules that can handle multiple objectives. Also, the framework is expected to be evaluated using real-world data to assess its practical effectiveness.

Fig. 1 .

 1 Fig. 1. The genetic programming framework

  / (a, b) returns "1" if b = 0, else "a/b" IF ternary IF (a, b, c) returns "b" if a ≥ 0, else "c" MIN MIN (a, b) returns "a" if (a < b), else "b" MAX MAX (a, b) returns "a" if (a > b), else "b"

  70, 3), (50, 70, 3), (25, 80, 4), (50, 80, 4), (25, 90, 5),(50, 90,[START_REF] Miyashita | Job-shop scheduling with genetic programming[END_REF]

Table 1 .

 1 The terminal set for the GP module.

	Node Name	Description
	rJ	The release date of job (j)
	RJ	The operation release date
	RO	Number of remaining operations within the job
	RT	Work remaining of the job
	PR	Operation processing time
	DD	Job due date
	CT	Current time (Machine ready time)
	WT	Current waiting time of the job (t -RJ)
	#	Ephemeral constant (Random number between 0 and 1)
	NPT	Processing time of next operation
	WINQ	Work in the next queue
	APR	Average operation processing time of jobs in the queue

Table 2 .

 2 The function set for the GP module.

Table 3 .

 3 The GP parameters.

	Parameter	Description
	Initialization	Ramped-half-and-half
	Crossover rate	85%
	Mutation rate	10%
	Elitism	5%
	Maximum depth	12
	Number of generations	50
	Population size	1000
	Selection	Double tournament selection (size = 5)

Table 5 .

 5 Performance of all rules for mean flow time.

	Rules	Case 1	Case 2	Case 3	Case 4	Case 5	Case 6
	SPT	394.2	792.58	526.28	1039.33	862.2	1687.42
	LPT	636.37	1265.64 1073.51 2153.54 2634.44	5184.4
	FIFO	482.54	961.23	689.75	1375.78 1278.46 2489.33
	LIFO	479.55	945.38	676.84	1357.58 1266.19 2478.04
	WINQ	437.35	870.25	598.22	1193.64 1032.13 2018.64
	SL	457.15	906.1	654.36	1292.04 1233.28 2409.71
	NSL	476.98	948.80	660.03	1323.17 1232.90 2389.51
	SPT + WINQ + SL	440.3	888.38	631.62	1273.64 1210.19 2360.26
	MWKR	567.54	1116.51	899.80	1761.97 1975.12 3847.52
	COVERT	479.79	965.82	679.78	1371.5	1259.02 2483.99
	EDD	454.16	897.46	644.57	1279.12 1225.65 2383.55
	MOD	449.96	897.08	628.36	1247.36 1049.82 2043.90
	FGP	383.45	775.87	502.04	995.8	762.6	1519.96
	Relative change (%)	2.73	2.11	4.61	4.19	11.55	9.92

Table 6 .

 6 Performance of all rules for mean tardiness.

	Rules	Case 1	Case 2	Case 3	Case 4	Case 5	Case 6
	SPT	29.79	57.91	64.09	125.49	248.16	470.32
	LPT	188.51	375.65	451.01	919.35	1780.28 3489.27
	FIFO	45.02	86.82	85.75	166.14	393.78	731.4
	LIFO	87.68	170.15	177.27	356.3	598.27	1153.94
	WINQ	38.43	74.01	71.44	137.53	275.59	510.57
	SL	19.58	38.57	39.14	71.87	306.65	562.62
	NSL	24.04	46.03	39.8	79.03	282.02	524.05
	SPT + WINQ + SL	15.95	32.12	33.58	63.39	288.09	526.25
	MWKR	151.2	291.25	343.15	667.36	1222.67 2344.97
	COVERT	42.71	84.99	82.73	161.73	381.02	727.08
	EDD	23.17	44.05	42.45	81.3	308.73	575.72
	MOD	21.78	42.21	38	72.52	240.47	448.61
	TGP	12.13	23.7	16.71	32.23	91.86	170.41
	Relative change (%)	23.94	26.20	50.24	49.16	61.80	62.00