Informing Decisions for Sustainable Development Map of the Lao Uplands

1 1 Development challenges in the Lao Uplands

Changing uplands -changing vulnerabilities

Over the past decade the Lao People's Democratic Republic (Lao PDR) has experienced a rapid economic growth with average Gross Domestic Product (GDP) growth reaching 7.8% per year thanks to intensive use of the country's natural resources through mining, forestry, and hydroelectric power (illustration). Today, Foreign Direct Investments on hydropower and mining represent one third of Lao economic growth whereas agriculture counts for only 15%. These large scale investments are essentially made in the upland areas, which make up about 75% of the national territory. More recently, regional integration of Lao PDR through China's Belt and Road Initiative and the Regional Comprehensive Partnership ASEAN emerged as a key driver of change, with major infrastructure expansion (e.g. railway and highways) underway at the national and regional levels.

Agriculture dominates household economies and livelihoods in the Lao uplands, although prevailing levels of labor productivity are low. Between 2000 and 2014, agricultural growth averaged only 3.4 percent per year mainly driven by an expansion in land under cultivation rather than through productivity improvements. Indeed, most upland farming households are engaged in subsistence cultivation, with less than 30 percent of farm households reporting to produce primarily for sale in 2011. While more than 1 million ha have been given over to agricultural land concessions such an alternative model of agriculture development has not noticeably contributed to sector growth, improved employment or productivity. The 'turning land to capital' policy implemented since 2007 was expected to boost agricultural productivity thanks to modern, so-called scientific, agricultural techniques that would be brought in by investors in charge of developing the land allocated under concession. But in most cases, the expected jump in productivity did not eventuate despite land use intensification. In many cases, the high use of chemical inputs and mechanical tillage led instead to land degradation, soil and water pollution, as well as threats to human health (e.g. banana, sugarcane, rubber plantations). Furthermore, the commodities produced have been exported to neighboring countries as raw agricultural products leaving very little added value to the local economy and increasing the dependence of upland farmers on external markets.

The benefits of these changes for local populations have been relatively limited as uplands livelihoods are predominantly agrarian and largely disconnected from these externally-driven development trends. However, many upland communities were impacted by the 'turning land into capital' model of development as they have been displaced or excluded from their traditional lands to make space for large projects and economic concessions. In general, the economic growth has not been inclusive and disparities have increased over time especially between upland and lowland areas [START_REF] Nier | Mindful Markets: Producer-Consumer Partnerships towards a New Economy[END_REF].

Although the country has achieved a national surplus in rice production, nearly half of Lao children under five years of age are chronically malnourished (stunted or height-for-age). The incidence of stunting is particularly high in uplands communities. Little progress has been observed over the past decade as 44% of children under the age of five are affected today by insufficient access to diverse foods as compared to 48% in the early 2000s. Upland areas still lag behind in terms of physical connectivity, access to water for domestic consumption and sanitation, and access to social services, education and economic opportunities. Despite good results of poverty reduction policies with absolute poverty falling from 34% to 23% between 2003 and 2013, the upland population remains vulnerable to frequent shocks in the absence of effective risk mitigation measures (WFP, 2015).

Vulnerability refers to the degree to which a system is unable to cope with, or adapt to, the negative effects of external shocks. Upland communities have long been exposed to climate disasters including damaging storms and droughts and have developed adaptive strategies over the centuries. But today, climate change is adding to the vulnerability by changing weather patterns, resulting in more frequent and severe events. Gradual integration to the market economy adds new risks related to market failures, unfair farming contracts, and price fluctuations undermining the capacity of upland communities to manage the risks of natural disasters and build more resilient livelihoods. Crop pests (including recent locust outbreaks) and livestock diseases are recognized by farming households as the most threatening hazards to their traditional livelihoods. However, these threats often are the symptoms of entrenched environmental changes related to land degradation, biodiversity loss, etc. that undermine agricultural productivity, increase risks and weaken farming-based livelihoods.

Quick facts

• 2/3 of the total area of Lao PDR is mountainous with poor infrastructure and socio-economic development is heavily dependent on agriculture.

• 1/4 of the total population in Lao PDR lives in mountainous areas and is exposed to new vulnerabilities related to climate change and market uncertainties.

• 1/3 of the population in upland areas is still below the poverty line and face malnutrition and marginalization.

• Doing more with less. Less ODA support after LDC graduation, combined with the reduction of civil servant numbers will require a reorganization of intervention mechanisms.

• Competing visions remain for green growth in the uplands, with alternative development pathways, i.e. niche vs industrial agriculture, non-farm jobs in mountain areas and ruralurban migration.

• Smallholder agriculture can ensure food security, sovereignty and safety in response to the increasing domestic (and tourist) demand for 'clean and green products' that is currently supplied by neighboring countries.

Re-connecting upland areas to economic growth

The 8 th National Socio-Economic Development Plan (NSEDP 2016(NSEDP -2020) ) aims at re-connecting upland communities with overall economic growth [START_REF] Wb | Taxonomy of the Poor and Its Usefulness in Policy Design[END_REF]. This objective is particularly challenging as rough terrain and remoteness are disadvantaging upland communities compared to communities in the lowlands, i.e. poor access to public services and markets. These constraints add to the country's development constraints in general, i.e. low population density limiting the available labor force, purchasing power and consequently the size of the domestic market, a landlocked geographic situation that limits integration and competition in international markets, low opportunities for remunerative employment alternatives in manufacturing or services sectors. The prospect of graduation from the Least Developed Countries (LDC) list may result in even more pressure on the Government as donor aid programs will be impacted by this change in official development status with an expected decrease in foreign aid. In this case, the Government would need to find other sources of revenue to replace the income lost from the Official Development Assistance (ODA) and the decline in the privileged trade treatment granted to Laos which would impact the manufacturing and export sectors. In addition, the Government policy of reducing the number of civil servants in an attempt to balance the national budget will have an impact on its capacity to implement the planned reforms in the coming years.

The proposed shift from a resource-based model of upland development to non-resource sector growth requires managing the natural resource base more sustainably. Green growth is defined as a path of economic growth that uses natural resources in a sustainable manner. It promotes inclusive development and innovation. The Lao National Green Growth Strategy has been put forward as a key instrument for a smooth transition in development models with a focus on agro-processing, tourism, and small and medium enterprises (SMEs) to achieve the SDGs. However, competing visions remain on how to implement green growth, namely green versus industrial agriculture. As Lao PDR cannot compete for volume market shares with neighboring agricultural commodity giants, it could supply niche and expanding markets for high-quality, consumer-safe, and environmentally sustainable foods and agricultural products. Under this paradigm, green growth translates as modern, scientific agriculture for productivity increases with small and medium enterprises as the main engine for materializing that strategy. An alternative vision consists of moving away from current dependency on natural resources through diversification of the national economy supported by non-farm activities and migration to cities. Such a vision for the uplands' future relies on (i) the development and promotion of non-farm activities in such areas as handicrafts to support green tourism, and other green jobs, and (ii) the strengthening of education and vocational training to build rural people's capacity in order to ensure their skills for work in urban areas.

In the context of the Lao uplands, smallholder agriculture appears as the most promising option towards food security, sovereignty and safety. However, it has not given yet its full potential as productivity and profitability are constrained by a number of structural problems such as low availability of high-quality seeds, limited access to irrigation, finance, markets as well as insecure land tenure. In the context of Lao PDR, organizing a lively debate involving multiple stakeholder groups and power imbalances was very challenging using a conference format. Compromises had to be found between the short time policy makers could dedicate to such an event and the necessary logistics to get a large group of 250 persons to engage in meaningful discussions. Splitting the assembly into smaller groups would have prevented all participants from gaining access to the overall picture, which was only possible by attending all sessions. It was therefore decided that the overall Lao Uplands Initiative would be organized over a six-month period (November 2017 to May 2018), and would take the form of a series of workshops and consultation meetings, including the Lao Uplands Conference organized in Luang Prabang from 12 to 14 March 2018 (laouplands.org). This conference was organized around the 'hot topics' that emerged during the preparatory meetings, e.g. reducing poverty in the uplands, addressing livelihood options and looking at the big development picture. We had to look at the respective roles of the Government, development agencies and donors, as well as the private sector in collectively designing concrete development pathways towards SDGs. Learning from projects and past interventions and looking at the conditions for the generalization of success stories: costs (financial, human) and conditions (enabling environment), were ambitious objectives for such a short timeframe.

It was particularly challenging to address cross-cutting issues with large multi-stakeholder groups with none of the participants feeling able to respond to crucial issues or to debate them publicly. We therefore decided to collect questions and comments during the successive events, using the SLIDO App. on smartphones (www. sli.do). SliDo allows participants to anonymously send questions and comments. It also provided an open arena to express their reactions to the presentations. However, direct responses could not be provided to all the participants' concerns due to the limited time available. The many questions and comments were further discussed within smaller groups during the wrap-up meeting of the Conference and shared on social media (Facebook). They were incorporated into the learning briefs that are the main output of the overall knowledge capitalization process. The briefs capture the essence of the debates that took place along the successive steps/ consultations and literature into clear and concise messages that can be shared with the general public.

Quick facts

• Key messages have been synthesized and discussed with participants during the six-month period of the Lao Uplands Initiative.

• The Lao Uplands Conference allowed participants to share the experiences of different uplands projects, with booths, videos, posters, and handson workshops.

• The main outputs of this knowledge capitalization process were then captured into learning briefs for further discussions with Government agencies and development partners.

landscape of opportunities LAO UPLANDS

INITIATIVE INITIATIVE

Adapting development models to changing policy contexts

The preparatory meetings for the conference showed that the drivers of change and their impacts on upland livelihoods have evolved over the past decade but that the development arena and intervention mechanisms through donor-led projects remained essentially the same. We pointed out three main dimensions to this issue: the problem-solution driven interventions, the role of empirical evidence in policy formulation, and the instruments of the modernization narrative.

Firstly, dealing with deeply entangled development issues, it is clear to all stakeholders that the problem driven or solution driven approaches to development that endeavor to find a solution to each problem identified cannot easily address complex, interconnected problems. For example, forest degradation issues have to be dealt with not only by foresters but addressed through multiple dimensions at the landscape level; the market biases due to middlemen monopsony in their allocated villages or the many tax collectors along the value chain can only be solved if solutions are found to compensate income losses of district Government agencies (e.g. DAFO, DoIC) from the lost trade license fees. In most cases these problems cannot be solved by the people who created them. They are essentially cross-sectoral and require an elaboration of theories of change or an exploration of scenarios; a path finding purpose-driven approach based on a collectively negotiated societal model/goal reflecting shared values. What is required in terms of intervention mechanisms is quite different from the ubiquitous problem-solution approach; typical project logframes often constrain collective creativity.

Secondly, experts have long been debating about the types of evidence actually used to feed policy processes in the context of the Lao uplands. Planners, academics, donors and NGOs engage in producing empirical evidence as a basis for sound policy design. Surveys are carried out, databases constructed, maps produced and interactive websites launched. However, policy making in Laos is mostly not data-driven. While official reports include plenty of facts and figures, these are often an expression of how things should be, rather than an accurate measure of how things really are. Official data tends to be biased by the strong pressure put on it by Government leaders on their agencies to achieve their plans. The reporting system is mostly organized around pointing to increases or decreases as compared to the objectives of the plans. The plans developed at village, district or province levels mostly ignore the systematic, transparent databases that exist but are often fragmented because they are project-driven. Also, policy makers know that projects sometimes oversell their results and to hide some weaknesses and failures in their evaluation reports, making it difficult for Government partners to distinguish between truth and hype and between insights and delusions. Consequently, the facts and figures in written reports from both Government departments and aid projects are unlikely to be the sole basis for making policy decisions [START_REF] Bartlett | A study carried out as part of the final evaluation of the Laos-Australia NGO Cooperation Agreement (LANGOCA) Program[END_REF]. In addition to these, Government officials rely on direct observation through exposure to real life activities and project achievements. Field visits are able to create learning processes in which local innovations are observed and discussed, and scaling up decisions are made. In the end, experience on the ground -success stories -are sought by policy makers, e.g. decree on conservation agriculture in 2005 after a visit to Brazil, decree on associations in 2009 based on the AGPC story in the Boloven, on the premises that field evidence will feed policies processes.

Thirdly, Government policy in Laos is supporting an upland development model based on the modernization narrative that typically involves commercial production, scientific technology and largescale projects. This narrative is embedded in the 8th NSEDP and the Nov 2017 Congress Resolution No13/GOV. Scientific techniques, with modern-industrial agriculture are seen as key instruments of development to increase agricultural productivity and help reach GDP targets. The country's leaders as well as ordinary Government officials explain the causes of poverty by the dependence of upland people on nature, their 'poor education' and their 'traditional methods' of farming. In short, the mainstream development narrative sees progress in terms of a shift from the traditional (backward) to the modern (forward). Under this narrative rural development and poverty reduction come from the combined efforts of research, education and extension, supporting the design of adapted modern technology, followed by large-scale adoption of improved, highly productive, farming practices (MAF's Agriculture Development Strategy to 2025 and Vision to 2030). Many projects have followed this model supported by ODA, and more recently through land concessions and private investments under the 'turning land to capital policy'. With the prospect of graduation from LDC status, the financial instruments for reducing production costs and increasing productivity are now sought from the promotion of Small and Medium Enterprises (SME) and new tax and tariffs policies that are expected to provide new financial resources while easing business environments.

Date Workshop topic Organizers

Nov Changes in the uplands as envisioned by conference participants 3 A vision & roadmap for action towards sustainable development the policy frameworks and intervention mechanisms over the past decades. On the other hand, policy makers and development practitioners see gaps in

Key lessons from the Lao Uplands Initiative

The main lessons from the Lao Uplands Initiative are described in the thematic learning briefs in this volume. They are organized according to three dimensions of changes: (i) a diagnosis of the current situation and exploration of future scenarios, (ii) a proposal for intervention mechanisms and agents of change, and (iii) recommendations to create an enabling environment towards achieving SDGs.

3.1.1.

CHANGING UPLANDS challenges & opportunities

Alternative futures in the Lao uplands

Participants explored some macro trends, especially how policy initiatives, land-based investments, and traditional agricultural systems interact in shaping uplands development pathways. Planners, development practitioners, academics and civil society organizations used scenario planning to better understand the critical uncertainties in the future trajectories of Lao Upland Development and identified different potential approaches to upland development based on lessons learnt from previous projects and within the framework of the MAF approved Upland Development Strategy. The collective process supported a dialogue around a societal model and associated values, before discussing how to achieve the model. Such an open, holistic, multi-sectoral perspective was expected to avoid the traditional problem-solution dialogic. The scenario building exercise showed that different groups of stakeholders tend to frame these uncertainties differently. On the one hand, academics and extension agents tend to organize future scenarios based on the technological progresses expected from the research and the capacity of academics while extensionists tend to educate the uplands populations toward higher adoption of improved technologies. Their scenarios still pertain to the technology transfer realm that has structured policy implementation and land management as key elements of uncertainty that structure their multiple scenarios. Lastly, foreign participants framed their scenarios using market development and access to natural resources as the main uncertainties. This exercise revealed that different stakeholder groups are working along different developmental frameworks. While it may be challenging to reconcile them, it is extremely important to reach an agreement about the societal model framing our interventions, otherwise we may only agree on the minimum common denominators that do not allow collectives to embrace common objectives.

Vulnerabilities and adaptation to change

Mountainous regions and their inhabitants have long been associated with certain stereotypes such as their low capacity to adapt to climate change due to poor socioeconomic development. Within this logic, upland people are perceived as highly vulnerable. Empirical evidences suggest, however, that the vulnerability of local communities is rooted in complex dynamics that call for investigations beyond stereotypes of high vulnerability. The relation between poverty, risk and vulnerability has not been properly investigated nor taken on board in designing Government policies. Many Government officials still understand vulnerability as being caused by geographical location rather than socio-economic factors. For example, communities are considered at risk from flooding because they live along river banks. So far, the main options for dealing with these problems were either to remove the risk (e.g. river bank stabilization), or move the community (e.g. resettlement). Again, these problem-solution perspectives tend to hide the multiple, complex dimensions of poverty. They will gradually become irrelevant as new vulnerabilities emerge from the economic trends presented above that come on top of already existing hazards, such as unusual weather events, floods, and pest or disease outbreaks that are also influenced by the changing environment, including climate change. Early warning systems based on Information and Communications Technology (ICT) tools have been tested in Laos and found to hold the potential of significantly reducing a wide range of vulnerabilities by providing relevant information. However, the lack of support from mid-level Government staff due to unclear data policies has prevented further deployment of these ICT tools.

3.1.2.

The rallying point of agroecology approaches is the need to improve the sustainability of agriculture by focusing on its various dimensions (e.g. institutional, agronomic, environmental, social, economic, and ethical) and at various scales (e.g., the plot/field, the farm, the landscape and the whole food system). Agroecology is thus not only about transforming agricultural practices, it is also about transforming the relationship between agriculture and the society. As a result, it also focuses on social values, such as trustworthiness, health, food sovereignty, youth development and improved livelihoods for upland communities (Loconto et al., 2017). This led us to explore the new links established between farmers for whom economic benefits appear as a key driver of the adoption of innovative practices such as agroecology [START_REF] Castella | Towards an agroecological transition in Southeast Asia: Cultivating diversity and developing synergies[END_REF] and consumers, who act as crucial drivers in these new initiatives. In this sense, agroecology represents an alternative economic model whereby producers and consumers jointly define the quality of products by taking into account other innovative components such as farmers' employment, food sovereignty, and biodiversity. Indeed, have the potential of becoming development actors rather than passive beneficiaries. The animated narrative contrasts with the vulnerability narrative, which sees people as negatively affected by forces beyond their control. Green Extension involves a basket of methods (Farmer Fields Schools, Participatory Land Use Planning, Farmer to Farmer Learning, Participatory Action Research, etc.) that are used to promote various types of content (agroecology practices, cooperatives, small and medium enterprises, etc.). Based on the experience of past and on-going projects, however, it is possible to identify a set of five guiding principles for green extension, namely: (i) participatory agroecosystem analysis, (ii) community planning, (iii) action-research, (iv) farmer to farmer learning, and (v) organizational development that includes leadership provided by public officials. These principles are common to different approaches whatever the specific names used by different projects: communitybased NRM, integrated landscape management, or Green Extension. The capacity to implement Green Extension already exists as projects have helped create this capacity, although projects are also a hindrance to mainstreaming. Activities are fragmented, often small scale, with no critical mass and weak local ownership. There are alternatives to project-driven agricultural development which need greater consideration, including revisiting the program-based approach, involving farmer organizations and networks, and possibly the private sector.

3.1.3.

ENABLING ENVIRONMENTS for uplands development ENGINEERING TRANSITION towards agroecology

Landscape approaches to agroecology

To compete and meet the demands of international and domestic markets for high-quality products, Lao PDR's agriculture needs to position itself as green and clean. Traditional farming systems have featured very limited use of synthetic fertilizers and agrochemicals, while 'imported' intensive mono-cropping systems have tended to involve excessive use of such inputs, resulting in water and soil pollution, as well as consumer food safety concerns. Lao PDR continues to have the potential to effectively compete in the production of high-quality food while improving upland livelihoods by extending effective agroecological practices, such as the System of Rice Intensification (SRI), Integrated Pest Management (IPM), conservation agriculture, agroforestry or integrated organic farming. Many success stories attest to the multiple benefits of agroecological practices to preserve livelihood systems and ecosystem services beyond simple agricultural production. Agroecology should therefore become a key instrument of green agriculture and green growth. However, its generalization is constrained by the complexity of low farmer productivity and lack of enthusiasm, combined with nature-based interactions that need to be developed and maintained to launch the system as compared to the simple use of external inputs in the case of intensive mono-cropping. Agroecology should be conceived and promoted at the landscape level, and not at the field or farm level such as in alternative approaches such a GAP. However, this requires a combination of local institutional leadership, incentives for farmers, and dramatic (high value and productive) results. For example, roaming animals can be a major constraint to the adoption of agroecological practices for sustainable crop intensification through improved fallow, cover crops or residue management. Reorganizing crop-livestock interactions at the landscape level is challenging because it requires behavior change and must overcome the lack of family labor, although necessary for maintaining or improving the performance and resilience of upland farming systems.

Learning processes and green extension

Learning and behavior change are essential for uplands populations to adapt to their changing environment. Learning processes need to be location specific, responsive to local needs, multi-stakeholder and iterative. This calls for major changes in extension approaches that would turn extension agents from expert lecturers to facilitators in adaptive learning approaches. This in turn requires a change in educational curricula at the vocational and university level. The objective would not be so much to educate as to motivate, to 'develop capacity' of ordinary people to gain greater control over their lives. Upland farmers Bringing agroecology to the market agricultural products are not the only goods being valued in this process: cultural traditions, ideas, visions and knowledge are also being exchanged. Ultimately, the very nature of markets is at stake, with a possibility to shift to what some call "mindful markets" (Van Willenswaard et al., 2015), i.e. markets that are inclusive and based on full awareness of whole food systems. This means taking into account healthy food self-reliance for farmers, farmers' families and rural landscapes, in short, bringing the market to agroecology beyond finding outlets for agroecological products. To reach this ambitious objective, specific partnerships need to be developed with social enterprises (e.g. ADP, Saffron) sharing common values. Risks should be buffered for innovators who engage in uncertain ventures through using self-supporting ITC tools (e.g. market information systems), or tailor-made risk insurance schemes. Last but not least, a transition is needed from value chain to food system approaches, meaning (i) acknowledging that consumers have leverage, (ii) promoting inclusiveness, and (iii) revealing the true value of food [START_REF] Kousonsavath | Boosting the demand for organic products: The role of consumer trust and product availability[END_REF].

Youths in agriculture

Lao PDR has a young population, with 54% of its about 7 million inhabitants below the age of 25 years and a median age of the overall population at 23 years. Such a young population, with 70% officially residing in rural areas, potentially represents an opportunity for agricultural development. However, parents prefer to see their children moving away from agriculture instead of enduring the tough lives they lead. The promises of a youthful population as an agent of agriculture change may not concretize as most of them want to enrol in the administration and move to the city. Their perception of agriculture is negative. How to keep rural youth connected to their roots, to their land, or to attract educated youths to come back to their land after their studies? How to turn agriculture into an attractive employment option? Firstly, communication tools should promote the quality agroecological practices and sensitize youths to organic products as an alternative to intensive agriculture and junk food. Beyond TV and radio programs to convince the parents, social media such as Facebook can directly reach the youth target population. Secondly, innovation requires investments in human capital development, particularly in the areas of education -training, protection -risk buffering, and participation. Most young people start thinking about becoming entrepreneurs not when they are in their village or studying at the University, but when their parents stop supporting them financially. They should then be equipped to seize opportunities with: knowledge-skills, connections-network, capital-land (primary accumulation through boom crop or migration, land title or university diploma as collateral for loans…). Thirdly, the Government and development partners should favor networking among social entrepreneurs and support peer to peer learning, farm visits, seed exchanges, etc. Learning is the key to the emergence of these 'start-ups' for youths in agriculture and then for the maintenance of the networks as participants may lose interest if they don't learn -and thus benefitanymore.

•

A roadmap towards green agriculture in the Lao Uplands

Beyond the inputs and reactions of participants to the Lao Uplands Initiative that were turned into 'key lessons' as presented above, the core group of organizers did reflect on key messages that would help transform good intentions into concrete actions and interventions. Sustainable Development Goals as applied at the global level provide a framework to analyze and compare the progress in different countries although are by no means a societal project that would mobilize all the active forces of the nation. SDG indicators are important as they link to the global system but a new development paradigm should rely on specific indicators of progress according to the intervention pathway collectively defined. This section introduces elements of a vision for the future of the Lao Uplands and a roadmap for action as they were discussed by participants in the Lao Uplands Initiative (laouplands.org).

Adding values to upland agriculture

Thinking and practicing agriculture differently is more than necessary, it is a question of survival for the Lao upland communities. Indeed, shifting cultivation systems are gradually disappearing with the increasing population density, the Government's eradication policy, and integration to market economy of the uplands as a result of agricultural land, mining, and hydropower concessions that all restrict the land available for smallholder farming.

On the other hand, intensive mono-cropping systems that were initially presented as a modern alternative to the former systems have shown their limits in terms of land degradation, pollution and indebtedness of upland farmers. Other agricultural practices and development pathways must be promoted as alternatives to these two approaches. These proposed practices are not new to any extent; they were already promoted in the Upland Development Sourcebook published in 2005 and were around a long time before. They are now grouped under the term 'agroecology' as they share common principles, e.g. recycling biomass within the farming system, maintaining multi-functional landscapes, relying on agrobiodiversity to control pests. However, large-scale adoption of these alternative practices, SRI, IPM, conservation agriculture, agroforestry, integrated organic farming, permaculture, etc., has always been confronted with a number of obstacles, especially poor recognition and limited rewards to their higher value and contribution to people's wellbeing beyond purely economic productivity.

Green agriculture, green growth, green extension… these ideas are not new either but they apply now to a modern context of ecological intensification. As core elements of green agriculture, which is the cornerstone of green growth, agroecology practices should be massively promoted as an instrument to maintain multifunctional landscapes and make upland agriculture more attractive to the young generation. Green extension as presented above is one instrument of this intended transformation of upland agriculture.

It entails five principles, namely: (i) participatory agroecosystem analysis, (ii) community planning, (iii) action-research, (iv) farmer to farmer learning, and (v) organizational development. However, these approaches can only be scaled-up if they are evaluated against the appropriate indicators.

Inside-out development process -co-designing the societal project and intervention pathway

Too often, changes are driven or influenced from the outside, such as foreign investments (e.g. mining, hydropower), global or regional policies (e.g. REDD+, ASEAN trade agreements) or donor priorities (e.g. local empowerment, good governance). Development processes from the inside-out, from Lao people for Lao people, was promoted by participants in the Initiative. The domestic market is narrow, so products and standards should be first designed according to the demand of the Lao market and later be expanded. Tourism is expanding because visitors want to discover the specificities of the Lao culture and agriculture. If Lao agriculture and livelihoods align on foreign standards then they will lose their attractiveness to the emerging tourist industry constituting a lost opportunity to contribute to green growth. This requires involving all sectors and levels of the Lao society in defining ultimate development goals and implementation pathways for the uplands. This may call for repoliticizing the development debate, which goes against the logic of the current project-led development process.

Based on a clear definition of green growth as an instrument to increase productivity, address poverty issues and economically empower farmers while ensuring sustainable agriculture, it should be possible to reorient development efforts away from the eradication of shifting cultivation, away from boom crops and raw commodity exports that contribute to the resource-based development model that degrade ecosystems and weaken livelihoods in the long term. An alternative development pathway should be imagined before resources, including forest and soils, are further degraded. Increasing competitiveness should be associated with increasing quality and safety of agricultural products based on Lao standards. The food systems should be envisioned beyond traditional value chains with mutually recognized standards, regulations, inspection, certification, accreditation procedures and information sharing mechanisms. Accurate and transparent information systems, i.e. ICT, social media, e-commerce, may increase the efficiency of agricultural supply chains, shorten market channels, and increase interactions between producers and consumers.

The overall transition from resource to non-resource based development, i.e. avoiding dependency on non-renewable resources, requires higher education and skilled workers. Supporting the transition towards agroecology, buffering negative consequences of climatic and economic shocks on vulnerable upland populations when entering the market economy, require efficient Government officers teaming up with the private sector to lead green extension. They must be equipped with relevant knowledge to facilitate local negotiations and empower upland populations. Human resource development is a key element of the required change in the learning process. Education in its diverse forms,

Quick facts

• Green agriculture is an essential component of green growth in the Lao Uplands. It should be supported by smallholder farmers engaged in agroecology practices.

• Indicators used to assess progresses (e.g. tax revenue, wellbeing) reveal a vision for the future of the Lao Uplands, influence the kind of progress we make, and increase potential to reach the goals once they are set collectively.

• Revealing the true value of things (e.g. agricultural products, ecosystem services), including moral and meaning values, will support the scaling-up of good practices towards green agriculture.

Full recognition of the multiple values of agriculture is essential to the large scale adoption of agroecology practices. Green agriculture is more than a provider of GDP growth based on export of raw agricultural products. Three values of agriculture should be put forward with equal importance and weight:

• Monetary value: increasing land and labor productivity through innovative crop-livestock and land management systems will remain a key feature of future upland agriculture.

• Moral value: resilience perspective should compromise the short term benefits with the long term productivity objectives. Beyond sustainability, equity should be rewarded by protecting the most vulnerable and buffering the risks for the innovative farmers.

• Mindful markets should reveal the true value of conventional products and practices, including environmental costs into the economic calculations, therefore increasing the attractiveness of green agriculture for both producers and consumers.

These balanced values will provide good arguments to raise the interest of youth for agriculture as it will provide an attractive image of a healthy and environmentally friendly activity.

Quick facts

• Agricultural changes should be driven from the inside and less influenced from the outside to preserve the values of Lao society while engaging in the profound changes from a resource to a non-resource development model.

• Increasing competitiveness could be associated with increasing quality and safety of agricultural products based on Lao standards through the generalization of a food system approach. (Li, 2015). Tania Li (2015) describes projects as 'a time bound intervention with a fixed goal and budget, framed within a technical matrix which renders some problems amenable to intervention, while leaving others out of account'. Projects create a silo effect, compartments between Government services and sectors, instead of creating the expected synergies. After managing the flow of ODA for a few years most projects close without leaving a durable impact on their beneficiaries and limited perspectives for scaling-up (Li, 2015). There is an urgent need to favor cross sectoral approaches despite the constraints embedded into the governance system, bureaucracy and an administration inherited from several decades of project-led interventions.

Green economic growth, public-private partnerships, responsible investments, support to agribusinesses and SMEs have been pushed forward by recent policy frameworks, including the 8th NSEDP, MAF's Agricultural Development Strategy to 2020 and Vision to 2030, and the Upland Development Program. However, the implementation mechanisms still rely on the project paradigm that, as mentioned before, is not conducive to policy implementation. Projects then become the very cause of the policy implementation gaps that they were supposed to mitigate. Partnering with the private sector imposes adaptation to the partner's governance mode, including flexibility and reactivity, and not imposing the slow pace and cumbersome procedures of a bureaucracy that systematically erodes competitiveness. While there is a general agreement on the benefits of stimulating agribusinesses to shift from the currently poor processing and postharvest facilities to more modern and environmentally-friendly technologies to improve the product value and reduce postharvest losses, the main challenge is to reduce the costs of doing business in the agriculture sector. These issues are interrelated and need appropriate public interventions that are beyond the scope of projects. For example, at present lending from commercial banks to agribusinesses is very limited, because most agro-enterprises are small and cannot afford market rates. However, these investments are crucial for the modernization of SMEs that often possess out-ofdate equipment, resulting in high losses in both value and volume during processing and postharvest stages.

Projects are based on the premises that they can provide technologies and support for upland populations, although in fact what is needed is to create an enabling environment that will allow people to develop themselves, which most projects cannot achieve. From project-driven initiatives to people-enabling initiatives, innovative support mechanisms are required. Program-based assistance (PBA) should be revisited based on the lessons learnt from the Northern Uplands Development Program (NUDP, 2009(NUDP, -2017)). Sufficient time should be allocated to the 3-I reform (institutions, indicators, incentives) before testing the PBA model again. In the meantime public and private sector efforts should be dedicated to vocational education and training to increase the knowledge and skills of all partners in future multi-sectoral development processes.

Epilogue

At the end of the day, the Lao Uplands Initiative (LUI) closes the learning loop of the Northern Uplands Development Program by drawing lessons about its own implementation and pointing to Lao Uplands Strategy implementation issues. The LUI was in itself an attempt to develop and implement a programmatic approach outside of a project framework. It was confronted with much skepticism concerning its ownership by the Government as it was not initially hosted by a particular department at MAF and was embracing a very large scope beyond any single department. Previous attempts to create project-led, open initiatives also ended up creating new entrenched projects that soon turned out to compete with other similar projects for resources and recognition instead of federating efforts from multiple existing projects. These initiatives are confronted with a lack of understanding of development partners, including Government officers, donor representatives and foreign advisors, of the overall intentions and implementation models as this would bring them out of the comfort zone of their traditional project intervention mode. Also, project proponents were very busy with the milestones and the deadlines of their own projects and could not dedicate sufficient time to collective reflections that were not initially included in their project logframe and for which they would be neither paid nor rewarded. This LUI experience clearly shows that collective mobilization beyond projects and across sectors necessary to meaningfully address the challenges described in this paper is difficult to achieve, and may be utopian. As long as they are not addressed, however, these thorny problems will remain.

2

  The Lao Uplands Initiative2.1. Objectives & methodsThe future model of agricultural development based on (i) productive climate-resilient farming systems; (ii) diverse, nutrientbalanced, and safe food consumption patterns; and (iii) improved rural income through commercialization combined with women's empowerment, may appear out of reach when analyzing the current status of upland agriculture. However, since the last Lao uplands conference on Poverty Reduction and Shifting Cultivation Stabilization in the Uplands of Lao PDR, more than a decade ago, which proposed a number of practical options in the form of a source book (Improving Livelihoods in the Uplands of the Lao PDR -2005), many projects have been conducted by the Government with the support of development partners, with some of them prefiguring or contributing to the future model of upland agriculture. In addition, from 2011 to 2015, the Northern Uplands Development Program (NUDP) supported a long series of consultations that involved a large range of uplands' stakeholders and led to the Lao Uplands Development Strategy and Program, officially endorsed by the Ministry of Agriculture and Forestry in 2016. In 2018, a consortium of projects and institutions organized the Lao Uplands Initiative to (i) reflect on recent transformations and their impacts on upland populations, (ii) take stock of main lessons from past and on-going interventions, (iii) review policy options for a green growth, and (iv) develop a road map toward achieving the Sustainable Development Goals.

  Changing uplands are putting stress on the smallholder farmers who are the main labor force and actors of future green growth scenarios, external interventions should buffer negative impacts of on-going changes and buffer risks for innovators and entrepreneurs. • Engineering transitions requires (i) innovative thinking, beyond current problem solving approaches, (ii) purpose-driven interventions according to a commonly agreed societal model and (iii) local ownership and empowerment of people to take control of their

Thematic workshops organized as part of the Lao Uplands Initiative Quick facts • Complex development problems require purpose-driven approaches instead of problem-solution driven interventions. • Policy making in Laos is mostly not data-driven; empirical evidence is based on direct observation through exposure to real life activities and projects' achievements. • Projects, supported by donors, Government agencies or the private sector are instruments of the modernization narrative towards increased productivity.

	23, 2017	Soil carbon is what we need!	DALaM, CIRAD, EFICAS
	Dec 9, 2017	We are what we eat	MAF, GRET, CIRAD, ACTAE
	Feb 9, 2018	Bringing agroecology to market	ALiSEA, NUoL, GRET, CIRAD
	Feb 23, 2018	Vulnerabilities and adaptation to changes in the Lao Uplands	DALaM, NAFRI, CIRAD, CDE, CARE, CCL, SAEDA
	Feb 27-Mar1, 2018	Green extension practitioner's workshop	DAEC, LURAS, FAO
	Mar 12-14, 2018	Lao Uplands Conference: landscape of opportunities	DALaM, NAFRI, CIRAD, CDE, TABI, LURAS
	May 2, 2018	a macro-level perspective Alternative Futures in the Lao Uplands:	NAFRI, DALaM, CDE, TABI

• The envisioned transition towards agroecology requires massive investment into capacity development to engage the next generation of Lao upland farmers to seize emerging opportunities that are brought in by the revolution in communication technologies.

  .e. academic and vocational training, e-learning, etc., should be harnessed to increase the skills of upland youths to seize opportunities and adapt to a diverse and dynamic labor market. The next green revolution will certainly be brought about by the generalization of communication technologies that will enhance networking among stakeholders of the agricultural sector, eventually bridging with other sectors to create jobs. Smart farmers will apply modern information and communication technologies in agriculture. The next generation of farmers is already sensitized to the use of these technologies. They now need support to make the best out of these new opportunities, such as supporting information exchange among practitioners or networking among young agroecology farmers through social media (see ALiSEA network at alisea.org).

	3.2.3.

i

From projects to policies… with the private sector

  Making the best out of emerging ITC-based opportunities requires designing new governance mechanisms that would combine old and new communication channels. How are we to combine top-down and bottom-up communication channels, mass-media and social-media networking, vertical with horizontal coordination mechanisms? These new challenges call for new types of development interventions, including innovative financing mechanisms. Indeed, the expected development processes cannot rely exclusively on the public sector. Firstly, because with LDC graduation ODA mechanisms will be deeply transformed; secondly, because Chinese investment mechanisms will force the reorganization of ODA and FDI governance; and thirdly because the development model based on projects is reaching its limits. Before the 2007 'turning land to capital' policy, projects were almost the only instruments for development intervention. The whole development aid sector is organized around projects that are used to 'render technical' highly political issues

• The projects-based development model as generally practiced is reaching its limits and must be reformed as it is not able to synergize multi-sectoral interventions in rural development, nutrition enhancement, poverty reduction, etc. • Innovative intervention mechanisms are required to create an enabling environment for agribusinesses and SMEs through partnering with the private sector. • Public and private sector efforts should be dedicated to vocational education and training to increase knowledge and skills of all partners in future multi-sectoral development processes.

Three directions and nine big ideas towards sustainable upland development.

Changing the development paradigm

• From foreign-led to inside-out development process.

• From project to policy approach -Program Based Approaches need to be revisited.

• Harnessing the private sector: public-private partnership, responsible investments, private sector involvement in vocational education and training to increase labor skills.

Changing assessment indicators

• Revealing the true value of things (monetary & moral), embedding environmental costs into product prices.

• Sustainability indicators and quality standards designed from the inside and not dependent on external influences.

• Compromises need to be negotiated between multiple objectives, mix of options, as not everything is achievable, using scenario analysis.

Changing intervention approaches

• Investing in youth -networking, social media, horizontal communication; and better integration with traditional vertical communication channels. • Multifunctional landscapes should be promoted through an integrated landscape approach to agroecology.

• Food system perspective in line with green agriculture.


Thierry Poibeau
email: thierry.poibeau@ens.psl.eu

On "Human Parity" and "Super Human Performance" in Machine Translation Evaluation

Keywords: Neural machine translation, evaluation, human parity, human factors, ethics

In this paper, we reassess claims of human parity and super human performance in machine translation. These terms have already been discussed, as well as the evaluation protocols used to achieved these conclusions (human-parity is achieved i) only for a very reduced number of languages, ii) on very specific types of documents and iii) with very literal translations). However, we think it is necessary to consider these questions again. We show that the terms used are themselves problematic, and that human translation involves much more than what is embedded in automatic systems. We also discuss ethical issues related to the way results are presented and advertised. Finally, we claim that a better assessment of human capacities should be put forward and that the goal of replacing humans by machines is not a desirable one.

Introduction

Machine translation (MT) has recently achieved impressive results. Thanks to the success of deep learning approaches, state of the art systems have been made available online, which translate quite accurately between a large variety of languages [START_REF] Bojar | Findings of the 2018 conference on machine translation (WMT18)[END_REF][START_REF] Barrault | Findings of the 2019 conference on machine translation (WMT19)[END_REF], although differences from one language to the other remain important. This success is clear, it has been widely acclaimed and end-users can feel this progress very directly in their everyday life. Whereas previous systems were providing translations that needed to be heavily revised and corrected, recent systems provide decent and readable translations most of the time. Of course, the quality depends on a lot of parameters, among others the language pair considered, the type of text and the required quality of the final result. Following this success, several teams have claimed to have reached super human performance for different pairs of languages. The term "human parity" [START_REF] Hassan | Achieving Human Parity on Automatic Chinese to English News Translation[END_REF] and even "super-human performance" [START_REF] Barrault | Findings of the 2019 conference on machine translation (WMT19)[END_REF]) have been used for example in the context of the Workshop on Machine Translation (WMT) evaluation campaigns 1 . This means that MT is supposed to be equal or even to have surpassed the quality of human translation, at least in this context. Practically, during the evaluation, the translation produced by a machine was regularly preferred to the one obtained from a professional translator. This sounds impressive, but should be discussed further and put into perspective. In this paper, we claim that 1 See for example: "English to German: Facebook-FAIR achieves super-human translation performance; several systems are tied with human performance." [START_REF] Barrault | Findings of the 2019 conference on machine translation (WMT19)[END_REF] the term "human parity" and, moreover, "super human performance" are misleading, in that they concern only very specific types of texts, for a very limited number of languages, in very specific conditions. Authors of course mention this when they discuss their results, see for example [START_REF] Hassan | Achieving Human Parity on Automatic Chinese to English News Translation[END_REF], but paper titles, and thereafter headlines in the press, leave apart these as details, making the general public confused about the real state of the art2 . Although our concerns are not new and have already been detailed by different authors, especially [START_REF] Toral | Attaining the unattainable? reassessing claims of human parity in neural machine translation[END_REF][START_REF] Toral | Post-editese: an exacerbated translationese[END_REF][START_REF] Toral | Reassessing claims of human parity and super-human performance in machine translation at WMT 2019[END_REF][START_REF] Läubli | Has machine translation achieved human parity? a case for document-level evaluation[END_REF][START_REF] Läubli | A set of recommendations for assessing human-machine parity in language translation[END_REF], we think it is important to discuss them again, as these terms are themselves problematic. They are reproduced in the media for a general audience without much care, as if MT was a solved task, at least between some languages, whereas it is clear that performance varies a lot from one domain to the other, or from one type of text to the other. MT is not a solved task and has not reached human parity between any language pair (although some results obtained in evaluation conferences on specific data from specific domains may suggest the opposite). We first describe previous work (Section 2), before detailing why we think it is not really appropriate to claim that MT has achieved "human parity" for some language pairs (Section 3). Finally we show that the notion of usability, especially the interaction between MT and translators (through the post-editing process) is a more fundamental concept that should be better taken into account (Section 4).

Previous Work

The terms "human parity" and "super human performance" have been used in the Machine Translation community, especially in the framework of the WMT conferences. [START_REF] Toral | Attaining the unattainable? reassessing claims of human parity in neural machine translation[END_REF] have demonstrated several shortcomings in the WMT evaluation. The main issues are that the translations to be evaluated were sometimes based on problematic source texts. For example, in some cases, the source text was itself a translation from another language, which entails biases, like issues with phrasing and idiomaticity, and is thus not considered appropriate for evaluation, Another issue is that evaluators were not always professional translators (evaluators include participants themselves, and often remote crowd-workers or "turkers" who tend to prefer calque or word-to-word translation, even if it is not fluid, compared to a professional translation where the translator has appropriately transposed an idea or a concept through a less direct translation). Regarding the latter, it should be noted that the perception of translation quality varies according to the evaluator (i.e. endusers vs MT developers vs professional translators), although professional translators tend to achieve higher inter-annotator agreement [START_REF] Toral | Attaining the unattainable? reassessing claims of human parity in neural machine translation[END_REF]. It is, however, possible that the non-experts used for the task were actually closer to the target audience for the type of texts being evaluated (news translation) than professional translators. The problems observed during the 2018 WMT evaluation campaign were partially corrected in 2019 [START_REF] Toral | Post-editese: an exacerbated translationese[END_REF]. Other potential problems in the type of evaluation performed at the WMT event remain: the evaluation proceeded by text segment, and thus cannot really take into account the text as a coherent whole, see for instance [START_REF] Läubli | Has machine translation achieved human parity? a case for document-level evaluation[END_REF]. [START_REF] Läubli | A set of recommendations for assessing human-machine parity in language translation[END_REF] come to similar conclusions when reassessing the findings of [START_REF] Hassan | Achieving Human Parity on Automatic Chinese to English News Translation[END_REF]: they show that "perceived quality in human evaluation depends on the choice of raters, the availability of linguistic context, and the creation of reference translations." Some efforts are now made to better take into account the translation context (i.e. the larger text) and achieve better evaluations. [START_REF] Läubli | A set of recommendations for assessing human-machine parity in language translation[END_REF] in particular propose a set of 5 recommendations to avoid the issue of over achievement claims (these 5 recommendations are: (R1) Choose professional translators as raters; (R2) Evaluate documents, not sentences; (R3) Evaluate fluency in addition to adequacy; (R4) Do not heavily edit reference translations for fluency; (R5) Use original source texts). Whereas these recommendations are a progress in the field (and claims of human of super-human performance have been less frequent these last months), we think that even if the comparison between humans and machine is legitimate to a certain extent, the obtained result is not the same and this comparison may just be misleading when it is just based on purely informative texts (like news). It should also be noted that translation quality assessment is performed very differently depending on the context (research oriented vs industrial oriented for example) and this should be taken into account in comparative experiments. Lastly, one should note that we do not address machine translation for under-resourced languages in this paper, a domain in which human expertise is still highly valued. It is however quite important to keep this in mind when addressing the place of humans in the current research paradigm (NMT works well for 20-30 languages, and the most popular online systems are available for 100+ languages but bilingual training corpora are then rare. This should be compared to the circa 7000 languages existing in the world).

Does it Make Sense to Speak of "Human Parity"?

Fundamentally, the whole idea of "super-human performance" is in our opinion problematic, since translations are difficult to evaluate and to compare, especially when they do not contain clear mistakes. It also gives the impression that MT is a problem solved, at least for some languages (and, if the results are said to be "super-human" for some languages, one can assume that automatic systems will soon be better than humans for a large number of other languages). A quick look at current systems clearly shows that we are still far from any "human parity".

MT Produces Very Literal Translations

Neural machine translation generally works at the sentence level, although more and more systems try to go beyond the sentence boundary to take a larger context into account, see for example [START_REF] Popel | English-Czech systems in WMT19: Document-level transformer[END_REF][START_REF] Popel | Transforming machine translation: a deep learning system reaches news translation quality comparable to human professionals[END_REF]. It was already the case with statistical MT (and segment-based MT) [START_REF] Koehn | Statistical Machine Translation[END_REF], but this previous approach suffered a lot from a more fundamental issue, since it had to assemble different fragments of texts that were not always fully compatible, hence the frequent problem with ill-formed sentences within this research paradigm. NMT, by directly manipulating representations at the sentence level, thanks to transformers [START_REF] Vaswani | Attention is all you need[END_REF], does not suffer so much from this kind of problems. However, it should be noted that, despite these recent advances in the field, machine translations remain quite literal, see for example [START_REF] Fonteyne | Literary machine translation under the magnifying glass: Assessing the quality of an NMT-translated detective novel on document level[END_REF]. This is due to the approach itself: NMT is still mainly based on knowledge inferred from large collections of parallel data, where one sentence in the source language corresponds to one sentence in the target language.

From this point of view, NMT is a direct continuation of the previous segment-based approaches. Equivalences between languages are found inside the sentence at a more or less local level, despite recent attempts to integrate a wider context in the process. The statistical nature of NMT also favours standard trans-lations over more original ones. Moreover, NMT does not include reformulation or paraphrase modules that would allow the system to produce more varied or more global decisions (for example, human translators choose some translation equivalencies depending on the context and the situation, cf. "(human) translators can adapt word choice according to different communicative demands and circumstances of language use" (Frankenberg-Garcia1, 2021). All this explains why, despite powerful architectures based on transformers and accessing the whole sentence in one go, translations remain literal and based on local equivalencies. The consequence is an accurate but very literal way of translating, that is not always convenient. This especially explains the large discomfort and disdain expressed by professional translators, especially when the target is a rather elaborate text. The question is not to translate literature, but even for articles in newspapers, NMT is often judged too literal

MT Works better on Purely Informative Texts

As we have just seen, the nature of the text to be translated plays a major role. Evaluations (and especially the annual WMT evaluations, within the workshop on machine translation) have shown that performance is now comparable with those of humans for some language pairs. However, this evaluation has been traditionally performed on short news, i.e. informative texts written in a direct and simple style so that they can be easily re-used by journalists to write more complete articles. The content of such texts can be translated quite literally and generally does not require much rephrasing in the target language [START_REF] Reiss | Type, kind and individuality of text: Decision making in translation[END_REF]. This is highly specific, and does not cover the full range of needs and types of text one may need to translate.

The WMT conferences organizers are aware of the problem and have recently introduced different tasks and different types of texts appropriately. However, the main evaluation (and claims of human parity performance) are still based on the main track concerning news translation. It is however interesting to have a look at the different test suites and changes of domain to examine how robust the systems are.

When one looks at other tasks, results are mixed and predictable issues appear with terminology and domain specific phrasing. For example, for WMT 2019, a subtask consisted in translating "audit reports and agreements from English to Czech, without domain adaptation [START_REF] Vojtěchová | Sao wmt19 test suite: Machine translation of audit reports[END_REF]." Although "syntax and overall understandability was scored on par or better than the human reference", "Terminological choices [were] a little worse" [START_REF] Barrault | Findings of the 2019 conference on machine translation (WMT19)[END_REF]. On the other hand, "the micro-study on agreements reveals that even very good systems produce practically useless translations of agreements because none of them handles document specific terms and their consistent translations whatsoever." (idem) Audit agreements are still technical texts and are not literature. Beyond terminological problems, other issues can be observed in more complex types of documents concerning syntax and semantics.

Super Human just does not Make Sense

Toral (2020) explains that the term "super-human" actually refers to recent advances in artificial intelligence, especially games like Go, where the machine can play against itself and, in doing so, can develop new and original strategies (these strategies are now studied by humans and are very different from anything proposed so far). Alpha Go, the program developed by Deep-Mind, beat the world champion of Go, Lee Sedol, in 2016 and has continued improving since. From this point of view, one can say that artificial intelligencebased Go programs are indeed 'super-human'. But the rules of Go are simple compared to the complexity of languages (the complexity of Go is mainly related to the number of possibilities at each stage of the game), and at least in the end, it is easy to see who has won.

With language, and especially translation, there is nothing to win and no unique solution. It should also be emphasized that the term ''human parity" is problematic by itself, since MT works very differently from humans. Human translation involves understanding the text on two levels, i.e. establishing relations between elements of the text themselves (text internal cohesion) and correspondences between elements of text and the real world (discourse external coherence). While the machine can reasonably be expected to handle issues of text cohesion at some point, discursive coherence is out of its reach, since it requires knowledge of the world.

It is also clear that most current systems work at the sentence level and are, for instance, very bad at dealing with pronouns, when the source and the target language differ in this respect ("Most MT systems at the sentence-level do not have access to adequate context that may be required for the translation of pronouns" [START_REF] Jwalapuram | Pronoun-targeted fine-tuning for NMT with hybrid losses[END_REF], see also [START_REF] Sennrich | Why the time is ripe for discourse in machine translation[END_REF]. Note however some preliminary attempts to address the problem, like in [START_REF] Luong | A contextual language model to improve machine translation of pronouns by re-ranking translation hypotheses[END_REF] or [START_REF] Fu | Reference network for neural machine translation[END_REF]). For example, when the source language only has one gender and the target language different pronouns for masculine and feminine, one gender has to be chosen, often at random, by the translation system, which is thus often wrong. This kind of mistake is frequent for any decision the system has to make that involves a context larger than a single sentence. This is thus understandable (and probably negligible from a statistical point of view in a BLEU score), but should prevent the use of terms like "human parity".

Putting Humans back in the Loop

We have shown in the previous sections that it does not really make sense to speak of human parity when eval-uating MT. In this section we defend the idea that translation of non purely informative texts involves specific features that are probably not fully attainable using today's technology.

NMT and Human Invisibilization

We have seen in the previous section that several studies (esp. [START_REF] Toral | Attaining the unattainable? reassessing claims of human parity in neural machine translation[END_REF][START_REF] Toral | Reassessing claims of human parity and super-human performance in machine translation at WMT 2019[END_REF] reassessed claims of human parity and highlighted that this is only true for certain types of texts, especially news. Although all studies outline that evaluation on other domains and languages that the ones addressed in a specific paper (generally centered around a specific language pair) remain to be done [START_REF] Hassan | Achieving Human Parity on Automatic Chinese to English News Translation[END_REF], all are optimistic about generalization capabilities.

However practical experiments have shown that generalizing performance beyond the news domain is far from obvious. A famous example is the translation of the book Deep learning [START_REF] Goodfellow | Deep Learning[END_REF] from English to French using NMT techniques, that has been widely advertised in the press as having been done automatically and praised as an astonishing success (which is undoubtedly the case). The process was in fact far from being fully automatic. A large amount of time had to be devoted to the preparation of the data for i) marking in the text technical zones that should not be translated (like pseudo-code and algorithms) and ii) manually translating technical terms (a dictionary of 200 technical term was manually elaborated before translation). The automatic translation of the rest of the text was also not perfect, as explained in [START_REF] Escribe | Human evaluation of neural machine translation: The case of deep learning[END_REF]: "category shifts and paraphrasing seem to be procedures that the NMT system did not implement, which sometimes caused the output to be too literal". All this was largely ignored by the press, leading to the impression that a whole book can be translated with very high quality in a few hours ("For comparison, a normal 'human' translation would have taken a few weeks. The computer only took twelve hours", https://today.rtl.lu/news/ business-and-tech/a/1253287.html) 3 , Indeed the preparation of the data and the revision steps look like fundamental in this experiment, that was nevertheless interesting as an example of translation of a 3 See also, on the MILA website: "A team of experts has managed this masterstroke in just two and a half months, thanks to artificial intelligence (AI). In fact, most of the work was completed within 12 hours by a machine translation tool based on deep learning. The following 10 weeks were devoted to a "human" revision aimed at correcting any inaccuracies, along with providing clarity and nuance." https:// mila.quebec/livretraduitia/. See the term "masterstroke", and the absence of details about the preparation of the data. A quick search shows that at least 3 months have been devoted to prepare the experiment, cf. "Trois mois de développement ont été nécessaires pour préparer l'IA à ce défi" (https://tinyurl.com/y6tff9bp). The reusability of parts of this groundwork is possible, but remains uncertain. complex and technical text, whose length was also remarkable. Instead of "Invisibilizing" human work to prepare the data and revise the automatic translation, everybody would have benefited from putting this forward as a successful collaboration between humans and machines in the domain of translation.

NMT and Human Creativity

If NMT can accurately translate different kinds of texts, why not try with literature? Some experiments have recently been done in this domain [START_REF] Toral | Machine translation of novels in the age of transformer[END_REF]. Taking the same idea as a starting point, [START_REF] Fonteyne | Literary machine translation under the magnifying glass: Assessing the quality of an NMT-translated detective novel on document level[END_REF] evaluate a translation obtained using a standard NMT system (their corpus being the translation from English to French of a novel from Agatha Christie). They report issues like 'mistranslation', and problems with 'coherence' and 'style & register', according to a specific taxonomy developed for this purpose. Moreover, the problem is maybe more fundamental than this kind of evaluation suggests. Machine translation, by providing accurate translations of informational texts, leads to think that even literary translation is within reach. But NMT may just lack fundamental features of literary translation, precisely the ones put forward in [START_REF] Fonteyne | Literary machine translation under the magnifying glass: Assessing the quality of an NMT-translated detective novel on document level[END_REF]: an explicit encoding of coherence, style and register for example, even if recent systems try to integrate discourse features in the translation process [START_REF] Sennrich | Why the time is ripe for discourse in machine translation[END_REF]. Note that the notion of literary translation is in itself very imprecise: the translation of poetry is certainly out of reach, whereas the translation of some modern prose is probably easier than the translation of complex technical books, full of jargon. Translation quality should also always be discussed. For example, [START_REF] Matusov | The challenges of using neural machine translation for literature[END_REF] advocates that NMT could be useful for literature, but observing that "the quality is often high enough to understand and even enjoy the story" seems a very low standard for literature, that precisely goes beyond just transmitting information. As we see, discussing human parity does not make much sense if one does not also discuss style and creativity in a broader context. Here again, everybody would gain in having a deeper analysis of analogies and differences in ways of reasoning between machines and humans (whatever reasoning could mean for a machine).

NMT and Human translators

As we have seen, terms like "human parity" and "super human parity" are problematic because they suggest that machines can replace humans (this is explicitly stated in [START_REF] Popel | Transforming machine translation: a deep learning system reaches news translation quality comparable to human professionals[END_REF] for example: "deep learning may have the potential to replace humans in applications where conservation of meaning is the primary aim"). This may have some advantages, from a certain point of view. First, and even if this is obvious, it needs to be reminded: There are lots of contexts in which texts would not be translated and thus would not be accessible by people who do not know the target language. Machine translation makes these texts accessible, and this is certainly a good thing. On the other hand, machines have the potential to reduce time and costs, and of course, they already replace humans in some contexts, for example MT agencies already widely use NMT (and humans then more and more intervene as post-editors, and not as pure translators) [START_REF] Koponen | Is machine translation postediting worth the effort? a survey of research into post-editing and effort[END_REF]. But should this really be seen as a desirable feature? This situation leads to an understandable apprehension from translators, as detailed in [START_REF] Breyel | Machine translation and "biotranslation": an unlikely marriage[END_REF]: "there is a fear for translators of losing their jobs or seeing the price of their work drop. There is also a reluctance of professional translators towards postediting", this reluctance being mainly due to the fact that "post-editing is considered an alienating activity devoid of creativity." One can advocate that this is unavoidable and just corresponds to the evolution of techniques, like power looms automated the textile industry and led to less weavers during the 19th century, but the implication of this state of affairs should be discussed in detail. The idea that technology can replace humans cannot be put forward as a success without some precautions and some discussion. The complex relations between AI and the future of work is a question that goes beyond the scope of this paper [START_REF] Benhamou | Artificial intelligence and the future of work[END_REF]. It would be interesting to study more precisely if automatic translation is just a threat for translators, or if it can also open new doors by decreasing the cost of translation (for example for publishers who cannot afford to translate all their books, but could use MT to evaluate the potential of some books in a given language). The minimum is at least to consider these questions seriously and not just put forward the replacement of humans as an unquestionable desirable advantage of technology.

Conclusion

In this paper we have shown why terms like "human parity" or "super human performance" are misleading and do not bring much benefit to the evaluation process. We have examined several important features to consider, like the type of texts used for evaluation and the literal (vs idiomatic) nature of the foreseen translations. Finally, we propose to better take into human aspects in the evaluation process, and in particular try to see how human can be integrated, rather than just replaced by machines. Humans play an important role for MT, especially in operational contexts, for example by providing an expertise in the preparation of the data, establishing the terminology of a domain and correcting MT output during the post-edition phase. All this should be valued and the goal of NMT should not be to replace humans in the first place.

For example, in the tech magazine TechRadar: "Microsoft's new AI translates Chinese-to-English as well as a human translator" https://tinyurl.com/2mctxzvc