IV

LIST OF TABLES

 III-VII ACKNOWLEDGMENTS Foremost, I would like to sincerely thank my superiors, Joanna DAABOUL and Julien LE DUIGOU. They helped me a lot on my research presented in this thesis and my Ph.D life in Université de Technologie de Compiègne. They gave me chances to attend international conferences and winter school. They were always patient in revising the manuscripts of my thesis and submissions. They taught me how to become a professional researcher. With their guidance, I could successfully complete my research work and get the doctoral degree.

I would also like to thank my parents for understanding that I have not been back to visit them for the past three and a half years because of the global Coronavirus (COVID-19) pandemic, and for still loving me all the time. I would like to thank my boyfriend Tianchi Wang and his family. Thank them for their understanding and support of my choice to study abroad for my PhD.

I would also like to thank my colleagues and friends for their help and kindness, especially for not minding my poor French and being willing to communicate with me.

I am very grateful to my mentor before I came to France, Professor Wu Feng from the School of Management of Xi'an Jiaotong University. Thanks to his guidance in the two years before I went abroad, I was able to grow up quickly so that I could get a scholarship from the China Scholarship Council to study abroad.

Finally, I would like to thank the China Scholarship Council for providing me with a scholarship to support my study and life in France for 42 months. I am very grateful to my country for giving me this opportunity to study abroad.

INTRODUCTION Research context

Today's manufacturing companies are trying to survive in an ever-changing market. Some of the changes come from external events, such as the coronavirus disease 2019 (COVID- [START_REF] Koren | Reconfigurable manufacturing systems[END_REF] and the Russia-Ukraine conflict which are episodic and occur suddenly; others sprout from players within the market, namely consumers who want customized and sustainable products and producers who seek solutions to overproduction.

The disruption to manufacturing caused by COVID-19 has severely impacted the operational, social, and financial sectors and has challenged organizations to accelerate change in their global value allocation models [START_REF] Jain | A TISM approach for the analysis of enablers in implementing mass customization in Indian manufacturing units[END_REF]. Demand for certain types of products, such as sanitizer, masks and personal protective equipment kits, increased significantly, while demand for other types of products decreased significantly [START_REF] Pansare | Repurposing production operations during COVID -19 pandemic by integrating Industry 4 . 0 and reconfigurable manufacturing practices : an emerging economy perspective[END_REF]. The existing production capacity could not cope with the sudden and acute demand [START_REF] Tareq | Additive manufacturing and the COVID-19 challenges: An in-depth study[END_REF].

To survive in an environment of increasingly frequent and unpredictable change, such as market changes or unpredictable variables like the COVID-19 pandemic outbreak, manufacturers need to be able to respond quickly and cost-effectively to change [START_REF] Guo | Synchronizationoriented reconfiguration of FPAI under graduation intelligent manufacturing system in the COVID-19 pandemic and beyond[END_REF]. This requires new manufacturing solutions, among others, like reconfigurability and flexibility. The reconfigurability of manufacturing systems was investigated as a way to address the challenges posed by disruptions caused by public health emergencies [START_REF] Epureanu | An agile production network enabled by reconfigurable manufacturing systems[END_REF]. In response to the COVID-19 outbreak, many companies have been reconfiguring their plants and networks to meet the surge in demand for specific products such as masks, sanitizing gels and ventilators, with little reconfiguration [START_REF] Napoleone | Reconfigurable Manufacturing : Lesson Learnt from the COVID-19 Outbreak[END_REF].

In the post-COVID-19 era, all influential countries will have to decide to adopt a combination of traditional manufacturing, automated special-purpose and Industry 4.0 facilities [START_REF] Deshmukh | Framework for Manufacturing in Post-COVID-19 World Order: An Indian Perspective[END_REF]. The term "Industry 4.0" describes a project that can be defined by two development directions: huge application-pull and exceptional technology-push. Application-pull induces a remarkable need for changes triggered by short development periods, individualization on demand, flexibility, decentralization, and resource efficiency. The technology-push can be identified as further increasing mechanization and automation, digitalization and networking, and miniaturization in industrial practice [START_REF] Lasi | Industry 4.0[END_REF]. In a post-COVID-19 world, the plant of the future will need to be not only epidemic-proof, but also flexible enough to be reused or reconfigured when needed [START_REF] Malik | Reconfiguring and ramping-up ventilator production in the face of COVID-19: Can robots help?[END_REF].

Pandemic and other forms of epidemic outbreaks are a unique case of manufacturing risk, characterized by high levels of uncertainty, constant transmission, and long-term disruption to manufacturers, supply chain participants, and end users and consumers [START_REF] Okorie | Manufacturing in the Time of COVID-19: An Assessment of Barriers and Enablers[END_REF]. Prior to the COVID-19, it was clear that the urgency of environmental risks had outweighed economic concerns of business leaders [START_REF] Leitold | Flood risk reduction and climate change adaptation of manufacturing firms: Global knowledge gaps and lessons from Ho Chi Minh City[END_REF]. The scarcity of resources and ecological imbalances have attracted governments and manufacturing organizations to the sustainable development of manufacturing [START_REF] Hariyani | Organizational barriers to the sustainable manufacturing system: A literature review[END_REF].

Current consumption and production patterns are unsustainable, leading to overexploitation of natural resources, loss of critical habitats and biodiversity, and pollution of the land, oceans, and atmosphere [START_REF] Heintz | Don't Let Another Crisis Go to Waste: The COVID-19 Pandemic and the Imperative for a Paradigm shift[END_REF]. The Russian-Ukrainian conflict has exacerbated the volatility of commodity prices, undermining global economic growth already hit by the COVID-19 pandemic [START_REF] Fang | The Russia-Ukraine conflict and volatility risk of commodity markets[END_REF]. War can jeopardize the implementation of the Sustainable Development Goals (SDGs), especially SDG 1 (no poverty), SDG 2 (zero hunger) and SDG 12 (responsible and sustainable consumption and production) [START_REF] Ben Hassen | Impacts of the Russia-Ukraine War on Global Food Security : Towards More Sustainable and Resilient Food Systems ?[END_REF].

Therefore, new forms of changeable and reconfigurable manufacturing systems are needed to support sustainable and circular business models and supply chains [START_REF] Brunoe | Changeable manufacturing systems supporting circular supply chains[END_REF]. In today's manufacturing environment, it is critical to develop and implement manufacturing systems that are changeable and reconfigurable to manage and take advantage of increasing market volatility, product variety, customization, and smaller lot sizes [START_REF] Andersen | Engineering education in changeable and reconfigurable manufacturing: Using problem-based learning in a learning factory environment[END_REF]. Due to the wave of Industry 4.0, various industries are now adopting reconfigurability in their manufacturing systems [START_REF] Jamwal | Deep learning for manufacturing sustainability: Models, applications in Industry 4.0 and implications[END_REF].

A Reconfigurable Manufacturing System (RMS) is designed at the outset for a rapid

shift in structure, as well as in hardware and software components, to quickly adjust production capacity and functionality within a part family in response to sudden changes in the market or regulatory requirements [START_REF] Koren | Reconfigurable manufacturing systems[END_REF].

Learning from the facts that during COVID-19, the ongoing lockdowns that have passed and are ongoing have put a lot of pressure on the manufacturing industry to increase their sales [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF], and the decision of many companies not to pay for pre-produced goods and to stop orders [START_REF] Perret | Production Scheduling of Personalized Fashion Goods in a Mass Customization Environment[END_REF], a shift to make-to-order is beneficial for manufacturing companies to reduce losses from unforeseen crisis, even to balance productivity and stock sizes on a daily basis.

In fact, fewer and fewer production systems are organized exclusively by make-to-stock (MTS) [START_REF] Adan | Combining make to order and make to stock[END_REF]. MTS systems offer a small variety of manufacturer-specified products and usually lower-priced products [START_REF] Soman | Combined make-to-order and maketo-stock in a food production system SOM-theme A: Primary processes within firms[END_REF]. In the era of digital transformation, the market is not limited to lowprice competition when customers' demands are for tailored, affordable, and high-quality products [START_REF] Aheleroff | The degree of mass personalisation under industry 4.0[END_REF]. Today's market requires well-designed competitive solutions, as there is a vast array of products available to meet customer requirements [START_REF] Cunha | Design for cost: Module-based mass customization[END_REF]. Providing customized products with high customer satisfaction has become an inevitable trend for modern companies to remain competitive [START_REF] Li | Customer satisfaction evaluation method for customized product development using Entropy weight and Analytic Hierarchy Process[END_REF]. Retailers always want the latest "best-before" and inventory may easily expire due to the best-before nature of the product [START_REF] Van Donk | MTS or MTO : The DP in the food processing industry[END_REF]. When the focus of the production strategy shifts to customized orders/products, the importance is no longer to meet the MTS demand, but to ensure customized delivery [START_REF] Pereira | Merging make-to-stock/make-toorder decisions into sales and operations planning: A multi-objective approach[END_REF]. But customization can negatively affect manufacturing costs and delivery lead times [START_REF] Merle | Perceived value of the masscustomized product and mass customization experience for individual consumers[END_REF].

Customers highly value the development of customized products because such products meet their specific needs and desires [START_REF] Smets | Shouldn't customers control customized product development?[END_REF][START_REF] Shao | What is the right production strategy for horizontally differentiated product: Standardization or mass customization?[END_REF]. In response to this growing demand for personalization in an increasingly competitive market, more and more companies are considering mass customization as a new production strategy to increase their competitive advantage and are planning to transit to MC systems [START_REF] Shao | What is the right production strategy for horizontally differentiated product: Standardization or mass customization?[END_REF]. Global mass customization takes the integrated economy farther, driving more economies of scale [START_REF] Onken | Strategic Global Mass Customization: Supporting Generic And Global Strategies[END_REF]. The ability to mass customize is positively correlated with the ability to innovate products, which directly increases servitization, which is seen as an important way for manufacturers to gain competitive advantage [START_REF] Qi | Manufacturing practices and servitization: The role of mass customization and product innovation capabilities[END_REF].

Mass customization (MC)

is the new frontier in business for both manufacturing and service industries. MC at its core, is a tremendous increase in variety and customization without a corresponding increase in costs. At its limit, it is the mass production (MP) of individually customized goods and services, and at its best, it provides strategic advantage and economic value [START_REF] Pine | Mass customization : the new frontier in business competition[END_REF].

Da Silveira, Borenstein, and Fogliatto described enablers of MC implementation into processes and methodologies (agile manufacturing, supply chain management, customer-driven design and manufacturing, and lean manufacturing) and enabling technologies (advanced manufacturing technologies, and communication and networks) [START_REF] Da Silveira | Mass customization: Literature review and research directions[END_REF]. Suzić et al. considered eight enablers to implement MC: group technology, part standardization, product modularization, process modularity, product platform development, IT-based product configuration, form postponement and concurrent product-process-supply chain engineering [START_REF] Suzić | Implementation guidelines for mass customization: current characteristics and suggestions for improvement[END_REF].

Companies need new MC technologies, systems and methods to create individually customized products with MP quantities, cost and efficiency [START_REF] Smith | Editorial: Advances in mass customization[END_REF]. Stump and Badurdeen discussed the potential of applying several manufacturing concepts to meet MC manufacturing requirements such as, flexible/reconfigurable manufacturing systems [START_REF] Stump | Integrating lean and other strategies for mass customization manufacturing: A case study[END_REF].

In MC, product variability might need specific manufacturing solutions to be guaranteed [START_REF] Montalto | An approach to design reconfigurable manufacturing tools to manage product variability : the mass customisation of eyewear[END_REF]. MC is only possible when flexible manufacturing processes are supported by an adequate customer co-design system [START_REF] Piller | Mass Customization: Reflections on the State of the Concept[END_REF]. Flexible and reconfigurable manufacturing systems are used to create high diversity in the final assembly by combining assemblies, thus achieving economies of scope [START_REF] Hu | Evolving paradigms of manufacturing: From mass production to mass customization and personalization[END_REF]. RMS promises customized flexibility on demand in a short time, while Flexible Manufacturing Systems (FMSs) provide generalized flexibility designed for the anticipated variations and built-in a priori [START_REF] El Maraghy | Flexible and reconfigurable manufacturing systems paradigms[END_REF]. The productivity of FMSs is low and another big problem is that output capacity is determined at the time of purchase, hence building an FMS for a high-volume industry in a competitive market environment poses a significant business risk [START_REF] Koren | The invention of reconfigurable manufacturing systems -A survey of RMS early patents[END_REF]. RMSs aim to collect the key advantages of both traditional and flexible manufacturing systems, combining flexibility with high throughput [START_REF] Bortolini | Reconfigurable manufacturing systems: Literature review and research trend[END_REF]. RMS is a system that can be easily modified (reconfigured) to meet demand, rather than using as many features as possible to cover as many requirements as possible (which is a flexibility-oriented approach). This permits facing the customer's wishes while maintaining high production volumes [START_REF] Urbani | Flexibility and Reconfigurability for Mass Customization[END_REF]. The reconfiguration of the production system makes it possible to change its production speed and product type with minimum cost and time [START_REF] Kombaya | Customization measurement in reconfigurable manufacturing systems (RMS)[END_REF].

Reconfigurable machine tools (RMT) are designed to be quickly customized to produce the desired product or product mix in the required quantities [START_REF] Molina | Next-generation manufacturing systems: Key research issues in developing and integrating reconfigurable and intelligent machines[END_REF]. This type of systems would provide the flexibility to customize for a specific part family and would be open-ended so it could be improved, upgraded, and reconfigured rather than replaced [START_REF] Mehrabi | Reconfigurable manufacturing systems: Key to future manufacturing[END_REF].

The implicit challenge for manufacturers is to deal with the high demand uncertainty arising from the provision of many variants while ensuring the maintenance of low operational costs and short and reliable delivery times [START_REF] Skipworth | Implications of form postponement to manufacturing a customized product[END_REF]. The complexity of the system and the efficiency requirements of the developed strategies/algorithms are also challenges [START_REF] Koren | Reconfigurable manufacturing systems: Principles, design, and future trends[END_REF]. At the production planning and control level, existing concepts and practices for different support functions, such as capacity planning, process planning, and scheduling, must be further developed and evolved to meet these challenges [START_REF] Azab | Reconfiguring Process Plans: A New Approach to Minimize Change[END_REF]. This work contributes to this research field by confronting the complexity of operation research arising from RMS reconfigurability and exploring the machine and layout reconfiguration in the optimization of MC production in a short planning horizon.

Research challenges

Production planning is one of the most attractive and important issues in manufacturing systems. It tries to effectively plan and coordinate all manufacturing activities in order to achieve the company's goals [START_REF] Tirkolaee | Multi-objective aggregate production planning model considering overtime and outsourcing options under fuzzy seasonal demand[END_REF]. Production planning includes deciding on the best use of available resources [START_REF] Dillenberger | On practical resource allocation for production planning and scheduling with period overlapping setups[END_REF].

For production in an RMS with several reconfigurable machines, the production planning needs to decide the machine with its configuration to perform each operation. This research is dedicated to solving production planning for MC products, hence it falls under the short-term production planning. The operation sequences to process each part should also be decided in production planning, as well as their processing sequences on machines. Given that the layout of an RMS for a family of MP variants must be adaptable to unexpected demands in almost every respect [START_REF] Heisel | Progress in Reconfigurable Manufacturing[END_REF], the decision to change machines' positions in the workshop for a specific production task is considered in production planning.

Decisions to select machine and configuration for operations and to determine the operation sequence to process parts belong to the process planning problem. Decisions to determine the processing sequences on machines and their beginning time belong to the scheduling problem. Decisions to change machines' positions in the workshop belong to the layout problem. These problems are highly interconnected and interdependent. They are also more complicated when the production task is an MC order having multiple customized-products, each composed of multi-unit parts belonging to different part variants. Indeed, MC and variety management are complex topics that have been extensively discussed in the scientific literature [START_REF] Medini | Teaching customer-centric operations management-evidence from an experiential learning-oriented mass customisation class[END_REF]. The challenge is to find the one production plan that meets the manufacturer's goals from among the many viable options.

Every customized-product has a customer lead time and thus a due date for completion of that product. If a product is not completed on time, the corresponding customer will be dissatisfied. The tardiness can be quantified by an economic penalty. This helps the manufacturer to find a suitable production plan to improve customer satisfaction. In addition, the prices of the mass-customized products cannot be high, hence the manufacturer has to find a production plan with the lowest possible total cost to gain more profit.

This poses two problems. Firstly, there is not necessarily a production plan that can satisfy both maximum customer satisfaction and minimum total cost. Secondly, so far, there has been no fully-developed strategy to balance the tradeoff between maximum customer satisfaction and minimum total cost.

The production planning problem is one of the most interesting applications of optimization tools using mathematical programming [START_REF] Mula | Models for production planning under uncertainty: A review[END_REF]. Considering all the above statements, the main research problem addressed in this work can be summarized by the following question:

knowing the customized demand and thereby the required parts to produce, how to optimize their process planning, the RMS machine and layout configuration, and production scheduling to minimize overall costs as well as to complete the customized products on time?

Besides, human awareness of sustainable development is growing worldwide [START_REF] Li | Internet-based intelligent and sustainable manufacturing: developments and challenges[END_REF]. A number of indicators that measure environmental sustainability are emerging as a new category of goals for manufacturers. For RMS, it is not only a new manufacturing model that offers the flexibility of customization, it is also the basis for developing a new generation of sustainable production systems [START_REF] Battaïa | Sustainable and Energy Efficient Reconfigurable Manufacturing Systems[END_REF]. The higher the degree of RMS adoption -that is, the more reconfigurable the manufacturing system within an organization -the higher its environmental performance [START_REF] Dubey | Explaining the impact of reconfigurable manufacturing systems on environmental performance: The role of top management and organizational culture[END_REF]. Therefore, the above research question should be and is addressed by considering some environmental sustainability indicators.

Research objective

This research's main objective is to generate a production plan that can produce multiple mass-customized products in RMS at low cost and finish them on time as much as possible, thus satisfying customers. For that, the tardiness penalty of production delay is the first objective to be considered. The total cost is the second objective considered. It covers the related costs to operations processing, WIP transporting and holding, machine reconfiguration and layout reconfiguration activities.

Main contributions

Considering the issues previously highlighted, this research focused on proposing a model and solution approach for production planning in an RMS producing mass customization products. For MC, customer satisfaction in terms of delivery time is as important as reducing overall cost. Hence, this leads to multi-objectives decision making. The main contributions of this research are described hereafter:

1. This research considered the reconfiguration of machine layout in production planning and control, rather than in the RMS design phase, which fills the research gap. The mathematical model developed to take this consideration into account is novel.

2. This research adopted some heuristics to solve the formulated problem. A difficulty is to handle the infeasible solutions generated during iterations. A problem-specific constraint handling technique is put forward to repair infeasible solutions that are not subject to flexible job-shop scheduling constraints. This technique can be adopted in any heuristic. The idea of how to properly repair a sequence is universal for a series of operation precedence considered job-shop scheduling problems in a multi-machine manufacturing system.

3. The general Pareto efficiency gives rise to redundant solutions to confuse decision makers when the number of objectives increases. This is observed in the presented study concerning the environmental sustainability in this research problem. Simply adding an environment-oriented objective to the dual-objective minimizing time and cost-oriented objectives does not assist manufacturers in decision-making on MC in RMS but increases their confusion. Therefore, the general Pareto efficiency is modified to get a reasonable number of Paretooptimal solutions for decision makers.

4. The performance of a heuristic strongly depends on the parameter setting and fluctuates widely in solving problems of different scales. To develop an intelligent solution approach, a Q-learning based reinforcement learning approach is proposed to solve the planning and scheduling integrated optimization for MC in RMS. Compared with the heuristic algorithms that are widely used in most studies to solve this research topic, the solution approach developed in this study is less parametric, as a consequence it requires less expertise on parameters setting. And to the best of my knowledge, there is no research using reinforcement learning methods to solve multi-objective optimization in this research field. This research aims to leverage the reconfigurability of RMS to achieve MC. It is expected that the developed solution approaches are effective and efficient to solve the research problem and facilitate practical MC implementation in the reconfigurable manufacturing environment.

Research structure

This thesis is organized in four chapters (highlights of each chapter are shown in Figure 1), added to an Introduction, a General Conclusion and Appendices sections. General Conclusion addresses limits and the perspectives of this research. Appendices provide some tables to complement Pareto-optimal solutions not presented in the other chapters.

Chapter I begins with a state of the art of MC and RMS. Then presents a literature review on planning, scheduling and layout optimization of RMS. Finally, it summarizes the research objectives, models and solution approaches of the selected research works on planning, scheduling and layout optimization in RMS.

Chapter II first describes the problem formulation. Then introduces a bi-objective model and solution approaches, including the exact solution approach which is exhaustive search, two approximate solution approaches which are Non-dominated sorting genetic algorithm II (NSGA-II) and Archived multi-objective simulated annealing (AMOSA), and constraint handling techniques. Finally, it presents the numerical experiment results and analysis to validate the mathematical model and solution approaches.

Chapter III first describes sustainable manufacturing and the two environment performance indicators usually used in operation research of environmental sustainability optimization. Then, it introduces an adapted tri-objective mathematical model and the modified Pareto efficiency. Finally, this chapter presents the numerical experiment results and analysis to show the significance of considering environmental indicators in the optimization of this problem and to examine the performance of the adopted approximate solution approach, Nondominated sorting genetic algorithm III (NSGA-III).

Chapter IV first describes the process planning and scheduling problem of an MC order production as a finite Markov decision process. Then introduce the reduced bi-objective mathematical model and the Q-learning based reinforcement learning approach. Finally, it presents the numerical experiment results and analysis to validate this proposed new solution approach.

Figure 1 Summary of all chapters presented in this research.

CHAPTER I -LITERATURE REVIEW

This chapter first defines and describes MC and RMS in Sections I.1 and I.2, respectively. Then Section I.3 discusses how to implement MC in RMS. After that, a literature review on planning, scheduling and layout optimization of RMS is presented in Section I.4.

Finally, the literature review is analyzed in Section I.5.

I.1 -Mass Customization

MC allows customers to have products with certain custom features, while still maintaining the cost and efficiency to produce them at close performance as that of mass production. Because of this, it improves price, profit, and company's success [START_REF] Smith | Mass customization in the product life cycle[END_REF]. Overall, it can be said that MC has become a consolidated strategy in industrial manufacturing engineering/operations management and related fields, with a large number of successful implementations both in academia and in the real world [START_REF] Fogliatto | The mass customization decade: An updated review of the literature[END_REF]. Companies such as Dell Computers, Motorola, IBM, 3Com, Sun Microsystems, Proctor and Gamble, Toyota, General Motors, Ford, Chrysler, and Hewlett-Packard have all chosen to use MC in their production [START_REF] Selladurai | Mass customization in operations management: Oxymoron or reality?[END_REF].MC is enabled through modular product architectures, from which a wide variety of products can be configured and assembled, forasmuch the literature on MC mentions modularity as one of its main enablers [START_REF] Mikkola | Management of product architecture modularity for mass customization: Modeling and theoretical considerations[END_REF].

Modularity as a main enabler for mass customization

MC is a strategy that seeks to exploit the need for greater product variety and individualization in markets [START_REF] Mccarthy | Special issue editorial: The what, why and how of mass customization[END_REF]. However, it also induces additional costs for consumers, and has a customization lead time trade-off [START_REF] Jost | Company-customer interaction in mass customization[END_REF]. Rapid (at least punctual) delivery and fair price are essential for customers [START_REF] Hvam | Mass customisation in the electronics industry: based on modular products and product configuration[END_REF]. In this context, research has found that such trade-offs can be properly managed by exploiting the module-based product family design [START_REF] Gauss | Module-based product family design: systematic literature review and meta-synthesis[END_REF]. Modular product architectures can be seen as a tool to develop, maintain, and manage the product assortment and, hereby, obtaining product variety in a cost effective manner [START_REF] Joergensen | Reconfigurable manufacturing systems as an application of mass customisation[END_REF]. Modularity assumes a particular product's functional and physical architecture and is credited with many of the products' successes, particularly low production costs [START_REF] Ulrich | Fundamentals of product modularity[END_REF]. Product modularity allows for a various product portfolio through the substitution and matching of different modules. By sharing most of the modules, products could achieve economies of scale in production at the module level, thus well solving the contradiction between economies of scale and product differentiation under the MC paradigm.

The essence of MC lies in the product and service developers' ability to perceive and capture latent market niches and subsequently to develop technical capabilities to meet the diverse needs of target customers [START_REF] Jiao | Towards high value-added products and services: Mass customization and beyond[END_REF]. In addition to the module design of products in MC, many research have focused on module-based service in MC. Moon et al. utilized common modules, variant modules, and unique modules for service family design and investigate strategic module sharing among services to create a service platform using a game-theoretic approach [START_REF] Moon | A module-based service model for mass customization: Service family design[END_REF]. J. H. Li established a primary architecture of MC-based service product innovation in service package and service product module level [START_REF] Li | Strategy of mass customization-based service product innovation[END_REF]. Aggarwal, Chan, and Tiwari have applied strategic sharing of modules in service family design using coalition game and Cournot game theories for improving the airline sector by introducing cost efficient and relatively high payout services [START_REF] Aggarwal | Development of a module based service family design for mass customization of airline sector using the coalition game[END_REF]. Xu, Fan, and Wang established a MC-oriented Customer Demand Response service platform, which is a group of application modules that provide services to the caller through external interfaces, transmit, control, and manage data through internal modules, and contribute to the loose coupling and integration between different applications [START_REF] Xu | Mass Customization-Oriented Customer Demand Response Service Platform Based on Cloud Computing and Internet of Things[END_REF].

In order to fulfil the personalized requirements of customers, modern manufacturing industries are facing more requirements for production flexibility, and how to carry out the module partition for production line is becoming the key for solving the problem [START_REF] Ren | Research on assembly module partition for flexible production in mass customization[END_REF]. Process Modularity is the practice of standardizing manufacturing process modules so that they can be resequenced easily or new modules can be added quickly in response to changing product requirements [START_REF] Tu | Measuring modularity-based manufacturing practices and their impact on mass customization capability: A customer-driven perspective[END_REF]. Modrak and Soltysova firstly analyzed relationship between product modularity and process modularity, then compared potential process modularity indices and assessed them on testing examples, finally formulated theoretical principles for designing manufacturing and assembly process structures [START_REF] Modrak | Process modularity of mass customized manufacturing systems: Principles, measures and assessment[END_REF]. They also proposed a new method to measure relative modularity of different assembly process structures for exploring the problem of quantification of process modularity degree [START_REF] Modrak | Development of the Modularity Measure for Assembly Process Structures[END_REF]. Z. Wang et al. empirically tested the mediating effects of two organizational learning practices-customization knowledge utilization and business process improvement-on the relationship between product/process modularity and MC capability, using data collected from Chinese manufacturers [START_REF] Wang | Modularity in building mass customization capability: The mediating effects of customization knowledge utilization and business process improvement[END_REF]. Viana, Tommelein, and Formoso illustrated how modularity can reduce the complexity of engineeredto-order industrialized building systems, in companies that adopt a MC strategy and demonstrated the need to adopt an integrated product and process-oriented conceptualization of modularity in industrialized building systems [START_REF] Viana | Using modularity to reduce complexity of industrialized building systems for mass customization[END_REF].

As the modularity for MC has well developed, it is necessary to build a platform to choose the modules that satisfy the customers' preferences to compose the customized products they want. Internet-enabled e-commerce provides capabilities for firms to reach global buyers and facilitates MC by gathering customers' preferences [START_REF] Helms | Technologies in support of mass customization strategy: Exploring the linkages between e-commerce and knowledge management[END_REF], thus E-mass customization emerges.

E-mass customization (E-MC)

The customization can be achieved effectively if the customers' requirements are characterized and captured [START_REF] Tseng | Design for Mass Customization[END_REF]. Effective definition of customer requirements is a pre-requisite for realizing MC [START_REF] Tseng | Design by customers for mass customization products[END_REF]. Realizing product customization as 'just a mouse click away' is the ultimate dream of many organizations and customers [START_REF] Yassine | Investigating the role of IT in customized product design[END_REF]. Technological advances in production systems and customer interfaces have enabled firms to offer affordable and timely mass customized products [START_REF] Hunt | Individual differences in consumer value for mass customized products[END_REF]. A well thought out and a well implemented MC strategycomplete, stand-alone in itself -would still deliver impressive financial results, because of the clear structure of production systems, web-interface for customer co-design, and operational choices associated with such a strategy [START_REF] Kumar | Mass customization: Metrics and modularity[END_REF].

Web interfaces are the ideal tool for a dialogue with the potential customer, providing necessary information about the product and collecting his/her preferences in a totally automated way [START_REF] Aichner | Customers' online shopping preferences in mass customization[END_REF]. As the integrated Internet marketplace progresses toward cross e-tail partnerships, service providers and e-tail initiatives will be differentiated by an ability to develop powerful, customer-centric Web interfaces that support the bundling of products and services within a customer-controlled customization process [START_REF] Grenci | Maximizing customer value via mass customized econsumer services[END_REF]. In setting up a Web-based interface for design and MC, a significant challenge is to help customers formulate their implicit requirements and make them explicit on the Web. Therefore, Web-based design must realize design options that lead to the design of useful products fulfilling customers' general needs [START_REF] Khalid | Mass customization and Web-based do-it-yourself product design[END_REF].

Options come from the initial pre-design that is to be modified by the amateur with the help of the provided design kit as part of the offered Do-It-Yourself environment, while the base structure of the product was to be predefined and even presumed to be mass manufactured [START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF]. Advances in information and communication technologies allow customers to select from list(s) of pre-planned and pre-designed product features and options using an online product configurator [START_REF] Elmaraghy | Product variety management[END_REF]. For example, the Land's End E-MC program offers many steps to customize a pair of pants, allowing customers to select the length, waist size, color, and other detailed style options [START_REF] Yoo | The effects of e-mass customization on consumer perceived value, satisfaction, and loyalty toward luxury brands[END_REF]. An electronic configure-to-order platform supports customization by allowing customers to create customized products via the Web, enabling guided product selection and configuration, facilitating variant production planning, and integrating with enterprise business systems [START_REF] Jiao | Development of an electronic configure-to-order platform for customized product development[END_REF].

The COVID-19 pandemic challenged the garment industry dramatically which will accelerate its shift to MC for it offers digital online options [START_REF] Sun | Research on the Technology of Mass Customization of Clothing[END_REF]. The adoption of Online selfcustomization strategy is found in a wide range of industries including apparel (e.g., Levi Strauss), sport shoes (e.g., Adidas), computers (e.g., Dell), cars (e.g., Land Rover), food (e.g., General Mills), cards (e.g., Hallmark), etc. [START_REF] Kwon | How online self-customization creates identification: Antecedents and consequences of consumer-customized product identification and the role of product involvement[END_REF]. It has been envisioned that e-commerce and MC will emerge as a primary style of manufacturing in the coming decade and beyond [START_REF] Helander | Research on E-product development (ePD) for mass customization[END_REF]. In any case, E-MC is seen as a monolithic web-based branding strategy which involves a customer co-design process and MP process by combining customers' individual needs with their online shopping experience and mass process effectiveness [START_REF] Yan | A Review of E-mass Customization as a Branding Strategy[END_REF]. This is supported by digital plants developing virtual product models and manufacturing processes. Manufacturing operations can be simply recalled from an existing virtual engineering process of a family of similar products [START_REF] Ahmed | Smart virtual product development (SVPD) to enhance product manufacturing in industry 4.0[END_REF].

Digital manufacturing

The driving forces for adapting digital technologies are time to market (short product development time), improved productivity (in terms of quality and reduced wastage), and managing cost and customized product requirements [START_REF] Paritala | Digital Manufacturing-Applications Past, Current, and Future Trends[END_REF]. The ability to hold stock as digital data and print on demand has potential for manufacturing the modular components commonly used for MC on demand [START_REF] Reeves | Additive Manufacturing for Mass Customization[END_REF]. In general, Digital Manufacturing/Industry 4.0 can be introduced as a sequence of industrial innovations beginning with the first and second industrial revolution, which was based on mechanization; the third industrial revolution, based on automation that focused on lean production; and the fourth industrial revolution, which is embedding the virtualization characterized by CPS, smart machines, smart products, and mobile devices, resulting in the smart factory paradigm [START_REF] Möller | Digital Manufacturing/Industry 4.0[END_REF]. The concept of digital manufacturing enables digital archiving of the design and manufacturing information associated with the part [START_REF] Gibson | Direct Digital Manufacturing[END_REF].

Digital manufacturing is a key for adaptation and is based on modern tools and techniques for engineering, control, supervision and management in a network [START_REF] Westkämper | Digital Manufacturing In The Global Era[END_REF]. A basic process of digitization is to convert nuggets of complex information into measurable figures or data, and then modeling these data, so that to eventually convert the information to a series of binary data, that could be used as input to computers for processing [START_REF] Chong | A review of digital manufacturing-based hybrid additive manufacturing processes[END_REF]. Digital manufacturing and digital factory concepts in the pre-industry 4.0 era focus mainly on product lifecycle management technologies as computer-aided design, engineering, process, product data and life-cycle management, simulation and virtual reality, process control, shop floor scheduling, decision support, decision-making, manufacturing resource planning, enterprise resource planning, logistics, supply chain management, and e-commerce systems [START_REF] Gerrikagoitia | Digital manufacturing platforms in the Industry 4.0 from private and public perspectives[END_REF]. A cloud broker (e.g., cloud-based storage and computing brokers) can help users identify, customize, and integrate existing design and manufacturing services, so as to improve the negotiation process between service providers and consumers as well as enhance security and privacy in cloud-based design manufacturing systems, which can be considered as a new, emerging paradigm that will revolutionize digital manufacturing and design innovation [START_REF] Wu | Cloud-based design and manufacturing: A new paradigm in digital manufacturing and design innovation[END_REF].

Digital manufacturing systems enable users to fabricate freeform materials, which lead to new functionalities and applications [START_REF] Lin | Digital manufacturing of advanced materials: Challenges and perspective[END_REF]. The three-dimensional printing process is considered as the direct digital manufacturing technology employed for making a functionally graded material prototype [START_REF] Chiu | Direct digital manufacturing of three-dimensional functionally graded material objects[END_REF]. This is an interconnection of (decentralized) additive manufacturing equipment and modern information and communication technology [START_REF] Chen | Direct digital manufacturing: Definition, evolution, and sustainability implications[END_REF]. Its technologies include both novel 3D printing and the more conventional digitally controlled machines [START_REF] Lyly-Yrjänäinen | Effects of combining product-centric control and direct digital manufacturing: The case of preparing customized hose assembly kits[END_REF]. Direct digital manufacturing systems are an innovative and on-demand technology that represents game changing advances to supply chains, consumer goods and economic growth [START_REF] Glavach | Applying and Assessing Cybersecurity Controls for Direct Digital Manufacturing (DDM) Systems[END_REF]. It is a combination of product modeling and manufacturing technology that directly converts digital models to physical objects without the need for tooling [START_REF] Holmström | Sustainability outcomes through direct digital manufacturing-based operational practices: A design theory approach[END_REF]. To date, direct digital manufacturing has primarily been used for prototypes and tooling, and to a limited degree for customized products [START_REF] Holmström | The direct digital manufacturing (r)evolution: definition of a research agenda[END_REF]. It offers MC and enables goods that are needed to be produced when and where they are needed [START_REF] Mitchell | Climate Change and Manufacturing[END_REF].

Conclusion

To conclude, MC has become a reality and is offered by several industries. The diversity of mass-customized products is offered by product and process modularity. E-MC opens up the way for information sharing to improve customer satisfaction. Enterprise-wide digital transformation of manufacturing operations provides effective digital product and asset allocation management, helping to improve production efficiency and reduce costs for MC.

Finally, in the context of multi-variety and low-volume, smart factories welcome flexible organizations that can be reconfigured and optimized in the manufacturing system. This translates into the need of reconfigurable manufacturing systems to efficiently and optimally provide mass customized goods.

I.2 -Reconfigurable Manufacturing Systems

Increasing product variants and customizable products request more flexible production systems [START_REF] Stark | Innovations in digital modelling for next generation manufacturing system design[END_REF]. The development of the early form of computers, such as numeric controllers, gave birth to automation and provided more flexibility to equipment such as industrial robots and machine tools [START_REF] Lee | Predictive manufacturing system -Trends of next-generation production systems[END_REF]. The FMS was presented as a new alternative in the venerable domain of manufacturers that use expensive computer numerical control machines with fixed hardware and software to produce a variety of products and product mixes [START_REF] Singh | Justification for the selection of a reconfigurable manufacturing system: A fuzzy analytical hierarchy based approach[END_REF]. However, the majority of users are not satisfied with FMS because of a variety of problems, including its lack of reconfigurability (i.e., its fixed capacity and functionality) [START_REF] Mehrabi | Trends and perspectives in flexible and reconfigurable manufacturing systems[END_REF]. This led to the development of reconfigurable manufacturing systems (RMS).

RMS is designed based on multiple part families, providing customized flexibility for all types of parts in the family, and is built with an adaptable structure that can be reconfigured for adjustments in productivity and functionality. It is encouraging for the next generation manufacturing systems. Next generation manufacturers must realize that competition in the twenty-first century is going to be different than before [START_REF] Soliman | The impact of some recent developments in e-business on the management of next generation manufacturing[END_REF]. To successfully develop and implement the next generation of products and services, industry must be successful in generating new product ideas as well as having the ability to quickly realize these into successful products and competitive production systems [START_REF] Jackson | Factory-in-a-box -Demonstrating the next generation manufacturing provider[END_REF]. Following, will be discussed how RMS is THE next generation manufacturing system, and the key characteristics of RMS with a focus on modularity. Then, the reconfigurability and the control in RMS will be respectively and briefly discussed.

RMS as the next generation manufacturing system

The modern industry requires the next generation of manufacturing systems to be intelligent, flexible, and interoperable [START_REF] Wang | Manufacturing System on the Cloud: A Case Study on Cloud-based Process Planning[END_REF]. Both FMS and RMS are considered to be the next generation manufacturing systems. An FMS is an integrated, computer-controlled complex of automated material handling devices and numerically controlled machine tools that can simultaneously process medium sized volumes of a variety of part types [START_REF] Stecke | Formulation and Solution of Nonlinear Integer Production Planning Problems for Flexible Manufacturing Systems[END_REF]. The FMSs activate the automatic operations using pre-instructed programs, therefore, these FMSs have difficulty reaching the requirements of flexibility, adaptability, and reliability [START_REF] Tran | Development of a smart cyber-physical manufacturing system in the Industry 4.0 context[END_REF].

Cellular manufacturing, which is an FMS, can respond to the increasingly competitive environment facing manufacturers [START_REF] Rezaeian | A hybrid approach based on the genetic algorithm and neural network to design an incremental cellular manufacturing system[END_REF]. It focuses on the creation and identification of manufacturing cells, which are dedicated to the processing of a set of part families [START_REF] Ponnambalam | Design of cellular manufacturing systems using objective functional clustering algorithms[END_REF].

Cellular manufacturing is an application of the group technology in manufacturing and involves processing a collection of similar parts (part families) on dedicated cluster of machines or manufacturing processes (cells) [START_REF] Singh | Design of cellular manufacturing systems: An invited review[END_REF]. Focusing on cellular manufacturing systems, once machine cells are designed, the physical relocation of the facilities included in each cell in response to new production requirements becomes difficult [START_REF] Bortolini | Reconfigurability in cellular manufacturing systems: a design model and multi-scenario analysis[END_REF].

The new paradigm needed by today's manufacturing industries should incorporate the advantages of FMS but also be simpler, responsive, and less costly [START_REF] Setchi | Reconfigurability and reconfigurable manufacturing systems -State-of-the-art review[END_REF]. RMS has emerged as a cost effective mechanism that helps the manufacturing industries to stay competitive in a global scenario where economic pressures and varying product and production demands are perpetually changing in a fiercely dynamic environment [START_REF] Singh | Reconfigurable manufacturing systems: journey and the road ahead[END_REF]. In the next-generation of sustainable manufacturing, it is necessary to bridge manufacturing processes with sustainability in design and optimization. Also, some quantifiable models for wastes and energy are essential [START_REF] Bi | Revisiting system paradigms from the viewpoint of manufacturing sustainability[END_REF]. RMS and its characteristics of scalability, convertibility, diagnosability, customization, modularity and integrability have emerged as a basis for living factories for next generation manufacturing that can significantly enhance the system sustainability by quickly adjusting system configuration and production processes to meet the market needs, and maintain the system values for generations of products [START_REF] Koren | Sustainable Living Factories for Next Generation Manufacturing[END_REF].

The autonomous mobility and intelligent decisions in case of unexpected events in production will be of essential importance in the next-generation manufacturing systems [START_REF] Brezocnik | Emergence of intelligence in nextgeneration manufacturing systems[END_REF], which are conceived to be intelligent enough to take decisions and automatically adjust themselves to situations such as variations in production demand and machine breakdowns [START_REF] Srivinas | Solving the machine-loading problem in a flexible manufacturing system using a combinatorial auction-based approach[END_REF]. Integrated manufacturing system consisting of engineering design, process planning, manufacturing, quality management, and storage and retrieval functions is a novel manufacturing environment which has been developed for the next generation of manufacturing and processing technologies [START_REF] Gen | Evolutionary techniques for optimization problems in integrated manufacturing system: State-of-the-art-survey[END_REF]. Next-generation manufacturing execution system can improve process performance through 'visibility' in manufacturing operations [START_REF] Mantravadi | An overview of next-generation manufacturing execution systems: How important is MES for industry 4.0?[END_REF]. Agentbased technology has been identified as an important approach for developing those [START_REF] Shen | Enhancing the performance of an agentbased manufacturing system through learning and forecasting[END_REF], for which Kumar et al. put forward a pioneering agent-based integrated yet distributed operations planning approach [START_REF] Kumar | Integrated yet distributed operations planning approach: A next generation manufacturing planning system[END_REF].

It is visionary that customers place orders for a great variety of products in small quantities, which requires the "virtual factory" to be both flexible and responsive [START_REF] Qiu | Manufacturing grid: a next generation manufacturing model[END_REF]. Virtual reality will play an important role in the next generation of manufacturing systems because it allows simulation results to be shown in a realistic, intuitive, economic and safe way [START_REF] Rubio | Virtual reality applications for the nextgeneration manufacturing[END_REF]. The purpose of so doing is to indicate how virtual environments have potential to realize a step change in best practice industry wide; such that the full life cycle engineering of next generation manufacturing systems can lead to business benefits [START_REF] Weston | Next generation manufacturing systems[END_REF]. The virtual enterprises concept offers the necessary theoretical foundations for the next generation manufacturing enterprises, providing not only reference models or approaches, but also sustainable business models [START_REF] Romero | Next generation manufacturing systems and the virtual enterprise[END_REF].

Scalability (design for capacity changes):

The ability to easily modify production capacity by adding or subtracting manufacturing resources (e.g. machines) and/or changing components of the system. 4. Modularity (components are modular): The compartmentalization of operational functions into units that can be manipulated between alternate production schemes for optimal arrangement. 5. Integrability (interfaces for rapid integration): The ability to integrate modules rapidly and precisely by a set of mechanical, informational, and control interfaces that facilitate integration and communication.

6. Diagnosability (design for easy diagnostics) : The ability to automatically read the current state of a system to detect and diagnose the root causes of output product defects, and quickly correct operational defects.

Three are system operational characteristics (scalability, convertibility and diagnosability) and the other three are system structural characteristics (modularity, integrability and customization) [START_REF] Koren | The Emergence of Reconfigurable Manufacturing Systems (RMSs)[END_REF]. It is also noted that these characteristics do not apply solely to the system level, but also to other components of the reconfigurable system such as reconfigurable machines and at even lower levels to machine elements [START_REF] Saliba | A heuristic approach to module synthesis in the design of reconfigurable manufacturing systems[END_REF]. While convertibility and customization help reducing the cost, the effort and the time of reconfiguration are reduced through integrability, diagnosability and modularity [START_REF] Benderbal | A new robustness index for machines selection in Reconfigurable Manufacturing system[END_REF]. Each of these characteristics has a role in the ability of RMS to be flexible, productive and profitable [START_REF] Capawa Fotsoh | A Classification for Reconfigurable Manufacturing Systems[END_REF]. Only when a manufacturing system possesses these six key characteristics, can it achieve the manufacturing goals of low cost, high quality, and rapid responsiveness [START_REF] Wang | Reconfiguration schemes evaluation based on preference ranking of key characteristics of reconfigurable manufacturing systems[END_REF]. Therefore, a lot of research study these characteristics to design or assess the RMS.

Farid has built upon the recently developed reconfigurability measurement process to produce measures of reconfigurability and its key characteristics of integrability, convertibility, and customization as applied to manufacturing systems [START_REF] Farid | Measures of reconfigurability and its key characteristics in intelligent manufacturing systems[END_REF]. Koren, Wang, and Gu elaborated on the economic value of scalability in RMS, and developed design-for-scalability principles and a systematic approach for scalability planning, so that the exact capacity needed to satisfy surging market demand could be added to the system at a minimum investment cost [START_REF] Koren | Value creation through design for scalability of reconfigurable manufacturing systems[END_REF].

Huang, Wang, and Yan proposed a delayed RMS to support partial production activities during reconfiguration and to ease the convertibility process without losing the scalability characteristic of RMS [START_REF] Huang | Delayed reconfigurable manufacturing system[END_REF]. Khan examined the diagnosability characteristic in a multi-stage RMS and analyzed it regarding product variation and system diagnosability to understand the impact of time-based diagnostics on the functionality performance of an RMS and the level of inventory used during production [START_REF] Khan | Problem-Specific Heuristics for Diagnosability and Inventory Analysis in a Reconfigurable Manufacturing System[END_REF].

Modularity in RMS

The design for changeable manufacturing systems contains three interrelated elements that is changeability need analysis, modularity design, and platform design [START_REF] Andersen | Towards a generic design method for reconfigurable manufacturing systems: Analysis and synthesis of current design methods and evaluation of supportive tools[END_REF]. Modularity is a key characteristic of an RMS and a basis for product design as well as the process design of an RMS, which allows RMSs to change a product without major changes in the process [START_REF] Abdi | Feasibility study of the tactical design justification for reconfigurable manufacturing systems using the fuzzy analytical hierarchical process[END_REF].

Modularity in the product design stage as well as in the process design stage enables an RMS to produce different product families with common resources by means of different configurations [START_REF] Abdi | A design strategy for reconfigurable manufacturing systems (RMSs) using analytical hierarchical process (AHP): A case study[END_REF]. The key feature of RMS is that its capacity and functionality are modular and not fixed, thus they can be integrated (added) to the system and removed to adapt for the market demand through what may be called capacity and functionality scalability [START_REF] Deif | A Systematic Design Approach for Reconfigurable Manufacturing Systems[END_REF].

Modules are assembled into configurations of RMTs. The capacity of the RMT (modular structure) can be changed by adding/removing spindle, which has a maximum capacity [START_REF] Huang | Delayed reconfigurable manufacturing system[END_REF]. The object of designing RMTs includes not only the modules that form them, but also their configuration. G. Wang et al. constructed a configuration tree with various nodes representing the modules of RMT configuration and edges representing the assembly relationship among modules, then proposed a tree-based method to determine the configuration design for reconfiguration of an RMT [START_REF] Wang | A tree-based decision method for the configuration design of reconfigurable machine tools[END_REF]. Ming et al. built a knowledge base by developing an ontology to formally represent the taxonomy, properties, and causal relationships of/among three domain core concepts, namely, machining feature, machining operation, and RMT module involved in RMT design. They considered a given part family as the input and the set of RMT configurations that are capable of machining the part family as the output. They also established a four-step sequential procedure to facilitate the utilization of encoded knowledge from the knowledge base to aid in the selection of appropriate RMT modules [START_REF] Ming | Ontology-based module selection in the design of reconfigurable machine tools[END_REF].

As far as the system level is concerned, it should have a modular structure and be ''open'' such that upgrading and customization of the system are practical and integration of new software is possible [START_REF] Meng | Modeling of reconfigurable manufacturing systems based on colored timed object-oriented Petri nets[END_REF]. Modular-based systems have many benefits that will make it possible to implement both RMSs and MC [START_REF] Maganha | Understanding reconfigurability of manufacturing systems: An empirical analysis[END_REF]. In Andersen, Nielsen, and Brunoe' study, the first observation is related to the criticality of having detailed knowledge of the production system and its constituents, as this is a prerequisite for being able to design modular systems that can be reconfigured [START_REF] Andersen | Prerequisites and Barriers for the Development of Reconfigurable Manufacturing Systems for High Speed Ramp-up[END_REF].

Reconfigurability in RMS

Based on its definition, an RMS can process diversified parts belonging to a part family.

In the group technology concept, a part family refers to a set of related parts with similarity in shape and geometry or similarity in production operation processes [START_REF] Mcgraw-Hill | McGraw-Hill Dictionary of Scientific & Technical Terms[END_REF]. Achieving the processing of diverse parts with limited production equipment entails the reconfigurability of RMS. Reconfigurability is defined as the ability to add, remove and/or rearrange in a timely and cost-effective manner the components and functions of a system which can result in a desired set of alternate configurations; chosen here to be the addition/removal of new products and resources [START_REF] Farid | A Design Structure Matrix Based Method for Reconfigurability Measurement of Distributed Manufacturing Systems[END_REF]. System reconfigurability can be classified in terms of the levels where the reconfigurable actions are taken. Reconfigurability at lower levels is mainly achieved by changing hardware resources, and reconfigurability at the higher levels is mainly achieved by changing software resources. [START_REF] Bi | Reconfigurable manufacturing systems: The state of the art[END_REF]. The reconfigurability of RMS hardware can be further classified at the system level or at the machine level.

Adding or removing machines and changing the position of machines are reconfiguration actions at the system level. The production capability of RMS could be adjusted by adding/removing machines or overtime work [START_REF] Liu | Optimisation of line configuration and balancing for reconfigurable transfer lines considering demand uncertainty[END_REF]. Adding or removing machines to match the new throughput requirements and concurrently rebalancing the system for each configuration, accomplishes the system reconfiguration [START_REF] Wang | Scalability planning for reconfigurable manufacturing systems[END_REF]. The process of adding and removing machines may have a high cost and requires quite amount of time. This is why the reconfiguration time's minimization plays an important role [START_REF] Kurniadi | Development of IOT-based Reconfigurable Manufacturing System to solve Reconfiguration Planning Problem[END_REF].

In MC context, the notion of planning before production is redundant, hence requirement for layout reconfiguration is frequent and at such short intervals that present systems are unsuitable [START_REF] Ogunsakin | Towards engineering manufacturing systems for mass personalisation: a stigmergic approach[END_REF]. The frequent layout reconfiguration means that the physical features and the manufacturing paths of different batch workpieces are mutable [START_REF] Chen | A practical strategy to layout adjustment of mobile robot based reconfigurable material handling system[END_REF]. The frequent reconfiguration and redesign of layout must also maintain the system high performance (productivity, reactivity, maintainability...etc.) [START_REF] Haddou-Benderbal | Layout evolution effort for product family in Reconfigurable Manufacturing System design[END_REF]. Focusing on assembly system level, Benkamoun, Huyet, and Kouiss classified configuration design methods oriented system reconfigurability and differentiated the logical layout (task assignment, with or without resources selections) method for variety from the physical layout (arrangement of workstations) method for variety [START_REF] Benkamoun | Reconfigurable assembly system configuration design approaches for product change[END_REF]. Shang et al. proposed a key feature-based method for reconfigurable inspection system configuration design that combine different reconfigurable inspection machines to form diverse reconfigurable inspection system configurations and reconfigure the reconfigurable inspection system by changing the reconfigurable inspection machine layout [START_REF] Montalto | An approach to design reconfigurable manufacturing tools to manage product variability : the mass customisation of eyewear[END_REF].

''Activation of reconfigurability'' within machine boundaries allows usage of existing structures in the manufacturing systems in order to react to the change drivers to ensure its sustainability [START_REF] Azab | Mechanics of change: A framework to reconfigure manufacturing systems[END_REF]. The module-based machinery design aims to support the transition towards the reconfigurability from an engineering design perspective [START_REF] Gauss | Module-based machinery design: a method to support the design of modular machine families for reconfigurable manufacturing systems[END_REF]. Machine tools with integrated reconfigurability have reconfiguration functions that are integrated into the machine module or machine tool [START_REF] Moon | Reconfigurable Machine Tool Design[END_REF]. RMT, which is the foundation block for RMS is envisaged as a solution that caters to the current market forces [START_REF] Singh | Reconfigurable machine tools: a perspective[END_REF]. Features based design specification of parts are used to carry machining as well as production information to define machine tool configuration such as number of axis, range and spindle orientations, wherefore RMT is characterized by changes and uncertainty of a part or part family [START_REF] Kannan | A feature-based generic setup planning for configuration synthesis of reconfigurable machine tools[END_REF]. Based on the structural features and the functions of the components that construct the machine tools, Y. Wang, Zhang, and Han generated the conceptual modules of RMTs such as the geometry modules and the basic structure modules. They proposed a method for constructing autonomous functional modules of RMTs at the conceptual design stage [START_REF] Wang | A methodology of setting module groups for the design of reconfigurable machine tools[END_REF]. Montalto et al. described a design approach to transform a dedicated manufacturing tool (i.e., the thermoforming mold of spectacles fronts) into a reconfigurable one to effectively manage the product variability [START_REF] Montalto | An approach to design reconfigurable manufacturing tools to manage product variability : the mass customisation of eyewear[END_REF].

Their study was successfully applied in the eyewear industry with respect to the MC paradigm.

Control in an RMS

The RMS is controlled by computer-based technology [START_REF] Xing | Reconfigurable manufacturing system for Agile manufacturing[END_REF]. Control should be designed as a real-time system to be reconfigured quickly as the system is changed from one configuration to another frequently [START_REF] Gadalla | Recent advances in research on reconfigurable machine tools : a literature review[END_REF]. Da Silva et al. present a control architecture and its design method for RMSs, applying Petri Net to model the workflow to solve unpredictable demands and to implement fault-tolerance behavior [START_REF] Da Silva | Control architecture and design method of reconfigurable manufacturing systems[END_REF]. Schütz et al. presented an agentbased control system, fully implemented on programmable logic controllers (PLC) in the languages of the IEC 61131. These software agents are able to detect sensor failures and reconfigure the control software in real-time, what leads to a higher dependability of the production system [START_REF] Schütz | Highly reconfigurable production systems controlled by real-time agents[END_REF]. Deif and ElMaraghy proposed a general capacity scalability controller design to improve the dynamic performance of RMSs in response to sudden demand changes [START_REF] Deif | A control approach to explore the dynamics of capacity scalability in reconfigurable manufacturing systems[END_REF].

Recently, the advent of industry 4.0 technologies and advanced distributed manufacturing control systems has provided additional plug-and-produce capabilities to these systems, further reducing reconfiguration times and efforts [START_REF] Colledani | Integrated production and reconfiguration planning in modular plug-and-produce production systems[END_REF]. While producing different product variants, the system should never shut down completely, allowing to deal with product customization and various plug & produce devices on-the-fly [START_REF] Dorofeev | Agile Operational Behavior for the Control-Level Devices in Plug&Produce Production Environments[END_REF]. Huang et al. proposed a "Plug & Produce" architecture, where the main modules can be removed/reinstalled easily and quickly [START_REF] Huang | Reconfigurable machine tools design for multi-part families[END_REF]. Morgan et al. represented a future-state model "Plug and Produce" architecture for the next-generation Industry 4.0 smart reconfigurable machines by: reconfigurable distributed control and decentralized control machines/ systems, intelligent CPS architectures, virtualization modelling, and enabling modular scalable technical frameworks [START_REF] Morgan | Industry 4.0 smart reconfigurable manufacturing machines[END_REF]. RMS enables building a ''live" factory that can quickly and cost-effectively respond to the changing customer needs [START_REF] Gu | Manufacturing system architecture for cost-effective massindividualization[END_REF]. Most recently, reconfigurable and changeable systems were introduced to manage different volume and variety levels when and where needed [START_REF] Deif | Variety and volume dynamic management for value creation in changeable manufacturing systems[END_REF]. For the goal of intelligent manufacturing in the revolution of Industry 4.0, the coordination of reconfigurable machines and other devices in an RMS can be controlled by a cyber-physical system (CPS) that connects physical and software components in a computing network and monitors them via sensing. Cyber-physical RMSs that are able to efficiently produce customized products in lot sizes of one have the potential to significantly advance MC [START_REF] Andersen | Exploring Barriers Toward the Development of Changeable and Reconfigurable Manufacturing Systems for Mass-Customized Products: An Industrial Survey[END_REF].

Conclusion

RMS is THE next generation manufacturing system. It has six key characteristics: scalability, convertibility, diagnosability, modularity, integrability and customization. The modularity of RMT and of the software at the system level provides the reconfigurability of machines in RMS and the fast changeover of functions to produce different parts. Both are conducive to MC. The reconfigurability of the layout facilitates the adaptation to the everchanging proportions of the various part variants in orders containing multiple mass-customized products. With the development of computers and information technology, it is easier and more intelligent to control the RMS. Above all, RMS seems to be the most appropriate manufacturing system for manufacturing companies to accommodate MC with robust performance. It is thus necessary to study how to implement MC in an RMS.

I.3 -Way to implement MC in an RMS

As discussed in the previous sections, RMS is of the most adapted manufacturing systems for MC. Reconfiguration is obtained by defining and scheduling different paths for the products and is achieved by the capacity of machines to perform different tasks for different product references [START_REF] Stief | Contribution to the design of reconfigurable multi-product assembly systems by architecture solutions generation through a new locating-driven approach[END_REF]. Yet, MC in an RMS has economies of scale only if multiple customized products are to be produced.

As discussed in Section I.1, mass customized products form product diversity through the combination of various modules. Hence, each mass-customized product is made up of multiunit parts, belonging to some part variants. As illustrated in Figure I-1, a part variant is derived by choosing a set of various components from the architecture set and the values for the attributes of these components from the corresponding attribute sets. Variety Component (VC) is either the functional component or the physical component of a part variant. The attribute sets limit the values that can be chosen for an attribute of a variety component, such as the diameter of a hole feature, the depth of a pocket feature, etc. [START_REF] Xia | Reconfigurable machining process planning for part variety in new manufacturing paradigms: Definitions, models and framework[END_REF]. In this example, there are two parts belonging to Part variant 2 but from Product 1 and Product 2, respectively. The final operation sequences to process them can be different. In fact, not only parts from the same part variant of different products can be processed in different operation sequences, but even parts from the same part variant of a product can be processed in different operation sequences. These imply the process variety of processing a part. This is clearly a deterministic operational decision-making problem. Although the customized flexibility and reconfiguration of the RMS have not been considered, the process variety has already complicated the operational decision-making.

In an RMS, there are several machines. Typically, each machine has more than one configuration and thus can perform more than one operation. A configuration refers to an assemblage of RMTs that is able to accomplish a specific machining step. Every operation is a single step that requires only one machine-configuration pair to be performed. A configuration is capable of performing at least one operation. One operation can be performed at least by one configuration. As a result, generating a process plan for a part should not only choose an appropriate operation sequence, but also select a proper machine with a proper configuration to perform each operation. This is the operational decision-making that involves the process planning optimization.

As there are multiple mass-customized products and each product is made up of multiunit parts, the total number of operations waiting to be performed in this MC production task is huge. Since all the parts decomposed from all the mass-customized products can be processed in this RMS, each operation must be performed on one of several machines. Operations that are assigned on the same machine should be performed in a processing sequence. A job refers to a complete work for processing a part. Consequently, there are plenty of operations on machines.

The decision-making on the processing sequence involves the flexible job-shop scheduling problem (FJSP) optimization. The FJSP is a generalization of the classical job-shop scheduling that allows to process operations on one machine out of a set of alternative machines [START_REF] Nouri | Solving the flexible job shop problem by hybrid metaheuristics-based multiagent model[END_REF].

FJSP is challenging due to the expanding machine tools capabilities and increased products variety [START_REF] Roshanaei | Mathematical modelling and a meta-heuristic for flexible job shop scheduling[END_REF].

Operations in a job are probably not assigned to be performed on the same machine. This means that a part will be transported from one machine to another before it is finished. A semi-finished part in a job is called a Work in Process (WIP). WIPs will be transported from one position to another in condition that two of its consecutive operations are performed on separate machines. The transportation time and cost will be determined by the distance between machines. The facility layout will present the machine positions and the distance between them. Therefore part scheduling is also affected by selecting different machines layout in terms of distances between machines [START_REF] Ghanei | An Integrated Multi-Period Layout Planning and Scheduling Model for Sustainable Reconfigurable Manufacturing Systems[END_REF]. Finding an effective layout is important because the layout determines how materials, manufacturing operations and processes flow through the system [START_REF] Haddou Benderbal | Exhaustive Search Based Heuristic for Solving Machine Layout Problem in Reconfigurable Manufacturing System Design[END_REF]. The potential is to frequently alter layouts, therefore, in a sense, transforming the modern layout problem from a strategic problem in which only long-term material handling costs are considered to a tactical problem in which operational performance measures such as reduction of product flow times, WIP inventories, and maximizing throughput rate are considered in addition to material handling and machine relocation costs when changing from one layout configuration to the next [START_REF] Meng | Reconfigurable layout problem[END_REF]. This makes it possible to organize a specific RMS machine layout that is profitable for a particular production task.

Core research questions

Based on the above description, the core research problem to achieve MC in an RMS integrates process planning, FJSP, and facility layout optimization. These three problems are highly interdependent and gain from joint optimization. To solve this problem, we should answer the following four research questions in detail:

Research question 1: What is the optimal operation sequence for processing each part decomposed from every mass-customized product?

This question makes sense when there are optional operation sequences for processing a part. A group of choices will apparently affect the processing sequences on machines, which involve the machine setup and reconfiguration activities. In return, the setup and reconfiguration effort will affect the selection of machine and configuration to perform operations.

Research question 2: Which machine with which configuration is best to perform each operation in the optimal operation sequence?

The operation processing parameters are directly dependent on the performed machine and configuration. In addition, the selection of machines for performing the operations determines the 'workload of machines' (number of operations assigned to machines).

Operations assigned to machines are elements in the processing sequence on machines. As operations in a job carry the due date information of the corresponding mass-customized product, which must be considered at the time to decide the processing sequence.

Research question 3: What is the optimal processing sequence and at what time are these operations processed on the machines?

Machine reconfiguration occurs when two consecutive operations in a processing sequence are performed by different configurations. Machine setup occurs when there is any difference (different operations/different configuration/different part variant) for two consecutive operations in a processing sequence. Both activities are prerequisites for preparing parts processing, thus taking time and increasing cost without adding values to the products.

For that, a processing sequence will affect the overall MC cost and the completion time of masscustomized products. Without the exact beginning time of processing operations, there are still some vague points in the production plan for manufacturers to implement MC in an RMS.

Hence, it is important to answer this research question in full.

Research question 4: What is the optimal machine layout for producing given multiple multi-unit mass-customized products?

When selection of machine and configuration pairs for operations as well as the operation and processing sequences are decided, it is obvious that the machine layout will affect the WIP transportation. Machine layout should be as compact as possible. However, the layout reconfiguration that changes the initial machine positions often takes more time and costs than WIP transportation, creating a trade-off between layout reconfiguration and machine selection.

From the above statements, it is possible to observe that these research questions are not separate and there is no inner logic in depth to answer them sequentially for obtaining global optimal solutions to this research problem. Decisions related to research questions 1, 2 and 4 can be made simultaneously, followed by decisions related to research question 3. This research problem is very complex, and a literature review to know how other researchers have addressed these research problems is necessary.

I.4 -Planning, scheduling and layout optimization of RMS

The keywords for searching scholarly literature on planning, scheduling, and layout optimization of RMS are "Reconfigurable manufacturing system" AND "Planning", "Reconfigurable manufacturing system" AND "Scheduling" and "Reconfigurable manufacturing system" AND "Layout", respectively.

Most of the literature searched comes from four databases : ScienceDirect (https://www.sciencedirect.com/), IEEE Xplore (https://ieeexplore.ieee.org/Xplore/home.jsp),

SpringerLink

(https://link.springer.com/), and Taylor and Francis Online (https://www.tandfonline.com/), which are huge, reliable, and highly authoritative resources.

Journal articles, especially published since 2000 are preferred because of their recent research content, complete presentation, and high scientific quality. The following sub-sections will present the literature review focusing on planning, scheduling and layout optimization in an RMS, respectively.

Planning in an RMS

Research on planning can be subdivided into those on process planning and on production planning. Process planning is a knowledge intensive activity that transforms product design model into manufacturing processes and determines the optimal operation sequences for production [START_REF] Xia | Reconfigurable machining process planning for part variety in new manufacturing paradigms: Definitions, models and framework[END_REF]. The decision-making process in production planning allows not only the resources needed to carry out future manufacturing operations to be determined, but also all the production activities performed to optimize companies' objectives to be effectively coordinated [START_REF] Guzman | Models and algorithms for production planning, scheduling and sequencing problems: A holistic framework and a systematic review[END_REF]. Research on process planning focuses on generating a process plan for fabricating a single product/part to accomplish its design. Research on production planning in RMS focuses on producing many products or parts.

I.4.1.1 Production planning in RMS

While the external business environment is facing rapidly changing and unpredictable customer demands, the internal manufacturing environment is challenged by diverse custom designs and enormous variations in production planning [START_REF] Du | A real-option approach to flexibility planning in reconfigurable manufacturing systems[END_REF]. The main enabler of reliable planning results with RMS in production planning and control has been the integration of scalability in terms of the capacity and functionality of systems [START_REF] Hees | A Production Planning Method to Optimally Exploit the Potential of Reconfigurable Manufacturing Systems[END_REF]. Considering different configurations and the scalability of functionality and capacity to increase the flexibility and adjustability of planning approaches for RMS, Hees et al. presented a system for production planning that continuously integrates the characteristics of RMS [START_REF] Hees | A production planning system to continuously integrate the characteristics of reconfigurable manufacturing systems[END_REF]. Tang and Qiu designed two attributed capacity automata for analytical and mathematical modeling of production planning of an RMS [START_REF] Tang | Integrated design approach for virtual production line-based reconfigurable manufacturing systems[END_REF]. Bruccoleri et al. proposed an agent based approach for the production planning activities in reconfigurable enterprises, composed by reconfigurable production systems that allow quick adjustment of production capacity and functionality consenting to manufacture different products of the same part family [START_REF] Bruccoleri | Production planning in reconfigurable enterprises and reconfigurable production systems[END_REF].

To ensure sustainable quality and good performance of production planning, reconfigurations and the ability of change in configurations of manufacturing systems as well as resources have to be considered in production planning and control [START_REF] Hees | Approach for production planning in reconfigurable manufacturing systems[END_REF]. Some research dealt with production planning in RMS merely by configuration selection at the system level.

Jain and Palekar solved the aggregate production-planning problem in the context of a continuous production process. They considered the product differentiation at various stages of manufacturing. They also considered for each stage of production, several available machines to produce different sets of products. These are reconfigurable and incurring a setup cost [START_REF] Jain | Aggregate production planning for a continuous reconfigurable manufacturing process[END_REF].

Diaz et al. presented a simulation-based multi-objective optimization approach for system reconfiguration of multi-part flow lines subjected to scalable capacities. It addresses the assignment of the tasks to workstations and buffer allocation for simultaneously maximizing throughput and minimizing total buffer capacity to cope with fluctuating production volumes [START_REF] Diaz | Optimizing Reconfigurable Manufacturing Systems for Fluctuating Production Volumes: A Simulation-based Multi-Objective Approach[END_REF]. Few works dealt with production planning in RMS by configuration selection at the machine level. Bortolini et al. studied production planning in cellular RMS for multiple parts flowing among RMTs in batches [START_REF] Bortolini | An optimisation model for the dynamic management of cellular reconfigurable manufacturing systems under auxiliary module availability constraints[END_REF].

Recent developments in wireless technologies have created opportunities for developing reconfigurable wireless manufacturing systems with real-time traceability, visibility and interoperability in shop-floor planning, execution and control [START_REF] Zhang | Agent-based workflow management for RFID-enabled real-time reconfigurable manufacturing[END_REF]. In a dynamic and stochastic context, the decision maker must find the best compromise in terms of configurations and operating costs while ensuring the on time delivery of the product and considering the random events [START_REF] Dammak | Dynamic production planning in unreliable Reconfigurable Manufacturing Systems[END_REF]. Several researchers consider the effects of stochastic orders arrival on the important variables of RMS production planning. Abbasi and Houshmand focused on the utilization state of an RMS and introduced a mathematical model using estimated values according to stochastic orders to manage and evaluate effectiveness of RMS [START_REF] Abbasi | Production planning and performance optimization of reconfigurable manufacturing systems using genetic algorithm[END_REF]. Hasan et al.

focused on determining optimal configuration of an RMS required by multiple part family orders whose arrival rate follows a Poisson distribution and identified the optimum sequence of part families on the basis of maximum benefit earned for a given system configuration [START_REF] Hasan | Optimum configuration selection in Reconfigurable Manufacturing System involving multiple part families[END_REF].

A handful of studies have been conducted to develop production planning with sustainability in RMS. Choi and Xirouchakis developed a production planning model with the multi-objective function for minimizing the energy consumption and maximizing the throughput of an RMS [START_REF] Choi | A holistic production planning approach in a reconfigurable manufacturing system with energy consumption and environmental effects[END_REF]. Liu et al. investigated a multi-module RMS for multi-product manufacturing and divided the production plan of the system into the system design phase and the manufacturing phase, where the installation and the energy consumption costs correspond to the two phases, respectively [START_REF] Liu | Energy-oriented bi-objective optimisation for a multi-module reconfigurable manufacturing system[END_REF]. Yazdani et al. considered social sustainability and environmental sustainability factors including the harmful liquids and gases emission during the production and its effects on the operator's body and the social manners of operators and people in the neighborhood for the multi-product process and production planning in an RMS [START_REF] Yazdani | Process and production planning for sustainable reconfigurable manufacturing systems (SRMSs): multi-objective exact and heuristic-based approaches[END_REF].

I.4.1.2 Process planning in RMS

The early research applied multi-agent negotiation to generate distributed process planning in RMS or any dynamic reconfigurable and distributed manufacturing environment [START_REF] Wang | DPP: An agent-based approach for distributed process planning[END_REF]. Later, the optimization of configuration selection including machine/workstation selection and arrangement is applied for process planning in flow-line RMS. Youssef and ElMaraghy proposed a novel "RMS configuration selection approach" that considers more than one aspect of the system configuration (arrangement of machines, equipment selection and assignment of operations), utilizes important system-level evaluation criteria (cost and availability) and involves stochastic analysis to take into consideration the smoothness of the anticipated reconfiguration process [START_REF] Youssef | Optimal configuration selection for Reconfigurable Manufacturing Systems[END_REF]. Dou et al. presented a systematic approach for the problem of determining the single-product flow-line configuration's parameters including number of workstations, number of paralleling machines and machine type as well as assigned operations for each workstation [START_REF] Dou | Precedence graph-oriented approach to optimise single-product flow-line configurations of reconfigurable manufacturing system[END_REF]. They also presented a genetic algorithm (GA) based approach for optimizing multi-part flow-line configurations of RMS for a part family [START_REF] Dou | Optimisation for multi-part flow-line configuration of reconfigurable manufacturing system using GA[END_REF].

RMTs enable flexible reconfigurability at the machine level, thereby the optimization of RMTs configuration selection is investigated a lot to generate a process plan for production flow lines. For the optimal machine assignment for a single part flow line allowing paralleling of similar machines, Goyal et al. proposed a novel approach based on the module interactions and machine capability to measure the machine reconfigurability and operational capability of an RMT, which will lead to a reduction in reconfiguration efforts while planning multiple period configurations in the RMS [START_REF] Goyal | Optimal configuration selection for reconfigurable manufacturing system using NSGA II and TOPSIS[END_REF]. Ashraf and Hasan solved an multi-objective optimization problem comprising conflicting objective functions like cost, reconfigurability, operation capability along with reliability to create better process plans by finding a comprehensive suitability of a feasible RMT configuration at each production state of a reconfigurable serial product flow line [START_REF] Ashraf | Configuration selection for a reconfigurable manufacturing flow line involving part production with operation constraints[END_REF]. Moghaddam et al. dealt with the increased demand of a product by replacing the existing auxiliary modules in-between existing RMTs and transforming one machine configuration to another for addressing the problem of capacity scalability in RMS where the demand of a single product varies throughout its production life cycle in a single-product flow line [START_REF] Moghaddam | Configuration design in scalable reconfigurable manufacturing systems (RMS); a case of single-product flow line (SPFL)[END_REF]. To further refine their research, two new approaches are put forward for a scalable reconfigurable RMS that produces different products of a part family, aiming at minimizing RMS design and reconfiguration costs by taking different production capabilities of RMTs into consideration [START_REF] Moghaddam | Configuration design of scalable reconfigurable manufacturing systems for part family[END_REF].

Recent studies consider both machines and RMTs selection to optimize process planning in job-shop RMS. In order to generate an optimal process plan to carry out a certain product with specific characteristics, Abderrahmane, Dahane, and Benyoucef proposed an adapted NSGA-II based approach for identifying the tool approach direction (TAD) and the type of tool required to carry out the operation, identifying among the available machines (the set of machines capable of performing that particular operation) and assigning the machines and the appropriate configuration to the particular operation of the subpart [START_REF] Bensmaine | A non-dominated sorting genetic algorithm based approach for optimal machines selection in reconfigurable manufacturing environment[END_REF]. For the purpose of developing an approach to ensure the best process plan according to the customized flexibility required to produce all parts of a given product, Haddou-Benderbal et al. addressed the problem of machines selections for RMS design under unavailability constraints and handled the assignment of operations to the selected machines by identifying the available RMTs in a machine and the TAD offered by this machine [START_REF] Haddou Benderbal | Flexibility-based multiobjective approach for machines selection in reconfigurable manufacturing system (RMS) design under unavailability constraints[END_REF]. Based on the co-evolution of process planning and machine configurations, Asghar et al. achieved an optimized reconfigurable framework to realize optimal production of a part family by selecting a set of machines to be involved for all the necessary operations and obtaining machine configurations from the combinations of TADs [START_REF] Asghar | Optimum machine capabilities for reconfigurable manufacturing systems[END_REF]. Haddou-Benderbal et al. studied the RMS design by selecting from a set of candidate machines the most suitable ones as well as the operation requirements of the RMT type and TAD to generate the best process plan that maximizes the system modularity, minimizes the system completion time and minimizes the system cost to complete a single unit of each product from the available ones in the product family [START_REF] Haddou Benderbal | Modularity assessment in reconfigurable manufacturing system (RMS) design: an Archived Multi-Objective Simulated Annealing-based approach[END_REF].

Touzout and Benyoucef developed and compared three hybrid metaheuristics to determine a chronological sequence of operations as well as the triplets defined by the selected machine, configuration and its RMT to perform each operation of the sequence for the multi-objective single-product multi-unit process plan generation problem in a reconfigurable manufacturing environment [START_REF] Touzout | Multi-objective multi-unit process plan generation in a reconfigurable manufacturing environment: a comparative study of three hybrid metaheuristics[END_REF]. In response to unpredictable demands in RMS, Kazemisaboor et al.

provided a two-step simulation-based optimization framework for part-family single-unit and multi-unit process planning, in which configurations, tools and TADs are adjusted for assignment of operations to machines [START_REF] Kazemisaboor | A simulation-based optimisation framework for process plan generation in reconfigurable manufacturing systems (RMSs) in an uncertain environment[END_REF]. Khan et al. involved the selection of machines, configurations, modular features, tools, and TADs, along with the greater number of possible production routes in the process planning of an RMS [START_REF] Khan | Modularity-based quality assessment of a disruptive reconfigurable manufacturing system-A hybrid metaheuristic approach[END_REF].

Few papers solve the process planning for producing a mass-customized product in an RMS via integrating the module product design strategy with the decision-making of better performing operations. For the sake of increasing the RMS's responsiveness and ability to deal with large product varieties, Sabioni et al. proposed a GA-based approach to concurrently optimize the module instances selection together with the operation identification to meet product functions required by the customer and the operation sequence with their machineconfiguration assignment [START_REF] Sabioni | An integrated approach to optimize the configuration of mass-customized products and reconfigurable manufacturing systems[END_REF]. Later, they combined a modified brute-force algorithm with GA and compared it with a CPLEX-based approach for the validation of their studies [START_REF] Sabioni | Concurrent optimisation of modular product and Reconfigurable Manufacturing System configuration: a customeroriented offer for mass customisation[END_REF].

Over the past three years, energy consumption, greenhouse gases (GHG) emission and hazardous waste have been increasingly considered for process planning optimization in sustainable reconfigurable manufacturing environment. In addition to the total production cost and the completion time, Touzout and Benyoucef used the amount of GHG emitted during the manufacturing process as a sustainability criterion for the process plan generation of a singleproduct single-unit [START_REF] Touzout | Multi-objective sustainable process plan generation in a reconfigurable manufacturing environment: exact and adapted evolutionary approaches[END_REF]. Based on process plan generation for a single unit of a product to be manufactured, Massimi et al. outlined sustainability in a reconfigurable environment from an energy consumption point of view for selecting the best set of modular machines from a list of candidates as well as the needed set of basic and auxiliary modules that compose these machines [START_REF] Massimi | A heuristic-based non-linear mixed integer approach for optimizing modularity and integrability in a sustainable reconfigurable manufacturing environment[END_REF]. Khezri et al. addressed an environmental oriented multi-objective process plan generation problem for a sustainable RMS and minimized the total production time, the total production cost and a sustainability-metric value considering both liquid hazardous waste and GHG emissions [START_REF] Khezri | Towards a sustainable reconfigurable manufacturing system (SRMS): multi-objective based approaches for process plan generation problem[END_REF]. Khettabi et al. proposed adapted versions of NSGA-II, NSGA-III, weighted GAs and random weighted GAs to solve this environmental-oriented single-unit process plan generation problem [START_REF] Khettabi | Sustainable reconfigurable manufacturing system design using adapted multi-objective evolutionary-based approaches[END_REF]. The evaluation of their new dynamic NSGA-II and new NSGA-III are presented in [START_REF] Khettabi | Sustainable multi-objective process planning in reconfigurable manufacturing environment: adapted new dynamic NSGA-II vs New NSGA-III[END_REF].

Apart from the most popular solution approach, GA, simulated annealing (SA)

combined with heuristic knowledge and metaknowledge is further developed to solve the process planning problem in RMSs. Musharavati and Hamouda devised three SA based algorithms that exploit auxiliary knowledge in different ways and employed them to handle a process planning problem for reconfigurable manufacturing [START_REF] Musharavati | Simulated annealing with auxiliary knowledge for process planning optimization in reconfigurable manufacturing[END_REF]. They also devised four configurations of SA algorithms and implemented them to solve an instance of a process planning problem in RMSs [START_REF] Musharavati | Enhanced simulated-annealing-based algorithms and their applications to process planning in reconfigurable manufacturing systems[END_REF].

In summary, there is a wealth of research on planning in an RMS. Although most of them spoke briefly of the advantages of RMS for MC, they placed more emphasis on the diversity of products planned. More and more researchers built two or three objective functions in their mathematical models for process/production planning optimization in RMS, in order to attract customers by compressing costs, and to improve the manufacturer's market

Scheduling in an RMS

Scheduling involves the sequencing of jobs to be processed by a given set of machines and the assignment of actual starting times to each individual job [START_REF] Gelders | Production planning: a review[END_REF]. In traditional manufacturing systems, the configuration of production system (e.g. flexible flow shop) is usually fixed within the planning horizon, and scheduling is the main way to adjust delivery times or through-puts of multiple products [START_REF] Dou | Mixed integer programming models for concurrent configuration design and scheduling in a reconfigurable manufacturing system[END_REF]. The scheduling problem can be divided into flow-shop scheduling and job-shop scheduling. In the flow-shop scheduling, all jobs pass through all machines in the same order [START_REF] Šeda | Mathematical Models of Flow Shop and Job Shop Scheduling Problems[END_REF]. In a job-shop, jobs can be processed on some machines (or perhaps all machines) in any order [START_REF] Parveen | Review on Job-Shop and Flow-Shop Scheduling Using Multi Criteria Decision Making[END_REF].

Research on scheduling in RMS is not as rich as research on process planning in RMS.

For the scheduling problem, the quantity of products or parts is predetermined, compared to the production planning problem, where the quantity of throughput is often seen as an objective to be maximized. The scheduling problem should determine the exact timing of the sequenced processes in order to provide an exact processing schedule, and it is highly dependent on the production and process planning decisions.

Most studies on scheduling for RMS, adopted job-shop scheduling principles. Yu et al.

suggested a practical priority rule based approach in which decisions for input sequencing and scheduling are done using a combination of dispatching rules in an RMS with a limited number of fixtures [START_REF] Yu | Input sequencing and scheduling for a reconfigurable manufacturing system with a limited number of fixtures[END_REF]. Wan and Yan studied the problems of integrated assembly job-shop scheduling and self-reconfiguration in knowledgeable manufacturing with the objective of minimizing the weighted sum of completion cost of products, the earliness penalty of operations and the training cost of workers [START_REF] Wan | Integrated scheduling and self-reconfiguration for assembly job shop in knowledgeable manufacturing[END_REF]. Reddy et al. presented a novel social network analysis based method to evaluate the reconfiguration effect i.e., identification of key machines and their influence on the system performance in the context of FJSP problem [START_REF] Reddy | Investigation of reconfiguration effect on makespan with social network method for flexible job shop scheduling problem[END_REF]. Prasad and Jayswal calculated the reconfiguration effort of the manufacturing system as criterion for scheduling and used the integrated approach of Shannon entropy and reference as an ideal method for the scheduling of the products [START_REF] Prasad | Reconfigurability consideration and scheduling of products in a manufacturing industry[END_REF]. Mahmoodjanloo et al. named the necessary time for an RMT to reconfigure as the configuration-dependent setup time and formulated two mixed integer linear programming models with the position-and sequence-based decision variables to minimize the maximum completion time (i.e., makespan) for FJSP [START_REF] Mahmoodjanloo | Flexible job shop scheduling problem with reconfigurable machine tools : An improved differential evolution algorithm[END_REF]. Two years later, The same research team studied a dynamic job-shop scheduling problem to minimize total weighted lateness in a static state in a distributed manufacturing system, in which each facility contains some reconfigurable machine tools to perform assigned operations [START_REF] Mahmoodjanloo | Distributed job-shop rescheduling problem considering reconfigurability of machines: a self-adaptive hybrid equilibrium optimiser[END_REF]. to minimize total tardiness for the integrated optimization of configuration generation and scheduling [START_REF] Dou | Bi-objective optimization of integrating configuration generation and scheduling for reconfigurable flow lines using NSGA-II[END_REF]. Then, the same research team adopted heuristics to efficiently find solutions with tradeoff between total cost and tardiness for the RMS with multi-part flow line configuration that concurrently produces multiple parts within the same family [START_REF] Dou | A multi-objective particle swarm optimisation for integrated configuration design and scheduling in reconfigurable manufacturing system[END_REF]. In order to solve the flexible flowshop scheduling problem, Shiyun et al. proposed an improved multiobjective particle swarm optimization algorithm based on Brownian motion and the Gaussian cumulative distribution function for the RMSs [START_REF] Shiyun | Multi-objective reconfigurable production line scheduling forsmart home appliances[END_REF]. Yang and Xu studied intelligent scheduling and reconfiguration with dynamic job arrival for a reconfigurable flow line using deep reinforcement learning [START_REF] Yang | Intelligent scheduling and reconfiguration via deep reinforcement learning in smart manufacturing[END_REF].

A couple of studies have addressed both planning and scheduling issues at the same time. A planning and scheduling architecture first proposed for the control of an RMS is comprised of four functions [START_REF] Liles | A computer based production scheduling architecture suitable for driving a reconfigurable manufacturing system[END_REF]:

 demand determination and decomposition,  aggregate process resource planning;

 resource requirement scheduling;

 production scheduling and control.

Chaube et al. solved the assignment of each operation to a machine and scheduling of operations in order to minimize the completion time and cost of the dynamic process plan for RMS [START_REF] Chaube | An adapted NSGA-2 algorithm based dynamic process plan generation for a reconfigurable manufacturing system[END_REF]. Bensmaine et al. proposed a heuristic considering the multi-configuration nature of machines to the integrated process planning and scheduling problem in RMS and compared it with a classical sequential process planning and scheduling strategy using a discrete-event simulation framework [START_REF] Bensmaine | A new heuristic for integrated process planning and scheduling in reconfigurable manufacturing systems[END_REF]. The results showed that the makespan of solutions of the integrated process planning and scheduling is shorter than that of the solutions obtained by the sequential approach. This leads to a better performing manufacturing delay for the integrated approach. This is due to the exploitation of RMT reconfigurability, by the integration between process planning and scheduling, which relaxes the rigidity of process plans resources

Layout optimization of RMS

The placement and arrangement of machines in the plant area, often considered as "facility layout problem", has significant effects upon manufacturing efficiency, productivity, economic benefits and even human factors of the systems [START_REF] Zheng | A simulation analysis of facility layout problems in reconfigurable manufacturing systems[END_REF]. Based on the type of material handling system in use, layout arrangements are classified as row layout, loop layout, and cluster/open-field layout [START_REF] Keller | Single row layout models[END_REF].

The reconfigurable layout problem is defined as the ability of the layout to rearrange frequently, with minimal effort, to adjust its configuration to new circumstances, considering system operational performance and providing the exact capacity and functionality needed, when required [START_REF] Maganha | A Theoretical Background for the Reconfigurable Layout Problem[END_REF].

'Relayout' or reconfigurability began to receive a higher attention in facility layout literature at the beginning of this century [START_REF] Kulturel-Konak | Approaches to uncertainties in facility layout problems: Perspectives at the beginning of the 21st Century[END_REF]. However, in the RMS literature, there are only a few researches on the layout design problem [START_REF] Yelles-Chaouche | Reconfigurable manufacturing systems from an optimisation perspective: a focused review of literature[END_REF]. Despite some of the studies mentioned above integrating RMS layout optimization with planning or scheduling problem, RMS layout optimization has not received enough attention.

Of these few findings, most researchers devoted to dedicated layout to offer allocable sites for machines distribution in RMS. Guan fixturing system with passive degrees of freedom and formulated a constrained nonlinear optimization problem to determine the optimal layout of reconfigurable fixtures for a given set of workpieces [START_REF] Gašpar | Optimal layout and reconfiguration of a fixturing system constructed from passive Stewart platforms[END_REF].

The open-field layout offers more flexibility for high-mix low-volume production systems compared to other layouts as it maximizes the number of equidistant machines [START_REF] Alduaij | Adopting a circular open-field layout in designing flexible manufacturing systems[END_REF].

So far, only very few researchers work on open-field layout problems without definite locations

for machines in RMS. For example, Wei et al. exploited a chaotic GA with improved tent mapping to solve the RMS configuration problem of the dynamic continuous facility layout [START_REF] Wei | Optimizing facility layout planning for reconfigurable manufacturing system based on chaos genetic algorithm[END_REF].

I.5 -Analysis of literature review

The shift from mass production to mass customization is continuously gaining interest in both academic and industrial worlds. Mass customization has become a reality and is offered by many industries. But it has several enablers and requirements such as modularity (in product and process), flexible production systems and smart production management to reduce overall costs while satisfying customer needs including order delay [START_REF] Maalouf | Production management for mass customization and smart cellular manufacturing system: NSGAII and SMPSO for factory-level planning[END_REF]. In a world where the desire for customization and personalization is increasingly present among consumers, it is crucial to minimize the harmful effects of human activity in the customizable product sector to make it sustainable [START_REF] Vandecasteele | MASS CUSTOMIZATION DESIGN , BETWEEN CUSTOMERS AND SUSTAINABILITY[END_REF]. Sustainable MC takes advantage of the manufacturing efficiency of the smart manufacturing transformation processes to achieve the desired levels of sustainability [START_REF] Martínez-Olvera | Towards the Development of a Digital Twin for a Sustainable Mass Customization 4.0 Environment: A Literature Review of Relevant Concepts[END_REF].

RMS is such a smart manufacturing system with high efficiency to transform processes flexibly.

It is most adapted for MC in the future, since RMT can be designed to give quick responses to the varying demands of the markets [START_REF] Habib | Modular Product Architecture for Sustainable Flexible Manufacturing in Industry 4.0: The Case of 3D Printer and Electric Toothbrush[END_REF]. The implementation of RMS relies heavily on production planning [START_REF] Shivdas | Proposed composite similarity metric method for part family formation in reconfigurable manufacturing system[END_REF]. Optimizing production in RMS for MC opens new challenges, because existing theories are not clear on how much reconfiguration effort should be invested for a particular production task. In addition, the production optimization in RMS is complex because of its combinatorial approach. This complexity increases especially when the number of variables increases [START_REF] Malhotra | Evaluation of barriers affecting reconfigurable manufacturing systems with graph theory and matrix approach[END_REF].

According to the schematic of RMS research perspectives shown in Figure I-3, research on planning, scheduling and layout in RMS is the main field of RMSs' applications. In this field, the literature analysis shows that the cost is the most widely analyzed KPI and that the use of meta-heuristic methodologies addressing such issues is widely spread [START_REF] Bortolini | Reconfigurable manufacturing systems: Literature review and research trend[END_REF]. optimization into the linear combination of weights method, the global criterion method, the 𝜖constraint method and the Pareto method [START_REF] Chiandussi | Comparison of multiobjective optimization methodologies for engineering applications[END_REF]. Marler and Arora thought one of the predominant classifications of multi-objective approaches is that of scalarization methods and vector optimization methods [START_REF] Marler | Survey of multi-objective optimization methods for engineering[END_REF]. The most used methods to solve multi-objective optimization are the following three:

1. The scalarization method: makes the multi-objective function create a single solution and the weight is determined before the optimization process [START_REF] Gunantara | A review of multi-objective optimization: Methods and its applications[END_REF].

2. The 𝜖-constraints method: optimizes one of the objective functions using the other objective functions as constraints, incorporating them in the constraint part [START_REF] Mavrotas | Effective implementation of the ε-constraint method in Multi-Objective Mathematical Programming problems[END_REF].

3. The Pareto method: keeps the elements of the solution vectors separate (independent) during optimization and the concept of dominance is there to differentiate the dominated and non-dominated solutions [START_REF] Gunantara | A review of multi-objective optimization: Methods and its applications[END_REF].

The foundation of the scalarization method is that optimal solutions of multi-objective optimization can be characterized as solutions of certain single objective optimization problems [START_REF] Jahn | Scalarization in Multi Objective Optimization[END_REF]. However, in multi-objective optimization, the objective functions probably conflict, which means there are trade-offs between different objective values of a solution. Therefore, and as expected, there is no work in this literature review using this method to solve multiobjective optimization in this research field.

The 𝜖-constraints method was used by Chankong and Haimes [START_REF] Changkong | Multiobjective Decision Making: Theory and Methodology[END_REF]. In this method, the decision maker chooses one objective to be minimized; the remaining objectives are constrained to be less than or equal to given target values [START_REF] Caramia | Multi-objective Optimization[END_REF]. There are less than five articles in this literature review using this method to solve the multi-objective optimization in this research field. The main drawback of the 𝜖-constraint method is that it is very sensitive to the number of objectives [START_REF] Copado-Méndez | Enhancing the ε-constraint method through the use of objective reduction and random sequences: Application to environmental problems[END_REF]. Ehrgott and Ruzika summarize this method and point out some weaknesses, namely the lack of easy-to-check conditions for properly efficient solutions and the inflexibility of the constraints [START_REF] Ehrgott | Improved ε-constraint method for multiobjective programming[END_REF].

The Pareto front consists of a set of acceptable trade-off optimal solutions. It, allows the decision maker to make an informed decision by seeing a wide range of options since it contains the solutions that are optimum from an "overall" standpoint; unlike single-objective optimization that may ignore this trade-off viewpoint [START_REF] Ngatchou | Pareto multi objective optimization[END_REF]. Most of the studies in the literature review adopted Pareto-optimal concept to solve their multi-objective mathematical models.

Pareto-optimal solutions are non-dominated to each other. This research employed the nondominated concept to find Pareto-optimal solutions. An exact Pareto-optimal cannot be dominated by any other feasible solution to the multi-objective optimization problem. An approximate Pareto-optimal solution cannot be dominated by any other feasible encountered in the current run of the approximate solution approach. The dominance relationship between two solutions is determined by comparing their values of each objective function. The different considered objective functions are equally important. For MC, time constraint is very important.

Customer satisfaction is the main focus of MC. That satisfaction includes being delivered on time. Therefore, this becomes an important objective function. It is as important as the overall manufacturing cost, since if the customer is not satisfied, the MC orders are lost. Therefore, both scalarization and 𝜖-constraints methods are less adapted to our problem.

The considered optimization problems in this research are mainly NP-hard problems which refer to those problems for which polynomial-time algorithms have not yet been found

to solve. This means that time complexity explodes as the size of the problem increases.

Unfortunately, layout problems are known to be complex and are generally NP-Hard [START_REF] Drira | Facility layout problems: A survey[END_REF].

FJSP is also an NP-hard combinatorial optimization problem since the proved NP-hard TSP problem is a special case of the job-shop scheduling with a single job (the cities are the machines and the salesman is the job). This research engages FJSP and the layout problem, therefore it is also an NP-hard problem.

As exact solution approaches to solve NP-hard problems are computationally expensive and extremely slow, the majority of the research in the literature review adopted approximate solution approaches to solve the numerical examples or practical cases to validate their mathematical models. Most of them used improved metaheuristics, like tabu search, SA, GA, particle swarm optimization and Artificial neural network. To give some summaries from a statistical point of view, Table I To conclude, there is still work to be done on planning, scheduling and layout integrated optimization of RMSs for achieving MC. This research follows the generic research process to formulate the integrated optimization problem into cost and time oriented multi-objective mathematical models and used the exact together with the approximate solution approaches for validation.

Although only one journal article in this literature review uses reinforcement learning to solve the RMS scheduling problem, the perspective of much literature suggests that machine learning methods are useful for this research area. With the availability of data in each phase of product life-cycle, and advances in algorithms and software tools, machine learning is emerging as an appropriate and promising tool for more agile, lean, and energy-efficient manufacturing systems [START_REF] Sharp | A survey of the advancing use and development of machine learning in smart manufacturing[END_REF]. A major advantage of machine learning algorithms is to discover formerly unknown (implicit) knowledge and to identify implicit relationships in data-sets [START_REF] Wuest | Machine learning in manufacturing: Advantages, challenges, and applications[END_REF].

However, heuristics need parameters tuning, since it impacts the results obtained of the optimization. Parameter tuning should be properly done, but it is very time consuming.

Observing that parameters tuning of any heuristic should be repeated when problem changes, an approximate solution approach based on reinforcement learning (a kind of machine learning method) is developed to solve the research problem.

CHAPTER II -Planning, scheduling and layout integrated cost and time oriented bi-objective optimization for MC in an RMS

This chapter introduces a novel deterministic model to integrating planning, scheduling and layout optimization in an RMS, knowing the number of mass-customized products, the components of these products and thereby the required parts to produce. Since it is a NP-hard problem that cannot be solved in an acceptable computation time when the scale of problem increases, two approximate solution approaches, NSGA-II and AMOSA are adopted and improved by the constraint handling techniques, an innovative job-shop scheduling constraint handling technique in particular. To evaluate their performances, especially the accuracy of them, the approximate Pareto-optimal solutions to a small numerical example obtained by these two solution approaches are first compared with the exact Pareto-optimal solutions obtained by the exhaustive search. Then the approximate Pareto-optimal solutions to a large numerical example are measured by four metrics to evaluate the effect of parameters of NSGA-II and AMOSA on their performance and the effect of constraint handling techniques on their performance. Results show NSGA-II with constraint handling techniques outperforms to solve the planning, scheduling and layout integrated optimization for MC in an RMS. This contributes to the research gap broached in the literature review by proving that it is practicable to get a truly optimal production plan for MC when fully availing RMS.

This chapter is organized as follows. First the problem formulation and the made assumptions are presented, then the mathematical model is detailed. Following, the solution approaches are described, and numerical experiments are used to validate both the mathematical model and the solution approaches.

II.1 -Problem formulation

MC is driven by customers' demands, implicating the information on the composition of parts in the corresponding customized products and their deliveries. The more demand a company has, the lower the unit cost to the MC. Demands for every mass-customized product are not released to the manufacturer one by one, and there is no regulation for producing them on a first-in, first-out basis. Products are manufactured once to meet quality standards; thus, remanufacturing is not considered in this study. In manufacturing, every mass-customized product has a due date to finish its production. This ensures on-time delivery to the customer.

All products must be finished eventually. To deliver every mass-customized product on time, the production of all the parts that make up every mass-customized product should be finished before the due date. In case some product is delayed, the manufacturer will receive a penalty to bear the customer's loss. The tardiness penalty depends on customers and the tardiness duration.

RMS is flexible enough to manage the product diversity of MC. Every part decomposed from a mass-customized product can be processed in the given manufacturing environment of RMS, which indicates that all the mass-customized products in the proposed MC task can be finished in the given manufacturing environment of RMS, in which all production facilities are in good working condition throughout the whole manufacturing duration and all equipment preparing for parts production are adequate. Resources for MC production are ample in the RMS workshop. Adding new machines is also not considered. These indicate that current RMS has sufficient production capacity to accomplish the proposed MC task. There is no decision on whether to add machines and their configurations. Process planning is a multi-decision-making activity that determines the operation selection and operation sequencing which involves a great deal of manufacturing data [START_REF] Shabaka | A model for generating optimal process plans in RMS[END_REF].

Due to the short life of a given layout and the availability of product data for a given period, it is possible to consider optimizing operational performance measures [START_REF] Maganha | The layout design in reconfigurable manufacturing systems: a literature review[END_REF]. As shown in

II.2 -Assumptions

The following assumptions help simplify the mathematical model:

1) Parameters about a WIP's transportation time and cost per distance unit, as well as parameter about a WIP's holding cost per time unit are only dependent on the type of part variant that this WIP belongs to.

2) Parameters about an operation's processing time, processing cost, setup time, and setup cost are dependent on the type of part variant the corresponding WIP belongs to, the kind of this operation, the executing machine, and the machine configuration.

3) Parameters about reconfiguration time and cost on a machine depend on the former configuration and the latter configuration.

4) At the beginning, all machines are idle without configurations and setups; hence there is no machine reconfiguration for the machines' first operations. Notations for parameters of mass-customized products are as follows. Notations for independent decision variables are as follows.

Notations for auxiliary decision variables are as follows.

Objective functions

There are two objectives in this mathematical model equally important. The tardiness of a mass-customized product 𝑇 in the first objective function is defined by the following equation:

𝑥

𝑇 𝑚𝑎𝑥 max ∀ ∈ ,∀ ∈ ,…, , 𝑐 , , ,| | 𝐷 , 0 ∀i ∈ I II-2
The completion time of a mass-customized product is determined by the last finished parts. If a product is completed no later than the due date, the tardiness value is 0. If not, the tardiness value is the portion of a product's completion time beyond the due date.

II.3.2.2 The second objective function on total cost

The second objective is to minimize the sum of layout reconfiguration cost (𝐿𝑅𝐶), the total machine reconfiguration cost (𝑀𝑅𝐶), the total setup cost (𝑇𝑆𝐶), the total processing cost (𝑇𝑃𝐶), the total WIP transportation cost (𝑊𝐹𝐶), and the total WIP holding cost (𝑊𝐻𝐶) for a given MC task.

𝑀𝑖𝑛 𝐿𝑅𝐶 𝑀𝑅𝐶 𝑇𝑆𝐶 𝑇𝑃𝐶 𝑊𝐹𝐶 𝑊𝐻𝐶 II-3

The five costs 𝐿𝑅𝐶, 𝑀𝑅𝐶, 𝑇𝑆𝐶, 𝑇𝑃𝐶, 𝑊𝐹𝐶 , and 𝑊𝐻𝐶 in the second objective function are defined by the equations II-4, II-5, II-6, II-7, II-8, and II-9, respectively.

Layout reconfiguration cost

𝐿𝑅𝐶 ∑ 𝐿𝐶 |𝑥 𝑋 | |𝑦 𝑌 | | | II-4
The layout reconfiguration cost is calculated by equation II-4. The layout reconfiguration occurs only once, before production. When the optimum layout is obtained, the RMS machines move from their initial positions to the optimum positions. The distance to move an RMS machine is equal to the sum of distances between these two positions on the X and Y coordinates. The reconfigured machine layout cannot be changed during production.

Total machine reconfiguration cost

II-6

The total setup cost is calculated by equation II-6. The setup occurs when two consecutive processes are performed for different kinds of parts or with different configurations, or they are different operations. There is always a setup for the first process on each machine.

If two consecutive operations performed on a machine are different, or they come from different part variants, or they are performed by different configurations, there will be a setup for the latter of these two operations.

Total processing cost

𝑇𝑃𝐶 ∑ ∑ ∑ ∑ 𝑃𝐶 , , , , , , , , , , , , | | ,
| | | | II-7
The total processing cost is calculated by equation II-7. Intuitively, the total processing cost is determined by the decision variables related to process planning. As all parts and processes have to be completed after all and their corresponding part variants and operations are already known, the selection of machine and its configuration for each process has a direct impact on the total processing cost. However, the decision regarding machine and configuration selection is influenced by the processing sequence and WIP management. It is therefore indirectly determined by the decision variables related to scheduling and machine layout.

Total WIP transportation cost

𝑊𝐹𝐶 ∑ ∑ ∑ ∑ 𝐹𝐶 ∆𝑥 , , , , , , , ∆𝑦 , , , , , , , | |
, | | | | II-8
The total WIP transportation cost is calculated by equation II-8. A WIP will be transported from one position to another if two consecutive processes are performed on different machines. The distance to transport a WIP is equal to the sum of distances between two machines on the X and Y coordinates. ,,,,,, ∆𝑦 , , , , , , , ∀𝛼 , , , 𝛼 , , , II-9

Total WIP holding cost

𝑊𝐻𝐶 ∑ ∑ ∑ ∑ 𝐻𝐶 𝛽 , , , 𝑐 , , , 𝐹𝑇 | | , | | | | ∆𝑥 ,
The total WIP holding cost is calculated by equation II-9. The WIPs may not go into processing immediately after transportation. If a WIP arrives at an occupied machine, it should be held until the machine is available for processing it. Excluding the time for transporting a WIP, all the remaining time between two consecutive operations in a job sequence but performed on different machines is counted as time spent holding that WIP.

Constraints

The auxiliary decision variables are defined in the following constraints. Besides, relationships between independent decision variables, such as non-overlapping on machine layout, operation precedence for process planning and machine occupancy for scheduling are expressed in this part. Domains of independent decision variables are also presented below.

Details are explained in the paragraphs after each constraint.

The decision variables are subject to the following constraints:

II.3.

II.4 -Solution approaches

As explained in the previous chapter, this problem is NP-hard because it involves FJSP and layout problems, which are NP-hard requiring exponential computation time to obtain the optimal solutions. Therefore, the exact solution approach can only be used to solve small-scale examples, whose solutions are not too difficult to be obtained by exhaustive search. For the large-scale examples that cannot be solved by the exact solution approach in an acceptable computation time, it is worthwhile to develop approximate solution approaches. This study adopted NSGA-II and AMOSA as the approximate solution approaches to solve the large-scale numerical example in the numerical experiments, because the exact solution approach cannot obtain exact Pareto-optimal solutions in an acceptable time. As described in details later, GA is the most popular approximate solution approach to solve this kind of research problem.

Considering that the mathematical model in this study has two objective functions, NSGA-II, which has been proven to have a better performance in many studies, is adopted for this multiobjective optimization. However, a disadvantage of GA is that it has a weak local search capability. Owing to that, AMOSA is also adopted as the approximate solution approach, seeing that it is for multi-objective optimization and it is highly capable of local search. For the reason that infeasible solutions are generated during iterations in these two approximate solution approaches, problem-specific constraint handling techniques are applied to handle infeasible solutions in these two approximate solution approaches.

This research uses Python programming language, which is object-oriented and functional programming. No matter in the programming of exact or approximate solution approaches, a solution is encoded as an object with independent decision variables as its properties and constraint handling techniques and operators (in exact and approximate solution approaches) into functions as its methods. The independent decision variables 𝑥 and 𝑦 for reconfigured layout are real values in two two-dimensional lists of length 𝑚, respectively. The independent decision variables 𝜌 , , , , 𝛼 , , , , and 𝜑 , , , for process planning are integer values in separate three four-dimensional nested lists, whereas the independent variables 𝛽 , , ,

for scheduling are real values in a four-dimensional nested list. As the value of every independent decision variable is easy to access by their indices set in the mathematical model, there is no decoding procedure.

Exact solution approach

This study used exhaustive search as the exact solution approach to obtain all solutions of the small-scale numerical example. Comparing their two objective values through the Pareto efficiency, the Pareto-optimal solutions are obtained.

In fact, it is impossible to obtain all feasible layouts in any case, because the independent decision variables of machine positions are continuous, which means that the number of positions for each machine is not countable and there is not a finite number of a machine's position. The green rectangle with white border in Figure II-8 illustrates the area available for placing machine 𝑚. There is no way to enumerate every 𝑥 , 𝑦 values within this range.

Figure II-6 Available area to place machine 𝑚.

To simplify the validation, the length and width of the rectangular workshop, the initial layout, the security distances of machines, and independent decision variables of 𝑥 and 𝑦 for reconfigured layout are set to be integers in the small-scale numerical example for mathematical model validation. In this context, every machine has finite positions, as shown in When the processing sequences are determined, the beginning time of each operation, 𝛽 , , , , is determined by taking the value of the earliest time it is feasible to be performed. One reason is related to the first objective to minimize the tardiness penalty of mass-customized products. It is advisable to begin the operation as early as possible in order to let more masscustomized products finish their production before the due date. Another reason is related to the holding cost in the second objective. The longer a WIP is not being processed or is not in the transported from one machine to another, the higher holding cost for the corresponding part. The exact solution approach is necessary to validate the mathematical model by checking the integrity of constraints and to evaluate the accuracy of the approximate solution approaches by comparing the approximate optimal solutions with the exact optimal solutions.

As shown in

The exhaustive search introduced above finds all feasible solutions by permutations and combinations of the integer decision variables as well as finds all exact Pareto-optimal solutions by compared two objective values of all feasible solutions. The Non-dominated procedure is useful in reducing the computation time as the dominated solutions are compared only once.

Approximate solution approach

The exact solution approach introduced above is no applicable for today's rising demand for customized products. Manufacturers must find an efficient solution approach with the existing computing power to generate an optimal production plan for large scale problems.

Based on the literature review, approximate solution approaches are advantageous to obtain approximate optimal solutions. In this study, NSGA-II and AMOSA are adopted as the approximate solution approaches. They are described in the following, along with the used constraint handling techniques.

II.4.2.1 NSGA-II

The population approach of EAs is an effective way to find multiple Pareto-optimal solutions simultaneously in a single simulation run [START_REF] Deb | Multi-objective Optimization[END_REF]. GA is a metaheuristic belonging to EA. A set of solutions are created as the initial population. Each solution acts like a chromosome and new solutions are generated by natural selection and the exchange of some genes (independent decision values) on the selected chromosomes (solutions) in the population, even mutating few genes on a couple of chromosomes with a small probability. After the evolution of many generations, GA could find the approximate optimal solutions by the population-topopulation search, which is more effective than the point-to-point search.

The generic seletion is computationally demanding since in the classical GA, at each generation, the individuals (refer to solutions) are randomly selected from the current population to reproduce children for the next generation through crossover and mutation [START_REF] Song | An improved knowledge-informed NSGA-II for multiobjective land allocation (MOLA)[END_REF].

To improve GA, Deb et al. suggested a non-dominated sorting-based multi-objective GA, called NSGA-II [START_REF] Deb | A fast and elitist multiobjective genetic algorithm:NSGA-II[END_REF], which is a multi-objective EA that is able to find more spread solutions over the Pareto front with low computational requirements, so as to be widely used in the literature to tackle plenty of practical problems.

The success of NSGA-II is due to the fact that the concept of Pareto-optimality was incorporated into the selection mechanism, and the population was ranked on the basis of the non-domination using elitism and crowding distance comparison operator [START_REF] Kumar | The elitist non-dominated sorting genetic algorithm with inheritance (i-NSGA-II) and its jumping gene adaptations for multi-objective optimization[END_REF]. Elitism can speed up the performance of the GA significantly in achieving better convergence and helps to prevent the loss of good solutions once they have been found by retaining the best individuals in a generation unchanged in the next generation [START_REF] Deb | A Fast Elitist Non-dominated Sorting Genetic Algorithm for Multi-objective Optimization: NSGA-II[END_REF]. The non-dominated sorting aims to divide a solution set into a number of disjoint subsets or ranks, by means of comparing their values of the same objective, and after non-dominated sorting, solutions in the same rank are viewed equally important, and solutions in a smaller rank are better than those in a larger rank [START_REF] Tian | Effectiveness and efficiency of nondominated sorting for evolutionary multi-and many-objective optimization[END_REF]. To evaluate the quality of all individuals in the same rank layer, the conventional NSGA-II uses the crowding distance as the metric of non-dominated individuals in the same rank layer [START_REF] Zhang | Multi-objective optimization for materials design with improved NSGA-II[END_REF]. After non-dominated sorting, each individual is assigned a crowding distance by sorting the population on each objective in turn and setting the crowding distance of an individual to the sum of distances along the objective axis to the closest individual, and individuals with high crowding distances are assumed to reside in sparsely populated areas and preferred over individuals with low crowding distances [START_REF] Jensen | Reducing the Run-Time Complexity of Multiobjective EAs: The NSGA-II and Other Algorithms[END_REF]. In NSGA-II the crowding distance operator will ensure diversity along the non-dominated front [START_REF] Dhanalakshmi | Application of modified NSGA-II algorithm to Combined Economic and Emission Dispatch problem[END_REF]. Above all, It has been proven that NSGA-II, in most of times, has much better spread of solutions and better convergence near the true Pareto-optimal than mostly Pareto-optimal methods [START_REF] Ouyang | The Improved NSGA-II Approach[END_REF].

This study adopted NSGA-II as the first approximate solution approach to solve the formulated problem and obtain the approximate Pareto-optimal solutions to the constructed mathematical model. The main steps to implement he adapted NSGAII in this study are shown in Figure II-14. Seeing that unconstrained archives themselves are necessary to prevent oscillations and retreat of the frontal set [START_REF] Fieldsend | Using Unconstrained Elite Archives for Multiobjective Optimization[END_REF], this study added an unconstrained archive to NSGA-II. From the experimental results for the small numerical example in the next section, the size of archive does not affect the diversity of the obtained approximate Pareto-optimal solutions, but unconstrained archive could save more solutions with the same objective values.

The archive is initialized with the approximate Pareto-optimal solutions from the first generation. Then in the following iterations, the archive is updated by comparing the solutions already present in the archive with those in the offspring population.

The main issue with GA and thus NSGAII is the treatment of unfeasible solutions. There are several ways to handle unfeasible solutions: adding penalty, disregarding the solution, or repairing it. Simply disregarding a solution is not recommended, since an unfeasible solution might lead with the different evolutions to an optimum one. Therefore, both penalty and repair approaches are preferred. The main challenge for repairing an unfeasible solution is that decision variables are interdependent and repairing the value for one decision variable to satisfy one constraint might probably lead to unsatisfying other constrains. Therefore, a repair technique with prioritized decision variables correction is proposed.

The constraint handling techniques are conducted after selection, crossover and mutation operators. Selection operator does not generate infeasible solutions, while crossover and mutation operator do. The infeasible solution handling follows the steps that first deals with the infeasible independent decision variables about layout (𝑥 and 𝑦 ), then repairs infeasible independent decision variables about process planning (𝜌 , , , , 𝛼 , , , and 𝜑 , , , ), and last repairs the infeasible independent decision variables about scheduling (𝛽 , , , ). This is because crossover operator. For the integer independent decision variables 𝜌 , , , , 𝛼 , , , , and 𝜑 , , , , the integer division (denoted by the symbol '//' in Figure II-15) is adopted to obtain integer values. Thus, these three independent decision variables are maintained as integer in the offspring. Although GAs are proven to be more efficient, they sometimes do not find the best solutions. Local search methods like SA can achieve optimum solution if enough time is given and this is the greatest advantage undoubtedly [START_REF] Zhang | Review of job shop scheduling research and its new perspectives under Industry 4.0[END_REF]. SA is motivated by an analogy to the statistical mechanics of annealing in solids and its techniques use an analogous set of "controlled cooling" operations for nonphysical optimization problems, in effect transforming a poor, unordered solution into a highly optimized, desirable solution [START_REF] Rutenbar | Simulated Annealing Algorithms : An overview[END_REF]. In this algorithm, a random initial solution is sent to a pre-defined neighborhood function to receive a new solution, which might replace the old solution by comparing their objective values [START_REF] Zaretalab | A knowledge-based archive multi-objective simulated annealing algorithm to optimize series-parallel system with choice of redundancy strategies[END_REF]. SA presents an optimization technique that can [START_REF] Ingber | Simulated annealing: Practice versus theory[END_REF]: a) process cost functions possessing quite arbitrary degrees of nonlinearities, discontinuities, and stochasticity; b) process quite arbitrary boundary conditions and constraints imposed on these cost functions; c) be implemented quite easily with the degree of coding quite minimal relative to other nonlinear optimization algorithms; d) statistically guarantee finding an optimal solution.

However, SA was originally designed for single-objective optimization and owing to its point-to-point search nature, very few attempts have been made to apply SA to solve multiobjective optimization problems [START_REF] Singh | A simulated annealing algorithm for constrained Multi-Objective Optimization[END_REF]. AMOSA is a SA-based multi-objective optimization algorithm that incorporates the concept of an archive where the non-dominated solutions seen so far are stored [START_REF] Bandyopadhyay | A simulated annealing-based multiobjective optimization algorithm: AMOSA[END_REF]. The domination amount makes searching gradually toward the region where global optimal solutions exist, thereby ensuring the likely jumping out of local search on the basis of the Metropolis criterion [START_REF] Li | An improved parallelized multi-objective optimization method for complex geographical spatial sampling: AMOSA-II[END_REF]. This study adopted AMOSA as the second approximate solution approach to solve the formulated problem and obtain the approximate At first, for an infeasible solution, the independent decision layout variables 𝑥 and 𝑦 are checked. If they do not satisfy the non-overlapping constraint, the values of this solution's two objectives will equal to a very large value. Accordingly, this infeasible solution is probably dominated by others and is unlikely to be selected in the parent population to generate the offspring population. The very large value, which is assigned to two objective values when the corresponding solution is infeasible, is set as a hyperparameter in approximate solution approaches. If the approximate Pareto-optimal solutions are feasible, this hyperparameter becomes 1. Based on this, feasible and infeasible solutions are able to be distinguished by observing the value of this hyperparameter. Thanks to that, infeasible approximate Paretooptimal solutions are excluded from the final archive, which saves the approximate Paretooptimal solutions seen so far. Nevertheless, infeasible solutions can still engage in the selection, crossover and mutation steps.

The constraint handling techniques make each iteration of the approximate solutions approaches efficacious for the sake of implementable production plans. The proposed method in the above pseudocode to repair FJSP independent decision variables is novel. In the following numerical experiments, the efficacy of constraint handling techniques is evaluated.

II.5 -Numerical experiment for validation

There are two numerical examples in this numerical experiment, a small and a large one. To confirm these findings, another four metrics defined by Nemati-Lafmejani et al. [START_REF] Nemati-Lafmejani | Multi-mode resource constrained project scheduling and contractor selection: Mathematical formulation and metaheuristic algorithms[END_REF] are adopted to measure the performance of NSGA-II and AMOSA through the objective values and numbers of the approximate Pareto-optimal solutions obtained by these two approximate solution approaches. These metrics are:

 Quality Metric (𝑄𝑀),  Mean Ideal Distance (𝑀𝐼𝐷),  Diversification Metric (𝐷𝑀),
 and Number of Pareto-optimal Solutions (𝑁𝑃𝑆).

The 𝑄𝑀 is determined by dividing the cardinal of the set of overall non-dominated solutions to the cardinal of the original set of Pareto-optimal solutions, and the higher 𝑄𝑀, the better the performance of the approximate solution approach [START_REF] Nemati-Lafmejani | Multi-mode resource constrained project scheduling and contractor selection: Mathematical formulation and metaheuristic algorithms[END_REF]. In this study, solutions obtained from the same parameters are used to calculate this metric by comparing all approximate Pareto-optimal solutions obtained from different runs.

𝑀𝐼𝐷 and 𝐷𝑀 are determined by the following equations [START_REF] Nemati-Lafmejani | Multi-mode resource constrained project scheduling and contractor selection: Mathematical formulation and metaheuristic algorithms[END_REF]: is the minimum value of them. 𝑓2 is the maximum value of the second objective among those solutions, while 𝑓2 is the minimum value of them. 𝑓 , 𝑓 are the first and second objective values of each approximate Pareto-optimal solution obtained, respectively. This metric measures the relative distance of approximate Pareto-optimal solutions, of which the lower value indicates the better performance of the corresponding solution approach [START_REF] Nemati-Lafmejani | Multi-mode resource constrained project scheduling and contractor selection: Mathematical formulation and metaheuristic algorithms[END_REF].

𝑀𝐼𝐷 ∑
𝐷𝑀 𝑓1 𝑓1 𝑓2 𝑓2 II-33
The metric 𝐷𝑀 defined by above equation shows the diversity of solutions, of which the higher value indicates the better performance of the approximate solution approaches [START_REF] Nemati-Lafmejani | Multi-mode resource constrained project scheduling and contractor selection: Mathematical formulation and metaheuristic algorithms[END_REF].

The metric 𝑁𝑃𝑆 is the number of the approximate Pareto-optimal solutions obtained in each run. Higher 𝑁𝑃𝑆 values are preferable [START_REF] Nemati-Lafmejani | Multi-mode resource constrained project scheduling and contractor selection: Mathematical formulation and metaheuristic algorithms[END_REF].

By using the Minitab software, DOE is carried out to investigate the effects of population size, mutation probability, and generation limit on NSGA-II performance, and the effects of iteration number and cooling rate on AMOSA performance, which is measured by the above four metrics along with the computation time (𝑡 ). Other parameters including maximum temperature (100), minimal temperature (1), the maximum size of the Archive on termination [START_REF] Leitold | Flood risk reduction and climate change adaptation of manufacturing firms: Global knowledge gaps and lessons from Ho Chi Minh City[END_REF], and the maximum size to the Archive before clustering [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF] are fixed.

Details of the DOE for this small numerical example are given in Table II-3. An intuitive finding is that although the hard size of the archive is designated as the number of the exact Pareto-optimal solutions, the approximate Pareto-optimal solutions obtained from AMOSA are less than five, whereas NSGA-II could obtain almost total Pareto-optimal solutions. This proves that the unconstrained archive added is better than the archive with the hard size. Also, the unconstrained archive does not need to consider how the size of the archive affects the obtained approximate Pareto-optimal solutions and how much the archive size should be set, especially not knowing the number of the exact Pareto-optimal solutions. Thereupon, adding an unconstrained archive in NSGA-II is a good adaptation to develop the approximate solution approach to the research problem of this thesis. [START_REF] Smets | Shouldn't customers control customized product development?[END_REF][START_REF] Yassine | Investigating the role of IT in customized product design[END_REF], [START_REF] Smets | Shouldn't customers control customized product development?[END_REF][START_REF] Yassine | Investigating the role of IT in customized product design[END_REF], [START_REF] Smets | Shouldn't customers control customized product development?[END_REF][START_REF] Yassine | Investigating the role of IT in customized product design[END_REF], [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Lasi | Industry 4.0[END_REF][START_REF] Elmaraghy | Product variety management[END_REF], [START_REF] Lasi | Industry 4.0[END_REF][START_REF] Elmaraghy | Product variety management[END_REF], (6, 126.5) 17.00 0.56 0.83 34.06 9 [START_REF] Smets | Shouldn't customers control customized product development?[END_REF][START_REF] Yassine | Investigating the role of IT in customized product design[END_REF], [START_REF] Smets | Shouldn't customers control customized product development?[END_REF][START_REF] Yassine | Investigating the role of IT in customized product design[END_REF], [START_REF] Smets | Shouldn't customers control customized product development?[END_REF][START_REF] Yassine | Investigating the role of IT in customized product design[END_REF], [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Jamwal | Deep learning for manufacturing sustainability: Models, applications in Industry 4.0 and implications[END_REF][START_REF] Helander | Research on E-product development (ePD) for mass customization[END_REF], (15, 96.5), (12, 109.2), (8, 110) 16.48 0.73 0.97 23. 41 11 (30, 84), [START_REF] Smets | Shouldn't customers control customized product development?[END_REF][START_REF] Yassine | Investigating the role of IT in customized product design[END_REF], [START_REF] Smets | Shouldn't customers control customized product development?[END_REF][START_REF] Yassine | Investigating the role of IT in customized product design[END_REF], [START_REF] Smets | Shouldn't customers control customized product development?[END_REF][START_REF] Yassine | Investigating the role of IT in customized product design[END_REF], [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Lasi | Industry 4.0[END_REF][START_REF] Yoo | The effects of e-mass customization on consumer perceived value, satisfaction, and loyalty toward luxury brands[END_REF], [START_REF] Lasi | Industry 4.0[END_REF][START_REF] Yoo | The effects of e-mass customization on consumer perceived value, satisfaction, and loyalty toward luxury brands[END_REF] [START_REF] Smets | Shouldn't customers control customized product development?[END_REF][START_REF] Yassine | Investigating the role of IT in customized product design[END_REF], [START_REF] Smets | Shouldn't customers control customized product development?[END_REF][START_REF] Yassine | Investigating the role of IT in customized product design[END_REF], [START_REF] Smets | Shouldn't customers control customized product development?[END_REF][START_REF] Yassine | Investigating the role of IT in customized product design[END_REF], [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Lasi | Industry 4.0[END_REF][START_REF] Elmaraghy | Product variety management[END_REF], (8, 91) 44.13 0.82 0.72 44. 10 11 (30, 84), [START_REF] Smets | Shouldn't customers control customized product development?[END_REF][START_REF] Yassine | Investigating the role of IT in customized product design[END_REF], [START_REF] Smets | Shouldn't customers control customized product development?[END_REF][START_REF] Yassine | Investigating the role of IT in customized product design[END_REF], [START_REF] Smets | Shouldn't customers control customized product development?[END_REF][START_REF] Yassine | Investigating the role of IT in customized product design[END_REF], [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Lasi | Industry 4.0[END_REF][START_REF] Yoo | The effects of e-mass customization on consumer perceived value, satisfaction, and loyalty toward luxury brands[END_REF], [START_REF] Lasi | Industry 4.0[END_REF][START_REF] Yoo | The effects of e-mass customization on consumer perceived value, satisfaction, and loyalty toward luxury brands[END_REF], (6, 121) 44.50 0.67 0.98 23. 41 12 (30, 84), [START_REF] Smets | Shouldn't customers control customized product development?[END_REF][START_REF] Yassine | Investigating the role of IT in customized product design[END_REF], [START_REF] Smets | Shouldn't customers control customized product development?[END_REF][START_REF] Yassine | Investigating the role of IT in customized product design[END_REF], [START_REF] Smets | Shouldn't customers control customized product development?[END_REF][START_REF] Yassine | Investigating the role of IT in customized product design[END_REF], [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Lasi | Industry 4.0[END_REF][START_REF] Yoo | The effects of e-mass customization on consumer perceived value, satisfaction, and loyalty toward luxury brands[END_REF], [START_REF] Lasi | Industry 4.0[END_REF][START_REF] Yoo | The effects of e-mass customization on consumer perceived value, satisfaction, and loyalty toward luxury brands[END_REF], [START_REF] Lasi | Industry 4.0[END_REF][START_REF] Yoo | The effects of e-mass customization on consumer perceived value, satisfaction, and loyalty toward luxury brands[END_REF] [START_REF] Smets | Shouldn't customers control customized product development?[END_REF][START_REF] Yassine | Investigating the role of IT in customized product design[END_REF], [START_REF] Smets | Shouldn't customers control customized product development?[END_REF][START_REF] Yassine | Investigating the role of IT in customized product design[END_REF], [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Lasi | Industry 4.0[END_REF][START_REF] Elmaraghy | Product variety management[END_REF], (8, 91) 18.47 0.57 0.85 27.20 14 [START_REF] Smets | Shouldn't customers control customized product development?[END_REF][START_REF] Yassine | Investigating the role of IT in customized product design[END_REF], [START_REF] Smets | Shouldn't customers control customized product development?[END_REF][START_REF] Yassine | Investigating the role of IT in customized product design[END_REF], [START_REF] Smets | Shouldn't customers control customized product development?[END_REF][START_REF] Yassine | Investigating the role of IT in customized product design[END_REF], [START_REF] Smets | Shouldn't customers control customized product development?[END_REF][START_REF] Yassine | Investigating the role of IT in customized product design[END_REF], [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Ben Hassen | Impacts of the Russia-Ukraine War on Global Food Security : Towards More Sustainable and Resilient Food Systems ?[END_REF][START_REF] Kwon | How online self-customization creates identification: Antecedents and consequences of consumer-customized product identification and the role of product involvement[END_REF], [START_REF] Ben Hassen | Impacts of the Russia-Ukraine War on Global Food Security : Towards More Sustainable and Resilient Food Systems ?[END_REF][START_REF] Kwon | How online self-customization creates identification: Antecedents and consequences of consumer-customized product identification and the role of product involvement[END_REF], [START_REF] Ben Hassen | Impacts of the Russia-Ukraine War on Global Food Security : Towards More Sustainable and Resilient Food Systems ?[END_REF][START_REF] Kwon | How online self-customization creates identification: Antecedents and consequences of consumer-customized product identification and the role of product involvement[END_REF], [START_REF] Lasi | Industry 4.0[END_REF][START_REF] Reeves | Additive Manufacturing for Mass Customization[END_REF], [START_REF] Lasi | Industry 4.0[END_REF][START_REF] Reeves | Additive Manufacturing for Mass Customization[END_REF], (8, 100) 16.00 0.80 0.92 24.17 10 (30, 84), [START_REF] Smets | Shouldn't customers control customized product development?[END_REF][START_REF] Yassine | Investigating the role of IT in customized product design[END_REF], [START_REF] Smets | Shouldn't customers control customized product development?[END_REF][START_REF] Yassine | Investigating the role of IT in customized product design[END_REF], [START_REF] Smets | Shouldn't customers control customized product development?[END_REF][START_REF] Yassine | Investigating the role of IT in customized product design[END_REF], [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Lasi | Industry 4.0[END_REF][START_REF] Sun | Research on the Technology of Mass Customization of Clothing[END_REF] [START_REF] Smets | Shouldn't customers control customized product development?[END_REF][START_REF] Yassine | Investigating the role of IT in customized product design[END_REF], [START_REF] Smets | Shouldn't customers control customized product development?[END_REF][START_REF] Yassine | Investigating the role of IT in customized product design[END_REF], [START_REF] Smets | Shouldn't customers control customized product development?[END_REF][START_REF] Yassine | Investigating the role of IT in customized product design[END_REF], [START_REF] Smets | Shouldn't customers control customized product development?[END_REF][START_REF] Yassine | Investigating the role of IT in customized product design[END_REF], [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Lasi | Industry 4.0[END_REF][START_REF] Elmaraghy | Product variety management[END_REF], [START_REF] Lasi | Industry 4.0[END_REF][START_REF] Elmaraghy | Product variety management[END_REF], (4, 106) 33.84 1 1.01 23.09 10 (30, 84), [START_REF] Smets | Shouldn't customers control customized product development?[END_REF][START_REF] Yassine | Investigating the role of IT in customized product design[END_REF], [START_REF] Smets | Shouldn't customers control customized product development?[END_REF][START_REF] Yassine | Investigating the role of IT in customized product design[END_REF], [START_REF] Smets | Shouldn't customers control customized product development?[END_REF][START_REF] Yassine | Investigating the role of IT in customized product design[END_REF], [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Lasi | Industry 4.0[END_REF][START_REF] Elmaraghy | Product variety management[END_REF], (8, 91) 37.75 0.8 0.97 23. 41 10 (30, 84), [START_REF] Smets | Shouldn't customers control customized product development?[END_REF][START_REF] Yassine | Investigating the role of IT in customized product design[END_REF], [START_REF] Smets | Shouldn't customers control customized product development?[END_REF][START_REF] Yassine | Investigating the role of IT in customized product design[END_REF], [START_REF] Smets | Shouldn't customers control customized product development?[END_REF][START_REF] Yassine | Investigating the role of IT in customized product design[END_REF], [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Lasi | Industry 4.0[END_REF][START_REF] Yoo | The effects of e-mass customization on consumer perceived value, satisfaction, and loyalty toward luxury brands[END_REF] The mean values of metrics obtained from three runs with the same parameter setting in every group are used to get the main effects plots for NSGA-II and AMOSA.

Figure II-23 shows the main effects plots for the computation time of NSGA-II and AMOSA. Increasing population solution and generation limit will drastically increase the computation time of NSGA-II, while the mutation probability impacts little on its computation time. This is beneficial as modifying the mutation probability will not destroy the computing efficiency of NSGA-II. Observing that a small decrease of the cooling rate will significantly reduce the computation time of AMOSA, while the iteration number has to be halved to achieve the same effect, it is a good idea of reducing the cooling rate to improve AMOSA performance. Increasing both population size and generation limit improves 𝑄𝑀 of NSGA-II. Nevertheless, increasing the generation limit is better than increasing the population size as the latter improves less on 𝑄𝑀 but leads to more increase on computation time. A small cooling rate has a positive impact on 𝑄𝑀 of AMOSA, while the iteration number almost has no impact on it. This further demonstrates that reducing the cooling time is effective to improve the AMOSA performance. It is obvious that large population size will obtain more approximate Pareto-optimal solutions.

Thus, there is a greater possibility to have more scattered approximate Pareto-optimal solutions, so that the objective values have wide ranges, and 𝑀𝐼𝐷 values become high. However, the mutation probability helps search jump out of the local optimum and have more different solutions but is surprisingly negatively correlated with the 𝑀𝐼𝐷 value. Both generation limit and iteration number refer to the repetitions of operators in NSGA-II and AMOSA, whereas the former more significantly affects the relative distance of approximate Pareto-optimal solutions obtained by the corresponding solution approaches. Although reducing the cooling rate makes the 𝑀𝐼𝐷 value higher, it is not preferable from the perspective of the literature on 𝑀𝐼𝐷 provenance. However, it is not always disadvantageous to have a dense solution distribution, small cooling rate is not inferior in general. Large mutation probability, small population size and generation limit lead to high diversity of approximate Pareto-optimal solutions obtained by NSGA-II, which are desirable in its performance. This is because a larger mutation probability, smaller population size and generation limit let the obtained solutions more convergent, thus losing their diversity. The cooling rate has a similar principle to the diversity of approximate Pareto-optimal solutions obtained by AMOSA, while iteration number hardly affects this metric value, which perhaps because even the small iteration number have already allowed solutions reach convergence at each stage. High diversity gives decision makers more optimal production plans to balance the tradeoff between two objectives. However, since the decision maker can ultimately choose only one of these options to implement MC production, it is more important to set proper parameters for suitable diversity. The experiments above have already investigated the accuracy and effectiveness of the improved NSGA-II and AMOSA introduced before. To verify the applicability of these two solution approaches, a large numerical example is used to conduct more numerical experiments. To make the composition of products clear, numbers of parts belonging to different part variants in every product are also given in a table in this figure. This numerical example is too complicated to get the exact Pareto-optimal solutions in an acceptable computation time. Thus, it is only solved by the approximate solution approaches. values of any operations' beginning time that are not subject to scheduling constraints will not be repaired. And the two objective values of these infeasible solutions will be equal to a very large value, so that they are probably dominated by other solutions in iterations and cannot be retained in the final archive which keeps the approximate Pareto-optimal solutions seen so far for NSGA-II or AMOSA in a run. Each approach is run ten times.

As shown in

II.6 -Conclusion

This study presented the concurrent optimization of process planning, scheduling and machine layout for realizing MC in an RMS, which has already achieved the research objective to generate a production plan that can produce multiple mass-customized products in RMS at low cost and finish them on time as much as possible, thus satisfying customers. To the best of our knowledge, it is the first time to formulate a mathematical model simultaneously considering layout reconfiguration in production planning by answering the core research questions of what is the optimal operation sequence for processing each part decomposed from every mass-customized product, which machine with which configuration is best to perform each operation in the optimal operation sequence, what is the optimal processing sequence and what time are these operations processed on machines and what is the optimal machine layout for producing given multiple multi-unit mass-customized products (details described in Chapter I section 3), which is specific to make a production plan for MC in RMS. To validate this novel mathematical model, exhaustive search is first used to solve a small numerical example. As constraints are correct and sufficient, approximate solution approach, NSGA-II and AMOSA are adopted to solve the formulated problem as the exact solution approach, such as exhaustive search is computational expensive and inefficient.

Some constraint handling techniques, such as penalty function method and repair method of infeasible independent decision variables with values outside their domain are equipped with these two heuristics. In particular, a repair method to modify the infeasible beginning time of operations is innovative and it is universal for all FJSP. Complying with the main scheduling arrangement in a solution, the proposed constraint handling method of Besides, the main effects between the general parameters in these two heuristics and their performance with the constraint handling techniques are studied using DOE, which is CHAPTER III -Planning, scheduling and layout integrated optimization for MC in an RMS considering environmental sustainability

The climate change and energy crisis brought about by the global rough development model are gradually challenging the survival of mankind. "Pollution first, then governance" is not a long-term solution, and may even cause a vicious circle. Sustainable development is not only to achieve the purpose of economic development, but also to protect the environment on which human beings depend for survival, so that future generations can live and work in peace and happiness. Environmental protection is an important aspect of sustainable development.

Manufacturing is an important driver of economic development [START_REF] Khezri | A Sustainable Reconfigurable Manufacturing System Designing with Focus on Environmental Hazardous Wastes[END_REF]. Since the industrial revolution, global economic development has relied on the use of the natural environment and resources. As technology continues to develop, factories have more opportunities than ever to reduce their impact on humans and nature. MC is an on-demand production, which can effectively avoid the waste caused by oversupply. Wisely using the reconfigurability of RMS can precisely control MC production, thus balancing the conflict between environmental protection and manufacturing. In recent years, the public health impact of human exposures to hazardous chemical wastes in the environment has been the object of increased governmental and public scrutiny [START_REF] Orloff | An international perspective on hazardous waste practices[END_REF]. Greenhouse gas (GHG) emissions, mainly carbon dioxide emissions, are on the global agenda with regard to climate change [START_REF] Branker | Greenhouse gases emitted in manufacturing a product-A new economic model[END_REF]. GHG emissions and hazardous waste discharge become the main indicators when sustainable manufacturing is concerned in operation research.

This chapter introduces a study to produce multiple multi-unit mass-customized products in a sustainable RMS. An additional objective to control the amount of hazardous waste and GHG emissions is first described. As there are three objectives in the mathematical model, general Pareto efficiency leads to more Pareto-optimal solutions, which may confuse decision makers. A new Pareto efficiency is proposed, which is problem-specific to find reasonable number of Pareto-optimal solutions to the joint optimization of planning, scheduling and layout. Since the improved NSGA-II with constraint handling techniques performs well in the study introduced in the previous chapter, an extension of NSGA-II, called NSGA-III, is adopted as the approximate solution approach to solve the tri-objective optimization in this study. The effect of NSGA-III parameters on its performance is investigated in the numerical experiments to solve a small numerical example, providing insights for parameter tuning if it is used to validate or apply this study in other scales. Results show that with the modified Pareto efficiency, the mathematical model considering RMS environmental sustainability in this study essentially picks out the MC production plans which are more ecofriendly among the Paretooptimal solutions obtained by the mathematical model in the study in Chapter II. This is consistent with the objectives of providing RMS manufacturers with fewer but greener options to decide which solution to adopt for MC production.

This chapter follows the same structure as chapter II. First, the problem is formulated and assumptions are presented. Then the mathematical model is presented, followed by the solutions approaches where the modified pareto efficiency is described as well as the adaptation of NSGAIII for this problem. Finally, numerical examples are used to validate the proposed model and solution approaches.

III.1 -Problem formulation

Customers today require unique personalized products as well as sustainable ones.

Manufacturing, the core of the industrial economy, must be sustainable to maintain the high standard of living achieved by industrialized societies and to enable developing societies to achieve the same standard of living on a sustainable basis [START_REF] Jayal | Sustainable manufacturing: Modeling and optimization challenges at the product, process and system levels[END_REF]. Taking into account the social importance of manufacturing in our societies, while considering its huge impact on energy consumption on the use of physical resources and emissions to the environment, sustainable manufacturing can be considered as one of the most important issues to address for pursuing the big picture of sustainable development [START_REF] Garetti | Sustainable manufacturing : trends and research challenges[END_REF]. It is useful in minimizing material and energy wastage as well as improving machine utilization and process productivity coupled with higher customer satisfaction [START_REF] Rehman | Strategic outcome using fuzzy-AHP-based decision approach for sustainable manufacturing[END_REF]. Production in RMSs requires particular attention to sustainable manufacturing, because frequent reconfiguration not only costs more, consumes more resources and energy, but also increases emissions of environmentally polluting substances. In addition, MC leads to more complex manufacturing processes, and thus the optimization of its production planning including process planning, scheduling and RMS layout should maximize resource utilization and reduce negative environmental impact, such as hazardous waste that could result in severe soil contamination and marine pollution, threatening the survival of humans and other creatures. Moreover, increase of GHG concentrations in the atmosphere causes frequent extreme weather and climate change globally.

Due to the flexibility and advantages of the RMS, implementing sustainability to it will lead to (1) reduced cost, (2) reduced wastes, (3) reduced power consumption, (4) enhanced operational safety, (5) improved environmental care, and (6) improved individual health [START_REF] Dahmani | Toward Sustainable Reconfigurable Manufacturing Systems (SRMS): Past, Present, and Future[END_REF].

For example, RMS convertibility could affect environmental performance metrics such as GHG emissions, liquid waste generation, water and energy usage, and idle energy losses [START_REF] Huang | Towards Developing Sustainable Reconfigurable Manufacturing Systems[END_REF]. The RMS already has the ability and capability to satisfy the social and economic parts of the sustainability, but not the environmental part yet [START_REF] Kurniadi | Maintaining Sustainability in Reconfigurable Manufacturing Systems Featuring Green-BOM[END_REF].

In view of the above, it is worthwhile to consider environmental sustainability from the very beginning of planning MC production in RMS. Designing a well environmental oriented model is well recognized [START_REF] Khezri | A Sustainable Reconfigurable Manufacturing System Designing with Focus on Environmental Hazardous Wastes[END_REF]. Joung et al. reviewed a set of publicly available indicator sets measuring sustainability of manufactured products and manufacturing processes, among which the environmental stewardship covers environmental impacts from pollution (hazardous substances, GHG, ozone-depleting gases, and other pollutants), emissions (effluent, solid waste emission, air emission, and waste energy emission), resource use (water use, material use, energy use, and land use), and ecosystem detriment of natural habitat conservation (biodiversity, habitat management, and conservation) from manufacturing processes and products [START_REF] Joung | Categorization of indicators for sustainable manufacturing[END_REF].

GHG emissions and hazardous waste discharge are the main causes that threaten environmental degradation. Since they are easy to measure, many studies have already considered these two indicators in objectives to develop sustainable manufacturing. For that, this study considers hazardous waste discharge and GHG emissions in the objective and constraints. Referring to the common method of GHG emissions measurement in the related literature, the amount of GHG emissions is derived from converting the total amount of energy consumption with a designated factor. This is also adopted in this study. As shown in Figure III-1, there are upper limits for the total amount of the hazardous waste discharge and GHG emissions in this study.

The allowed average hazardous waste per time unit is estimated from the boundary of the hazardous waste for the manufacturer in a year or a certain duration. The allowed average GHG emissions per time unit is estimated as above. The hazardous waste discharge and GHG emission for a production should be subject to some constraints that are presented in details in mathematical model (section III.3). Other details in the problem formulation in this study are the same with those described in Chapter II. The differences of these two studies are compared in Table III-1. This study is an extension of the study in Chapter II. All assumptions, notations for parameters, indices and decision variables, objectives and constraints in the mathematical model from Chapter II are adopted in the mathematical model in this study. In order not to take up space repeatedly, only the added content will be introduced below.

III.2 -Assumptions

In addition to the assumptions made in Chapter II, here are some additional assumptions:

1) Parameters about the energy consumption of a WIP's transportation are only dependent on the type of part variant that this WIP belongs to.

2) Parameters about an operation's processing energy consumption, hazardous waste, and setup energy consumption are dependent on the type of part variant the corresponding WIP belongs to, the kind of this operation, the executing machine, and the configuration.

3) Parameters about reconfiguration energy consumption on a machine depend on the former configuration and the latter configuration.

4) Energy consumed from different activities is quantified with the same unit, which can be added directly to estimate the amount of GHG emissions.

III.3 -Mathematical model

The mathematical model for this work has been refined from the mathematical model introduced in Chapter II to minimize the loss of delay, total cost, and harm to the environment.

Hence, it is still a mixed integer nonlinear programming mathematical model.

Notations

The Hazardous waste is only discharged when performing operations, of which the amounts differ by operations, part variants, performing machines and configurations. Energy is consumed from the operations setup and processing, WIP transporting, machine reconfiguration and layout reconfiguration activities. The modelling of the energy consumption parameters above is similar to those of the parameters on operation processing and setup time and cost, WIP transportation time and cost, WIP holding cost as well as machine reconfiguration time and cost introduced in Chapter II. To be more specific, amounts of energy consumption from operation setup and processing activities differ by operations, part variants, performing machines and configuration. Amounts of energy consumption from the WIP transporting differ by the WIP types (part variants) and the distances between the two machines concerned with the WIP transportation. Amounts of energy consumption from machine reconfiguration differ by two configurations before and after the corresponding reconfiguration.

Amount of energy consumption from layout reconfiguration differ by machines and the distances they move.

As the Figure III-1 in problem formulation illustrated, the GHG emission is estimated by the amount of energy consumption. They are usually assumed to be positively correlated and converted with a coefficient. The limit amounts of the hazardous waste discharge and energy consumption during a production depend on its makespan. They can be calculated by the given amounts of the limits per time unit. The added parameters corresponding to these three constant values are presented as follows:

𝑈𝐻𝑊, Limit of the average hazardous waste per time unit.

𝑓, Emission factor for energy consumption.

𝑈𝐺𝐸, Limit of the average GHG emissions per time unit.

III.3.1.2 Decision variables

Notations for the additional auxiliary decision variables in this mathematical model are as follows.

𝑐

The completion time of the whole MC production 𝑇𝐻𝑊 The total amount of hazardous waste 𝐿𝐻𝑊 The allowed amount of hazardous waste 𝐸

The total amount of GHG emissions 𝐿𝐺𝐸

The allowed amount of GHG emissions 𝐸

The total amount of consumed energy 𝐸𝑆𝐶

The amount of consumed energy for all setup 𝐸𝑃𝐶 The amount of consumed energy for all processing 𝐸𝐹𝐶 The amount of consumed energy for all WIP transportation 𝐸𝑅𝐶 The amount of consumed energy for all machine reconfiguration 𝐸𝐿𝐶 The amount of consumed energy for the layout reconfiguration

Objective function

There are three objectives in this mathematical model. The first two objectives are the same as those introduced in Chapter II. The added third objective function is as follows:

𝑀𝑖𝑛 𝑇𝐻𝑊 𝐿𝐻𝑊 𝐸 𝐿𝐺𝐸 III-1
The third objective is to minimize the value of the environmental performance indicator defined as the aggregate of the normalized hazardous waste item (the ratio of the total amount of hazardous waste 𝑇𝐻𝑊 and the allowed amount of hazardous waste 𝐿𝐻𝑊 ) and the normalized GHG emissions item (the ratio of the total amount of GHG emissions 𝐸 and the allowed amount of GHG emissions 𝐿𝐺𝐸).

Constraints

Most of added constraints are definitions related to terms in the third objective, including total amounts of hazardous waste and GHG emissions, together with allowed amounts of hazardous waste and GHG emissions. Total amount of GHG emissions calculation is defined below. There are only two constraints indicating the relationship between total amounts of hazardous waste and GHG emissions with allowed amount of hazardous waste and GHG emissions. Expressions and thorough explanations of each constraints are presented in the following subsection.

III.3.3.1 Definitions of total/ allowed amount of hazardous waste and GHG emissions

The total amount of the hazardous waste 𝑇𝐻𝑊 and the allowed amount of it 𝐿𝐻𝑊, as well as the total amount of GHG emission 𝐸 and the allowed amount of it 𝐿𝐺𝐸 in the above objective function, are defined by the following equations:

𝑐 𝑚𝑎𝑥 𝑐 ∀i ∈ I, ∀v ∈ V, ∀j ∈ 1, … , J ,

III-2

This constraint defines the completion time of the whole MC production.

𝐿𝐻𝑊 𝑐 𝑈𝐻𝑊 III-3

The allowed amount of hazardous waste 𝐿𝐻𝑊 is estimated by multiplying the time of the whole MC production with limit of the average hazardous waste per time unit.

𝐿𝐺𝐸 𝑐 𝑈𝐺𝐸 III-4

The allowed amount of hazardous waste 𝐿𝐺𝐸 is estimated by multiplying the time of the whole MC production with limit of the average GHG emissions per time unit.

𝑇𝐻𝑊 ∑ ∑ ∑ ∑ 𝐻𝑊 , , , , , , , , , , , , | |
, | | | | III-5
The discharge of hazardous waste comes from the processing activity only. The above equation calculates the total amount of hazardous waste.

𝐸 𝐸 𝑓 III-6

The amount of GHG emissions are determined by the amount of energy consumed, thus it is estimated by multiplying the total amount of consumed energy with emission factor for energy consumption.

𝐸 𝐸𝑆𝐶 𝐸𝑃𝐶 𝐸𝐹𝐶 𝐸𝑅𝐶 𝐸𝐿𝐶 III-7

This equation implies the total amount of consumed energy comprising the total energy consumed from the setup ( 𝐸𝑆𝐶 ), processing ( 𝐸𝑃𝐶 ), WIP transport ( 𝐸𝐹𝐶 ), machine reconfiguration (𝐸𝑅𝐶), and layout reconfiguration (𝐸𝐿𝐶) activities. They are calculated by the following equations which are similar to those calculating the corresponding cost in the second objective function presented in Chapter II.

𝐸𝑆𝐶 ∑ 𝐸𝑆 , , , ,

, 𝑣 , ∨ ∀𝜌 , 𝜌 , ∨ ∀𝜑 , 𝜑 , III-8 𝐸𝑃𝐶 ∑ ∑ ∑ ∑ 𝐸𝑃 , , , , , ∑ 𝐸𝑆 , , , , , , | | ∀𝑣
| | | | III-9 𝐸𝐹𝐶 ∑ ∑ ∑ ∑ 𝐸𝐹 ∆𝑥 , , , , , , , , , , , , | | ,
| | | | III-10 𝐸𝑅𝐶 ∑ ∑ 𝐸𝑅 , , , , , , ∆𝑦 , , , , , , , | | ,
∑ 𝐸𝐿 |𝑥 𝑋 | |𝑦 𝑌 | | | III-12 III.3.3. , , | | III-11 𝐸𝐿𝐶

Constraints on hazardous waste discharge and GHG emission

The added auxiliary decision variables are subject to the following constraints:

𝑇𝐻𝑊 𝐿𝐻𝑊

III-13

This constraint restricts a solution to be feasible when the total amount of hazardous waste does not exceed the maximum allowed amount.

𝐸 𝐿𝐺𝐸 III-14

Ditto, this constraint restricts a solution to be feasible when the total amount of GHG emissions does not exceed the amount it is allowed.

The above constraints plus all constraints presented in Chapter II make up the full set of constraints for the mathematical model in this study. This study brings forward a constructive mathematical model to take on an MC order fabricated in an RMS-equipped factory for reduced loss of delay, total cost and harm to the environment. The solution approaches to gain the optimum value of the above decision variables are introduced hereafter.

III.4 -Solution approaches

This work employs the solution representation and constraint handling methods described in Chapter II in its solution approaches. A reference-point-based many-objective EA following NSGA-II framework (called NSGA-III) proposed by Deb and Jain has been proven to be able to successfully find a well-converged and well-diversified set of points repeatedly over multiple runs [START_REF] Deb | An evolutionary many-objective optimization algorithm using reference-point-based nondominated sorting approach, Part I: Solving problems with box constraints[END_REF]. Since this mathematical model has three objectives, the NSGA-III is adopted to obtain approximate Pareto-optimal solutions. A problem is that there are too many Pareto-optimal solutions when the number of objectives increases. This study modified the general Pareto efficiency to reduce the optimal solutions in the Pareto front, inasmuch as multiobjective optimization will probably have redundant solutions to confuse decision makers. The modified general Pareto efficiency is actually a two level of non-domination sorting by two indicators respectively from the economic and environmental points of view. It is a novel proposal that can be extended to any multi-objective optimization.

Modified Pareto efficiency

The probability that two solutions are non-dominated to each other increases in multiobjective optimization, when the number of objectives increases. For a bi-objective optimization, there are nine relationships between two solutions (e.g., solution 𝑢 and solution 𝑣) entailing their first and second objective values 𝑓 , 𝑓 , 𝑓 and 𝑓 , which are:

(1) 𝑓 𝑓 and 𝑓 𝑓 ;

(2) 𝑓 𝑓 and 𝑓 𝑓 ;

(3) 𝑓 𝑓 and 𝑓 𝑓 ;

(4) 𝑓 𝑓 and 𝑓 𝑓 ;

(5) 𝑓 𝑓 and 𝑓 𝑓 ;

(6) 𝑓 𝑓 and 𝑓 𝑓 ;

(7) 𝑓 𝑓 and 𝑓 𝑓 ;

(8) 𝑓 𝑓 and 𝑓 𝑓 ;

(9) 𝑓 𝑓 and 𝑓 𝑓 .

Among above, these two solutions are non-dominated to each other when they match

(3), ( 5) or [START_REF] Deshmukh | Framework for Manufacturing in Post-COVID-19 World Order: An Indian Perspective[END_REF], i.e., there is one-third probability for two solutions to be non-dominated to each other.

For a three-objective minimizing mathematical model, solution 𝑢 and solution 𝑣 sorted by the general Pareto efficiency have twenty-seven relationships, which are (the third objective values of these two solutions are denoted by 𝑓 and 𝑓 in the following expressions):

(1) 𝑓 𝑓 and 𝑓 𝑓 and 𝑓 𝑓 ;

(2) 𝑓 𝑓 and 𝑓 𝑓 and 𝑓 𝑓 ;

(3) 𝑓 𝑓 and 𝑓 𝑓 and 𝑓 𝑓 ;

Owing to the fact that the first objective function and second objective function are quantified in currency, the sum of these two objectives (𝑓 𝑓 ) could be interpreted as the economic indicator. If 𝑓 𝑓 is set as the first objective function and 𝑓 is set as the second objective function to formulate a bi-objective model directly, two solution 𝑢 and solution 𝑣 are This is a problem-specific sorting method. It can be used in both exact and approximate solution approaches. As NSGA-III has been proven more efficient for optimization with more objectives, it is adopted as the approximate solution approach to solve the mathematical model in this study and is described in the next subsection.

NSGA-III

NSGA-III was proposed to solve many-objective optimization problems having more than three objectives exclusively, although NSGA-III was demonstrated to work well on threeobjective optimization problems [START_REF] Seada | U-NSGA-III: A Unified Evolutionary Optimization Procedure for Single, Multiple, and Many Objectives: Proof-of-Principle Results[END_REF]. According to the research of Campos Ciro et al., the NSGA-III has a better performance than NSGA-II if the size of the instance problem is increased and it is better adapted to solve problems with more than 2 objectives [START_REF] Ciro | A NSGA-II and NSGA-III comparison for solving an open shop scheduling problem with resource constraints[END_REF]. Crowding distance does not estimate crowdedness well in the case with tri-objective test problems and presumably also in cases of more objectives [START_REF] Kukkonen | Improved pruning of non-dominated solutions based on crowding distance for bi-objective optimization problems[END_REF]. Therefore, the crowding distance is not used in NSGA-III and a set of uniformly distributed direction vectors is used for diversity maintenance, of which the basic idea is to find a non-dominated solution around the intersection of the Pareto front and each reference line [START_REF] Ishibuchi | Performance comparison of NSGA-II and NSGA-III on various many-objective test problems[END_REF], while NSGA-II uses crowding distance operator for the same purpose [START_REF] Seada | U-NSGA-III: A Unified Evolutionary Algorithm for Single, Multiple, and Many-Objective Optimization[END_REF].

NSGA-III is exactly the same as NSGA-II until the merged parent and offspring population is applied with its selection operator [START_REF] Vesikar | Reference Point Based NSGA-III for Preferred Solutions[END_REF]. NSGA-III focus its search on finding an associated Pareto-optimal solution for each reference point and keeping NSGA-II's emphasis on non-dominated solutions intact, its elitist selection mechanism was modified to incorporate This study uses NSGA-III in its entirety with the addition of constraint handling techniques. In NSGA-III, the number of the reference points, mutation probability, and the generation limit stipulating the number of iterations in a run mainly will probably influence the performance of this approach. Factor analysis is carried out in Minitab to check the dependency relating the effectiveness and efficiency of this approach with the three parameters (number of reference point, mutation probability, and generation limit). A small numerical example is used to validate the tri-objective mathematical model and test the effectivity and efficiency of NSGA-III with the applied constraint handling techniques and unconstrained archive introduced in Chapter II.

III.5 -Numerical experiment for validation

This work uses a small numerical example to validate the mathematical model, the Exact Pareto-optimal solutions By using the modified Pareto efficiency, there are three exact Pareto-optimal solutions obtained from the exhaustive search to this small numerical example. Table III-2 lists the values of the three objectives in each solution.

Table III-2. The values of the multi-objective functions in the exact Pareto-optimal solutions for the small numerical example Table III-3 displays the optimal layout in each exact Pareto-optimal solution. In fact, it is the same as the initial layout, reflecting that layout reconfiguration is the inferior strategy to arrange optimal production management. Table III-4 gives the optimum beginning time to produce every part in the optimal operation sequence on the best machine with the selected configuration. I can be observed that every job is completely processed on one machine in each solution, which makes it seem that machine reconfiguration is superior to the WIP transport for this small numerical example.

Table III-4. The optimal production scheme in the exact Pareto-optimal solutions for the small numerical example

There are seven exact Pareto-optimal solutions for the model without the third objective function and the relevant parameters in this problem formulation. The values of the two objective functions, the optimal layout, and the optimal process planning and scheduling arrangement in each solution are respectively presented in Table III

(𝑖, 𝑣, 𝑗, 𝑞) 1, 1, 1, 1 1, 1, 1, 2 1, 1, 1, 3 2, 2, 1, 1 2, 2, 1, 2 2, 2, 1, 3 Solution 1 𝜌 , , , 0 1 2 0 1 2 𝛼 , , , 1 1 1 0 0 0 𝜑 , , , 1 2 0 1 0 0 𝛽 , , , 3 27 45 3 12 23
Solution 2

𝜌 , , , 0 1 2 0 1 2 𝛼 , , , 0 0 0 0 0 0 𝜑 , , , 1 0 0 1 0 0 𝛽 , , , 31 48 56 3 12 23
Solution 3

𝜌 , , , 0 2 1 0 1 2 𝛼 , , , 0 0 0 0 0 0 𝜑 , , , 1 0 0 1 0 0 𝛽 , , , 31 43 59 3 12 23

Approximation optimization

Three approximate Pareto-optimal solutions are obtained in a run by setting 𝑝 2, mutation probability (0.05), and generation limit (2000). Hence, the number of the reference points is equal to six, twice the number of the exact Pareto-optimal solutions.

The objective values, optimal layout, and the optimal production scheme in these approximate Pareto-optimal solutions are displayed in Table III-8, Table III-9 and Table III-10. For the optimal layout, only Machine 2's position on the X-coordinate is changed compared with that in the exact Pareto-optimal solutions. The process plans for these solutions are almost the same as that of the exact Pareto-optimal solutions, while the scheduling is quite different as the beginning times of operations alter a lot.

0 0 0 0 0 0 𝜑 , , , 1 0 0 1 0 0 𝛽 , , , 5 39 55 13 22 33
The computation time of the corresponding run is 7.828125 seconds, saving 99.6% of the computation time to obtain the exact Pareto-optimal solutions (2093.28125 seconds). The value of the effectivity metric 𝑒𝑓𝑓 is 0.256988154, which means the deviation of the mean value for three objective values is less than 26%.

There are three general parameters in NSGA-III influencing its effectivity and efficiency, which are the number of reference point, mutation probability and generation limit. To describe their effects on the performance of NSGA-III, a factor analysis is used. There are two levels for each factor in factor analysis, as shown in

III.6 -Conclusion

Sustainability is becoming a necessary decision criterion for manufacturing. It is becoming more and more urgent to consider environmental performance when designing and optimizing manufacturing. This chapter presented an extension to the mathematical model in the previous chapter, building a multi-objective optimization minimizing not only the total tardiness penalty and the total cost for MC, but also a combined ratio of total amount of hazardous waste discharge and GHG emissions to the allowed amount of them, which is in line with the international goal of realizing low-carbon development. show that this approximate solution approach is effective and efficient enough, which are proved by the performance of NSGA-III in solving a small numerical example. The factor analysis and main effects of general parameters, number of reference points, mutation probability and generation limit in NSGA-III on its performance are also investigated in the numerical experiments.

The limitation of this study is deficient validation with regard to the solution approaches.

Furthermore, numerical experiments presented in the previous and this chapter reflect that the performance of heuristics relies heavily on expertise on parameters setting, such as EA (including NSGA-II and NSGA-III) and AMOSA. Machine learning and deep learning may cope with this limit and show a great deal of promise for intelligent manufacturing [START_REF] Wang | From Intelligence Science to Intelligent Manufacturing[END_REF]. A novel Q-learning based reinforcement learning method is proposed to solve the research problem integrating MC process planning and scheduling in RMS. It is introduced in the next chapter .

CHAPTER IV -Planning and scheduling integrated optimization for MC in an RMS by using Q-learning based reinforcement learning

This chapter introduces a study using a machine learning method to solve the process planning and scheduling integrated optimization problem for MC in an RMS. This is due to the fact that the results of numerical experiments in the previous two studies imply that the performance of heuristics (NSGA-II, AMOSA and NSGA-III) is highly influenced by the expertise on parameters setting for different scales of the formulated problem, therefore it is imperative to develop intelligent approximate solution approaches to self-adjust their performance. Reinforcement learning is a type of machine learning that can be used as approximate dynamic programming to obtain the approximate optimal solutions to a Markov decision process. It is possible to define an Markov decision process for combinatorial optimization problems [START_REF] Mazyavkina | Reinforcement learning for combinatorial optimization: A survey[END_REF]. Process planning and scheduling for an MC production task in an RMS is that of a finite Markov decision process in which deciding machine and configuration to perform operations in sequences is discrete-time. On the other hand, layout reconfiguration as considered in this research is not a sequential decision-making process for MC production.

Therefore, it is not considered in this study.

This study adopted Q-learning, a reinforcement learning algorithm as the approximate solution approach to obtain approximate optimal solutions for the bi-objective mathematical model introduced in Chapter II (excluding decisions and constraints related to layout reconfiguration). To the best of our knowledge, it is the first time to use this method for multiobjective optimization. Since it is the first attempt to use reinforcement learning method solving multi-objective optimization for production planning in RMS, it is easier to solve the biobjective mathematical model in Chapter II than the three-objective mathematical model in Chapter III. The problem-specific Q-learning based reinforcement learning procedure is introduced in this chapter. To validate this proposal, two numerical examples in small and large scale respectively are used to evaluate the accuracy and efficiency of this approximate solution approach. Both numerical examples used the improved NSGA-II with constraint handling techniques introduced in Chapter II to solve them. In the numerical experiments to solve the small example, four metrics comparing the approximate Pareto-optimal solutions with the exact Pareto-optimal solutions are adopted to evaluate the accuracy of the proposed Q-learning based approximate solution approach. The results show that it tends to obtain the exact Pareto-optimal solutions that could balance the workload on each machine. In the numerical experiments to solve the large numerical example, another four metrics comparing the approximate Paretooptimal solutions with each other, the computation time, and the mean values of two objective are adopted to prove the efficiency and effectiveness of the proposed Q-learning based approximate solution approach. The results show that the performance of this approximate solution approach is substantially more advantageous.

This chapter starts with a literature review of reinforcement learning for MC in RMS and introduction of Q-learning. Then it presents the updated problem formulation, assumptions, and mathematical model respectively. Following, it describes how Q-learning is implemented for this problem. Finally, numerical experiments are described and a conclusion is presented.

IV.1 -Literature review of reinforcement learning for MC in RMS and introduction of Q-learning

Literature review

With the development of Artificial intelligence (AI), computers have the ability to learn, that is, they can not only make better decisions faster than humans on many problems, but also make self-adaptive adjustments according to different problems and environments. With the advent of the Industry 4.0, copious availability of data, high-computing power and large storage capacity have made of machine learning approaches an appealing solution to tackle manufacturing challenges [START_REF] Usuga Cadavid | Machine learning applied in production planning and control: a state-of-the-art in the era of industry 4.0[END_REF]. A statistic of studies related to manufacturing and machine learning from 2000 to 2020 shows that the number of studies has been increasing rapidly, with the proportion of deep learning increasing in recent years and the number of reinforcement learning increasing in the last 3 years [START_REF] Nassehi | Review of machine learning technologies and artificial intelligence in modern manufacturing systems[END_REF]. Reinforcement learning, as an important branch of AI algorithms, originally owns an outstanding capability of sequential decision-making [START_REF] Li | Deep reinforcement learning in smart manufacturing: A review and prospects[END_REF].

It is born out of mathematical psychology and operations research, provides qualitative and quantitative computational-level models of these solutions [START_REF] Dayan | Reinforcement learning: The Good, The Bad and The Ugly[END_REF]. It is the current most popular optimization solution approach to the Markov decision processes, in which the transition of two consecutive states is only determined by the former state and the decision. The core of a reinforcement learning system is the agent that operates in an environment that models the task that it has to fulfill [START_REF] Canese | Multi-agent reinforcement learning: A review of challenges and applications[END_REF]. In the sequential Markov decision processes, an agent continually takes an action to the environment and learns from the reward that the environment gives back to the agent. The optimal decisions are found when the desired state is reached or when the reward is maximum.

When the keywords "Reinforcement learning" AND "Reconfigurable manufacturing system" AND "Mass customization" AND "Planning" AND "Scheduling" are used to search the related studies in publications, there are few studies that were very relevant to them and most of them are conference papers even preprints on arXiv (a free distribution service and an open access archive for scholarly articles), which implies that this research is still in its infancy.

Here comes a brief literature review about it.

Most of the search results are studies using reinforcement learning method, especially deep Q-networks, to solve the scheduling problem. They generally mentioned MC and the need of flexibility for manufacturing systems in the introduction to express the prospect of intelligent manufacturing and smart factories, but do not specifically consider the reconfigurability of RMS in their studies. For example, Zhou et al. presented new cyber-physical integration in smart factories for online scheduling of low-volume-high-mix orders by proposing AI schedulers with novel neural networks for each unit (e.g., warehouse, machine) to schedule dynamic operations with real-time sensor data and designing new reward functions to improve the decision-making abilities of multiple AI schedulers based on a reinforcement learning method, deep Q-networks [START_REF] Zhou | Multi-agent reinforcement learning for online scheduling in smart factories[END_REF]. Kim et al. present a smart manufacturing system using a multiagent system and reinforcement learning (a deep Q-network), which is characterized by machines with intelligent agents to enable a system to have autonomy of decision-making (evaluate the priorities of jobs and distribute them through negotiation.), sociability to interact with other systems, and intelligence to learn dynamically changing environments [START_REF] Kim | Multi-agent system and reinforcement learning approach for distributed intelligence in a flexible smart manufacturing system[END_REF].

Few search results are studies using reinforcement learning method to solve process planning problem. For example, Wu et al. employed deep reinforcement learning (An actorcritic algorithm) for process planning, aiming at promoting the response speed by exploiting the reusability and expandability of past decision-making experiences [START_REF] Wu | A fast decision-making method for process planning with dynamic machining resources via deep reinforcement learning[END_REF]. Similar to the above, the machine or layout reconfiguration is not considered in these few studies.

Two studies following this sentence used reinforcement learning method to solve process planning and scheduling integrated problem without considering it in a reconfigurable environment. Mueller-Zhang, Antonino, and Kuhn presented a digital twin based self-learning process planning and scheduling integrated approach using Deep-Q-Network that is capable of identifying optimized process plans and workflows for the simultaneous production of personalized products [START_REF] Mueller-Zhang | Integrated planning and scheduling for customized production using digital twins and reinforcement learning[END_REF]. Based on real-time data acquisition and fusion under the framework of advanced shop-floor communication and computing technologies such as wireless sensor, actuator network, and edge computing, Tan et al. established a multi-agent model of industrial robot assemble process and proposed a multi-agent reinforcement learning approach (Q-learning) for planning and scheduling of an industrial robot assembly [START_REF] Tan | Modeling , planning , and scheduling of shopfloor assembly process with dynamic cyber-physical interactions : a case study for CPS-based smart industrial robot production[END_REF].

There are only four studies using reinforcement learning method in RMS and all of them solved the scheduling problem. Tang et al. first used a reinforcement learning method (Deep Q learning) to train a job releasing schedule agent for an RMS scheduling problem that aims at the completion of the assigned order lists while minimizing the reconfiguration actions [START_REF] Tang | A Deep Reinforcement Learning Based Scheduling Policy for Reconfigurable Manufacturing Systems[END_REF].

Then they found a dynamic control policy for the scheduling problem of RMS on multiple products via a group of deep reinforcement learning agents that embedded with a shared value decomposition network, aiming on minimizing the makespan of a constant updating order group by guiding a group of automated guided vehicles to move modules of machines, raw materials, and finished products [START_REF] Tang | Reconfigurable manufacturing system scheduling : a deep reinforcement learning approach[END_REF]. Yang et al. first studied the intelligent scheduling and reconfiguration with dynamic job arrival for a reconfigurable flow line using deep reinforcement learning (the advantage actor-critic) [START_REF] Yang | Intelligent scheduling and reconfiguration via deep reinforcement learning in smart manufacturing[END_REF]. Then they proposed a novel variant of deep Q-network -expected deep Q-network to solve this real-time and concurrent optimization of scheduling and reconfiguration for a dynamic reconfigurable flow shop with new job arrivals [START_REF] Yang | Real-time and concurrent optimization of scheduling and reconfiguration for dynamic reconfigurable flow shop using deep reinforcement learning[END_REF].

Based on the above, there is no study using the reinforcement learning method to solve the integrated planning and scheduling problem for MC in RMS. Besides, Q-learning is the most popular reinforcement learning method used to do operation research. Therefore, this study adopted Q-learning as the solution approach to obtain the optimal solutions to the formulated bi-objective mathematical model introduced in the previous section for MC process planning and FJSP in an RMS.

Introduction of Q-learning

Q-learning is a form of model-free reinforcement learning [START_REF] Watkins | Q-learning[END_REF]. It is referred to the class of methods that identifying an optimal decision strategy in a sequential decision setting to estimate the optimal Q-function, which measures the expected cumulative utility of each currently available decision, given that the decision maker will follow the optimal decision strategy in the future [START_REF] Azab | Mechanics of change: A framework to reconfigure manufacturing systems[END_REF]. As the principle of reinforcement learning shown in Figure IV-1, there is at least one agent in this method to make decisions on the environment. The agent chooses an action at a state and then the environment evaluates the action with a reward as feedback. The state changes after the action is performed. By learning rewards and new states, the agent continuously adjusts the action to reach ideal state or some goal in the environment.

Figure IV-1 The basic principle of reinforcement learning

The main procedures of Q-learning is shown in Figure IV-2. For a Markov decision process, there is a Q-table to record Q-values, which are cumulative scores for evaluating actions in every state. A sequence of states corresponding to the complete Markov decision process is an episode. In the training period, the Markov decision process repeats many times and there is a limit to the episodes. In each state, the agent chooses an action based on 𝜖-greedy strategy, which is similar to the mutation probability in GA by setting a threshold, ϵ, between 0 and 1 and creating a small probability of 1 𝜖 to choose the action randomly. When the action is chosen, the state transits to a new one. which is between 0 and 1, giving the weight to learn from the temporal difference that will introduced later. 𝑟 is the reward to evaluate the chosen action 𝑎 , which is determined by the designed principles. 𝛾 is another parameter called discount factor, which is the weight to take the influence of the next state into consideration. max 𝑄 𝑠 , 𝑎 is the estimate of optimal future value, which is the maximum value of the Q-values for all the actions that can be chosen

IV.2 -Problem formulation

The decision-making on process planning and scheduling for MC in an RMS can be formulated into a Markov decision process. To be more specific, the first three core research questions in this thesis that determine the operation sequence to produce every mass-customized product, the selected machine and configuration to perform each operation and the processing sequence on machines to finish the whole multiple multi-unit MC production could be answered in a sequential event. Having as an input the number and the requested mass-customized products, and since the parts that make them up and the operations to process each part are known, the integrated process planning and scheduling for MC in an RMS is a finite Markov decision process as the total number of operations can be counted. On each machine, operations are performed one after another. The decision-making occurs when each machine finishes an operation. It should decide which operation for which part from which mass-customized product will be the next and this operation should be performed by which configuration.

Different operations will have different costs on machine reconfiguration, setup and processing activities and their corresponding WIP's transportation and holding activities. The decision will also affect the final tardiness penalty since it is related to the completion time of every masscustomized product and the makespan of the entire MC production. In this thesis, the last core research question on layout reconfiguration is not a sequential decision-making event, as the problem formulation in Chapter II describes that the reconfigured layout cannot be changed during the MC production. Therefore, this study only considers the integrated process planning and scheduling problem for MC in an RMS. The current state consists of the state of the mass-customized products and the state of machines in the RMS. The state of every mass-customized product can be further divided into the state of jobs in it. The state of each job can be further divided into the state of operations in that job.

There are two state types of an operation, which are the state of a finished operation and the state of an unfinished operation. The state of a finished operation has the processing information, including the selected machine and configuration as well as the beginning time and the completion time to perform it. The state of an unfinished operation is empty but the information that which machine with which configuration can perform it is given by parameters, as well the precedent operations to this unfinished operation. The state of a machine at any moment has the utilization related information, including the current processed operation, the corresponding part variant and the configuration. Based on the above states and the related parameters, the central agent should select an operation that can be processed on the idle machine making sure all its precedent operations have been finished. The selected operation must be identified by the mass-customized product which it is from, the type of part variant and the job ID. Besides, the agent should also select a configuration suitable for processing the selected operation on this idle machine. All the information in the above decision form an action.

This time, the same two numerical examples (excluding layout reconfiguration) are investigated by a new approximate solution approach, which is on the basis of Q-learning, the most popular reinforcement learning method. Different from the improved heuristics in the previous two studies generating all the values of the independent decision variables for a solution at once, this method generates the values of independent decision variables for a solution in a sequential Markov decision process. The introduction of this Q-learning based reinforcement learning solution approach is the main content of this chapter. The results of numerical experiments presented after the introduction of this proposed new approximate solution approach show that it is promising to be developed as the most practical and intelligent method for operational decision-making in MC production planning in the context of RMSs.

IV.3 -Assumptions

All the assumptions introduced in Chapter II are kept in this study, except the fifth assumption is modified since layout reconfiguration is not considered in this study: 5) Only machine reconfiguration is considered in this RMS.

IV.4 -Mathematical model

IV.5 -Solution approach

This study adopts the exhaustive search introduced in Chapter II as the exact solution approach to obtain all the exact Pareto-optimal solutions to the small numerical example. The proposed new Q-learning based reinforcement learning solution approach is introduced as the approximate solution approach to obtain some approximate Pareto-optimal solutions to the small and large numerical examples in the next section. No infeasible solutions are generated during the application of this novel approximate solution approach as it is a sequential dynamic programming process in which all the constraints related to the value of each independent decision variable are considered when it is determined. Therefore, there is no need to use the constraint handling techniques introduced in Chapter II.

This study proposes a Q-learning based reinforcement learning method as the approximate solution approach to solve the formulated problem that is simulated into a sequential decision-making event where a central agent is employed to assign jobs' operations to machines, as described previously. Assignments are chronological, depending on the state of mass-customized products and the state of machines. The number of assignments ( 𝑛 ) is determined, since the mass-customized products and their components are given and the operations to process every part variant are known. The state of product 𝑖 contains the values of three independent decision variables for all assigned operations and a set of all unassigned operations in this product ( 𝑁𝑂𝑇_𝐷𝑂 ). The state of machine 𝑚 contains the current configuration (𝑔 ), the part variant of the current processing job (𝑣 ), the current processing operation (𝑒 ) and its completion time (𝑐𝑡 ). 𝑐𝑡 0 for the initial state of machine 𝑚.

During the MC production in an RMS, the assignment occurs at each time when there is an ideal machine. As the state of machines knows the completion time of the processing operation on each machine at any time, the first ideal machine 𝑚 * , which means the machine that first finishes its operation processing, is determined by the following equation:

𝑚 * arg min ∈ 𝑐𝑡 IV-3
As the machine for assignment is determined, the operation to be processed on it should be identified by the corresponding mass-customized product, the part variant and the job ID. Considering the first objective to minimize the total tardiness penalty, the mass-customized product 𝑖 * from which one operation is selected to be performed on the determined machine is found by the following equation:

𝑖 * arg max ∈ 𝑓 𝑖 𝑒 * IV-4
Observing this equation, it is obvious that the mass-customized product with earlier due date (𝐷 ) and greater parameter of tardiness penalty per time unit (𝑊 ) is more likely to be selected. Especially those that are already delayed at the assignment moment, their values of the function 𝑓 𝑖 in the above equation are greater than 1, while the value of this function is less than 1 for any mass-customized product that is not finished and has not been delayed yet.

If more than one machine/product meet the conditions, the selection of 𝑚 * /𝑖 * will be stochastic among all those alternatives.

The final decision is to select an operation that can be performed on the selected machine 𝑚 * from the set of all unassigned operations 𝑁𝑂𝑇_𝐷𝑂 * in the selected product 𝑖 * with a proper configuration. To be more specific, only the operations with no precedent operation or for which all precedent operations are done, are ready to be selected.

IV-6

The first policy 𝑃 is to select an operation from any part (identified by part variant, job ID, and the determined mass-customized product 𝑖 * ) and a configuration that performs the selected operation on the determined machine 𝑚 * in the shortest completion time, which is the sum of the selected operation's beginning time 𝛽 * , , , and processing time 𝑃𝑇 , , * , . This benefits the first objective which is to minimize the total tardiness penalty, as it helps to reduce the makespan of the whole MC production in RMS by trying to complete all operations with proper configurations as early as possible. No matter which operation is selected, the ordinal (𝑞) of it is always the same and known. As the state of product 𝑖 * contains the values of three independent decision variables for all assigned operations, the performing machine (𝑚 ) and The second policy 𝑃 is to select an operation from the determined mass-customized product 𝑖 * and a configuration to perform the selected operation on the determined machine 𝑚 * with the lowest cost. Whether there are setup cost (𝑆𝐶 , , * , ) and reconfiguration costs (𝑅𝐶 , , * , ) depends on the state of machine and the selected configuration. The corresponding WIP's transportation cost (𝑊𝐹𝐶 , , * , ) and holding cost (𝑊𝐻𝐶 , , * , ) are determined by the completion time (𝑐 , , , ) of the last operation just completed in the operation sequence of the corresponding part from the determined mass-customized product 𝑖 * and the machine (𝑚 ) to complete the previous operation. To be more specific, these two costs can be calculated by the following two equations: This study draws on the idea of the Q-learning for finding an optimal decision strategy path of better obtaining the approximate Pareto-optimal solutions to the formulated problem.

However, different from the traditional idea using Q-value to evaluate actions, this study uses Q-value to evaluate two policies 𝑃 and 𝑃 introduced above in order to know which objective should be biased by the corresponding policy at different stages of production to make all assignments appropriate, so as to obtain the final approximate Pareto-optimal solutions with two small objective function values. As the values of two objective functions will be obtained until all the 𝑛 operations are assigned, and this pair of objective values should be compared with those of the other solutions to know whether this solution is Pareto-optimal or not, the Qvalue of each policy should be upgraded after all the assignments in one episode are finished.

While at each assignment, the actions selected by two policies 𝑃 and 𝑃 can also be compared to evaluate the two policies and update their Q-values. Therefore, one Q-table is not enough. It is favorable to have two Q-tables, one of which is for Q-value update during the assignment procedure in an episode, and another one of which is for Q-value upgrade at the end of an episode when all the assignments are finished. updated after the second episode. There are two conditions to stop the iteration. Besides the common episode limit, the number of continuously obtaining a solution that is not dominated by any Pareto-optimal solutions saved in the Pareto-optimal solution set, 𝑛𝑡, also has a limit.

Setting this limit is because the episode limit, 3 , is an exponential function. When the number of operations increases, there will be an exponential explosion to the value of this limit. As the complexity of the production planning problem are determined by the number of machines (𝑚) in the RMS and the number of operations (𝑛) for all mass-customized products, the value of 𝑚 𝑛 is set as the second limit to control the iteration of learning. The updated Q-values 0.5, 0.5 . This section introduced Q-learning, a reinforcement learning method which is totally different from heuristics, to solve the formulated process planning and scheduling integrated problem for MC in an RMS. Combining the traditional learning principles with the Markov decision process formulated in this study, the problem-specific Q-value update and upgrade techniques are proposed to simplified this adjusted approximate solution approach. In fact, The essence of using Q-values to evaluate two policies 𝑃 and 𝑃 for selecting actions (operation and configuration) is to alternate time-oriented (in favor of the first objective that minimizing the total tardiness penalty) and cost-oriented (in favor of the second objective that minimizing the total cost) strategies at each decision-making state to get an optimal MC production plan with considering the machine reconfiguration in the RMS.

If

IV.6 -Numerical experiment for validation

The same two numerical examples presented in Chapter II to validate the new proposed approximate solution approach, a Q-learning based reinforcement learning method, are used.

The reason for this is to show that the results of all numerical experiments are not from intentionally chosen numerical examples, on which the evaluated solution approaches have obtain the approximate Pareto-optimal solutions to the small numerical example in this numerical experiment. With the most appropriate parameter setting such as the population size (𝑝 ) is 40, the mutation probability (𝑝 ) is 0,1, and generation limit (𝑔 ) is 1000, the improved NSGA-II with the constraint handling techniques can obtain all exact Pareto-optimal solutions.

However, the computation time of this approach is more than twenty times that of the exhaustive search to obtain all the exact Pareto-optimal solutions. This proves that without parameters tuning, the performance of this approach, like all other heuristics, is not always good for different problems, whereas the parameters setting highly depends on the expertise of manufacturers on production planning for MC and requires extra time. This demonstrates the significance of using reinforcement learning. To let the improved NSGA-II with the constraint handling techniques have the similar performance of the Q-learning based approximate solution approach, the above three parameters are modified to 𝑝 10, 𝑝 0.1 and 𝑔 50. At this time, eight of the ten runs could obtain at least one exact Pareto-optimal solutions, which is the same to that of the Q-learning based approximate solution approach. At this time, the deviation of the approximate solutions obtained by the improved NSGA-II with the constraint handling techniques is larger than that of the approximate solutions obtained by Q-learning based approximate solution approach, while both approaches could obtain similar portion of exact Pareto-optimal solutions, which indicates nearly the same accuracy of them.

With small parameters, the computation time of the improved NSGA-II with the constraint handling techniques reduced a lot, but it is still larger than that of the proposed Q-learning based approximate solution approach.

From the obtained results, it is shown that the proposed Q-learning based approximate solution approach performs better to this small numerical example. To further investigate its advantages, a large numerical example is also adopted in the numerical experiments. Results and analysis are described in the following subsection.

Large Numerical example Pareto-optimal solution with larger objective values, so that the distribution of these solutions is more dispersed. All in all, the proposed Q-learning based approximate solution approach has the best performance for both small and large numerical example.

IV.7 -Conclusion

This chapter introduced the application of a reinforcement learning method, Q-learning, to solve the integrated process planning and scheduling problem to produce multiple multi-unit mass-customized products in an RMS. To the best of our knowledge, it is the first attempt to use this kind of method solving for multi-objective optimization in the research field of RMS production by considering machine reconfiguration in the process planning and scheduling integrated problem for MC. Therefore, it is considered as the third contribution of the research work presented in this thesis. The time and cost oriented bi-objective mathematical model and its mixed integer nonlinear programming introduced in Chapter II are adopted in the problem formulation of this study, except for few parts related to the layout reconfiguration.

The intention to carry out this study is from the observation that the heuristic methods, which are most widely used to solve the operation research for MC in RMS, have very inconsistent performance for different scales of problems. And parameters tuning takes time and requires expertise, which is not intelligent to cope with the goal of Industry 4.0. As machine learning, especially reinforcement learning, could interpret some transcendent insights for manufacturers to have intelligent decision-making capability in production planning and operation management, this study conducted a brief literature review on it and adjusted the Qlearning to solve the formulated MC planning and scheduling problem in RMS.

A major difference is that the Q-values in this study do not evaluate actions, but rather two policies 𝑃 (minimum time taken by the selected operation and configuration) and 𝑃 (minimum cost for the selected operation and configuration) for choosing actions that favors A limitation of this study is that the layout reconfiguration is not considered. As it is explained in the first section, the layout reconfiguration occurs only once in this problem formulation, and for the moment, no thought has been given to how to incorporate it into the Markov decision process described in this chapter. Besides, neural networks for deep learning can be combined with the Q-learning based approximate solution approach in this study, so that the learning experience for different MC production problems in a certain RMS can be accumulated to improve the intelligence of decision-making.

GENERAL CONCLUSION

MC has become a reality and cannot be neglected as one of the leading strategies in satisfying customers and assuring companies survival in today's markets [START_REF] Daaboul | Design for mass customization: Product variety vs. process variety[END_REF]. To facilitate the increased diversity of customized products and low cost of mass production, RMS is greatly recommended as the most suitable manufacturing system to MC. The production planning of MC is inherently complex because the operations that express the different characteristics of multiple mass customization products vary and result in a very large number of different operation sequences by permutations and combinations. An RMS consists of multiple reconfigurable manufacturing tools performing different operations to achieve variable functionality. For MC, this further complicates the decision-making in process planning for performing operations. RMS is relatively a new manufacturing mode that forms suitable production capacity and functions through the restructuring of manufacturing systems and can respond to changes of market needs quickly. Moreover, facility layout problems have always been significant content and key technology in terms of researches on manufacturing systems and reconfigurable layout strategies are analyzed based on variations of products, quantity, and order lead time [START_REF] Al-Zubaidi | Analysis of drivers for solving facility layout problems : A Literature review[END_REF]. The reconfigurability of RMSs provides flexibility for the production of small-lot, multi-variety, mass customized products without losing the economies of scale and efficiency of mass production. In spite of that, the frequent machine and layout reconfigurations take time and make MC production more complicated.

A systematic method for rapid reconfiguration of RMS includes multi-dimensional optimization variables and constraints [START_REF] Leng | Digital twin-driven rapid reconfiguration of the automated manufacturing system via an open architecture model[END_REF]. By knowing all the requirements of the masscustomized products, how to achieve MC in an RMS is an important research problem. To solve this research problem, variables about the processing of all parts that composing masscustomized products and about the machine positions of reconfigured layout in RMS should be determined. In response to the low-cost nature of mass customization, the desire of manufacturers to deliver products on time to improve customer satisfaction, and the requirements of sustainable manufacturing, objectives and constraints orient time, cost and some environmental indicators are generally set in the mathematical model in this research field.

Based on the above, the research in this thesis presented the multi-objective mathematical model that simultaneously considers the process planning and scheduling for MC as well as layout reconfiguration of RMS, in order to get an optimal production plan.

Generative process planning, where all of the plan details, including the operations and the open-field layout. Then a bi-objective nonlinear programming mathematical model minimizing the total tardiness penalty of multiple multi-unit mass-customized products and the total cost covering the processing and setup, WIP transportation and holding, machine and layout reconfiguration activities are introduced with some assumptions. This model is evaluated by a small numerical example using the exhaustive search to find exact Pareto-optimal solutions.

As this exact solution approach is not practical for any scales of the problem, two heuristics, NSGA-II and AMOSA, are adopted, incorporating the proposed constraint handling techniques to deal with the infeasible solutions. In addition, the set of the archive to save the final approximate Pareto-optimal solutions in NSGA-II is set to be unconstrained, in contrast to AMOSA, whose archive has a fixed size. The results of the numerical experiments showed that the proposed constraint handling techniques are effective and the improved NSGA-II is efficient.

This conclusion is obtained by using several metrics to evaluate the performance of these two approximate solution approaches with/without the constraint handling techniques.

Due to the undeniable importance of sustainable development, chapter III introduced a further study by considering environment sustainability in the formulated problem described in Chapter II. A new objective is added to minimize a value measuring the hazardous waste discharge and GHG emission. The general Pareto efficiency is modified to obtain a reasonable number of solutions obtained by the exact and approximate solution approaches. NSGA-III is adopted with the proposed constraint handling techniques and the modified Pareto efficiency to validate the new three-objective model.

Since a main limit of the use of heuristics is their performance dependence on their parameters tuning requiring time and expertise, chapter IV introduced a further study by using a reinforcement learning method, Q-learning, to solve the formulated problem described in Chapter II. This approach copes with this identified limit of heuristics. However, layout reconfiguration is not considered. The proposed Q-value update and upgrade techniques as well as the idea to evaluate two designed policies for selecting actions is novel and problem-specific.

The results of the numerical experiments showed that the Q-learning based approximate solution approach performs better than the improved NSGA-II with the proposed constraint handling techniques.

Main contributions

The main contributions of the research presented in this thesis are:



Research limitations and future work directions

One of the obvious limitations is using numerical examples to validate the proposals.

The values of parameters are not from real industrial manufacturers or any case in other studies. This is because the mathematical models in this thesis are completely novel. And unfortunately, there is no opportunity to have real applications for validation. However, conducting numerical experiments is not always a disadvantage. Since by the simulation, there is no loss caused by mistakes in the proposals. Principally, in conformity with that the improved solution approaches perform well for the numerical examples generated randomly, they are credible to also perform well for real instances.

This work still has a lot to develop in the future. From the perspective of mathematical model, stochastic mathematical models are solvable to the research problem of MC production planning in RMS. With technologies such as digital twin, cloud computing and internet of things escalating the realization of the cyber-physical production system for industry 4.0, a realtime mathematical model for integrating planning and scheduling by considering reconfiguration in an RMS is possible as any information on production progress and about RMS becomes easy to access. This mathematical model relaxed some assumptions, like machine breakdowns in the RMS and defects of mass-customized products. In fact, the aspect of quality can be embedded in a reconfigurable process plan, so that the evaluation of cost, time, and responsiveness can have meaningful insights [START_REF] Khan | An analysis of the theoretical and implementation aspects of process planning in a reconfigurable manufacturing system[END_REF].

From the perspective of solution approach, reinforcement learning is suitable for the research problem of this work as planning and scheduling are described as finite Markov decision process in Chapter IV. As machine learning and big data technologies have gained increased traction by being adopted in some critical areas of planning and control on the strength of that more data and computation power became available [START_REF] Morariu | Machine learning for predictive scheduling and resource allocation in large scale manufacturing systems[END_REF], other solution approaches, for example, deep reinforcement learning that combines deep learning and reinforcement learning, can be adopted to generate optimal production plans for MC in the reconfigurable manufacturing environment. For the moment, the proposed Q-learning approach do not integrate layout optimization, nor the environmental performance as an objective. Therefore, further work is necessary to investigate how to adapt the Q-Learning approach to include them Since there are too many values of independent decision variables in the approximate Pareto-optimal solutions in this large numerical example to build the table above, all of them are available in the repository Sini-GAO/Data-for-Sini-s-manuscript-to-JIM (github.com). This solution is the same to the first solution presented in the above Run 1.

This solution is the same to the third solution presented in the above Run 6. NSGA-II with the constraint handling techniques introduced in Chapter II : 𝑝 40, 𝑝 0.1, 𝑔 1000 (Run 1), (Run 2) and (Run 3) obtained all ten exact Pareto-optimal solutions which are the same to those first ten solutions presented in Table II

1
This solution is the same to the third exact Pareto-optimal solution presented in Table II These three solutions are the same to the first three exact Pareto-optimal solutions presented in Table II These two solutions are the same to the first and the third exact Pareto-optimal solutions presented in Table II-1 in Chapter II.

3-4

These three solutions are the same to the ninth and the tenth exact Pareto-optimal solutions presented in Table II

1
This solution is the same to the third exact Pareto-optimal solution presented in Table II-1 in Chapter II.

2-4

These three solutions are the same to the fifth, seventh and eighth exact Paretooptimal solutions presented in Table II-1 in Chapter II. [START_REF] Epureanu | An agile production network enabled by reconfigurable manufacturing systems[END_REF] This solution is the same to the forth solution presented in the above Run 1.

(Run 9)

1-2 These two solutions are the same to the sixth and eighth exact Pareto-optimal solutions presented in Table II-1 in Chapter II. 3-4 These two solutions are the same to the last two solutions presented in the above Run 1.

(Run 10)

1-2 These two solutions are the same to the second and forth exact Pareto-optimal solutions presented in Table II This solution is the same to the forth solution presented in the above Run 1.

2 .Figure 1

 21 Figure 1 Summary of all chapters presented in this research. ......................................................

Figure I- 1

 1 Figure I-1 Illustration of the concept of part family [190]. ...................................................... Figure I-2 Mass-customized products decomposition. ............................................................ Figure I-3 Schematic of RMS research perspectives [44]. .....................................................

Figure II- 1

 1 Figure II-1 Schematic diagram of jobs. .................................................................................... Figure II-2 Schematic diagram of operations in jobs. .............................................................. Figure II-3 Open-field machine layout in a rectangular workshop. ......................................... Figure II-4 Machine layout reconfiguration. ............................................................................ Figure II-5 Input and output for MC production planning in RMS. ........................................ Figure II-6 Available area to place machine 𝑚. ....................................................................... Figure II-7 Finite feasible positions for machine 𝑚. ................................................................ Figure II-8 Schematic diagram of the layout solution. ............................................................. Figure II-9 Schematic diagram of a very small example. ........................................................ Figure II-10 Schematic diagram of a scheduling example for two operations on a machine. .Figure II-11 Strategy to determine the beginning time of operations. ..................................... Figure II-12 Schematic diagram of Pareto front and a Pareto-optimal solution. ..................... Figure II-13 The main Non-dominated procedure in exhaustive search. ................................. Figure II-14 The main procedures of NSGA-II used in this study. .......................................... Figure II-15 The crossover operator in the adopted NSGA-II. ................................................ Figure II-16 The mutation operator in the adopted NSGA-II. ................................................. Figure II-17 The main procedures of AMOSA used in this study. .......................................... Figure II-18 Constraint handling procedure. ............................................................................ Figure II-19 Mass-customized products and part variants in this small numerical example. .. Figure II-20 The initial layout in this small numerical example. ............................................. Figure II-21 Machine (Machine 1 and Machine 2) reconfiguration time and cost. ................. Figure II-22 Processing and setup parameters of each part variant in small numerical example. .................................................................................................................................................. Figure II-23 Main effects plot for computation time of NSGA-II and AMOSA. .................... Figure II-24 Main effects plot for metric 𝑄𝑀 of NSGA-II and AMOSA. ............................... Figure II-25 Main effects plot for metric 𝑀𝐼𝐷 of NSGA-II and AMOSA. ............................. Figure II-26 Main effects plot for metric 𝐷𝑀 of NSGA-II and AMOSA. ............................... Figure II-27 Main effects plot for metric 𝑁𝑃𝑆 of NSGA-II and AMOSA .............................. Figure II-28 Mass-customized products and part variants in this large numerical example. ... Figure II-29 The initial layout in this large numerical example. .............................................Figure II-30 Machine 3's reconfiguration time and cost. ......................................................... Figure II-31 Processing and setup parameters of each part variant in large numerical example. ................................................................................................................................................

Figure II- 11

 11 Figure II-1 Schematic diagram of jobs. .................................................................................... Figure II-2 Schematic diagram of operations in jobs. .............................................................. Figure II-3 Open-field machine layout in a rectangular workshop. ......................................... Figure II-4 Machine layout reconfiguration. ............................................................................ Figure II-5 Input and output for MC production planning in RMS. ........................................ Figure II-6 Available area to place machine 𝑚. ....................................................................... Figure II-7 Finite feasible positions for machine 𝑚. ................................................................ Figure II-8 Schematic diagram of the layout solution. ............................................................. Figure II-9 Schematic diagram of a very small example. ........................................................ Figure II-10 Schematic diagram of a scheduling example for two operations on a machine. .Figure II-11 Strategy to determine the beginning time of operations. ..................................... Figure II-12 Schematic diagram of Pareto front and a Pareto-optimal solution. ..................... Figure II-13 The main Non-dominated procedure in exhaustive search. ................................. Figure II-14 The main procedures of NSGA-II used in this study. .......................................... Figure II-15 The crossover operator in the adopted NSGA-II. ................................................ Figure II-16 The mutation operator in the adopted NSGA-II. ................................................. Figure II-17 The main procedures of AMOSA used in this study. .......................................... Figure II-18 Constraint handling procedure. ............................................................................ Figure II-19 Mass-customized products and part variants in this small numerical example. .. Figure II-20 The initial layout in this small numerical example. ............................................. Figure II-21 Machine (Machine 1 and Machine 2) reconfiguration time and cost. ................. Figure II-22 Processing and setup parameters of each part variant in small numerical example. .................................................................................................................................................. Figure II-23 Main effects plot for computation time of NSGA-II and AMOSA. .................... Figure II-24 Main effects plot for metric 𝑄𝑀 of NSGA-II and AMOSA. ............................... Figure II-25 Main effects plot for metric 𝑀𝐼𝐷 of NSGA-II and AMOSA. ............................. Figure II-26 Main effects plot for metric 𝐷𝑀 of NSGA-II and AMOSA. ............................... Figure II-27 Main effects plot for metric 𝑁𝑃𝑆 of NSGA-II and AMOSA .............................. Figure II-28 Mass-customized products and part variants in this large numerical example. ... Figure II-29 The initial layout in this large numerical example. .............................................Figure II-30 Machine 3's reconfiguration time and cost. ......................................................... Figure II-31 Processing and setup parameters of each part variant in large numerical example. ................................................................................................................................................

Figure II- 30

 30 Figure II-1 Schematic diagram of jobs. .................................................................................... Figure II-2 Schematic diagram of operations in jobs. .............................................................. Figure II-3 Open-field machine layout in a rectangular workshop. ......................................... Figure II-4 Machine layout reconfiguration. ............................................................................ Figure II-5 Input and output for MC production planning in RMS. ........................................ Figure II-6 Available area to place machine 𝑚. ....................................................................... Figure II-7 Finite feasible positions for machine 𝑚. ................................................................ Figure II-8 Schematic diagram of the layout solution. ............................................................. Figure II-9 Schematic diagram of a very small example. ........................................................ Figure II-10 Schematic diagram of a scheduling example for two operations on a machine. .Figure II-11 Strategy to determine the beginning time of operations. ..................................... Figure II-12 Schematic diagram of Pareto front and a Pareto-optimal solution. ..................... Figure II-13 The main Non-dominated procedure in exhaustive search. ................................. Figure II-14 The main procedures of NSGA-II used in this study. .......................................... Figure II-15 The crossover operator in the adopted NSGA-II. ................................................ Figure II-16 The mutation operator in the adopted NSGA-II. ................................................. Figure II-17 The main procedures of AMOSA used in this study. .......................................... Figure II-18 Constraint handling procedure. ............................................................................ Figure II-19 Mass-customized products and part variants in this small numerical example. .. Figure II-20 The initial layout in this small numerical example. ............................................. Figure II-21 Machine (Machine 1 and Machine 2) reconfiguration time and cost. ................. Figure II-22 Processing and setup parameters of each part variant in small numerical example. .................................................................................................................................................. Figure II-23 Main effects plot for computation time of NSGA-II and AMOSA. .................... Figure II-24 Main effects plot for metric 𝑄𝑀 of NSGA-II and AMOSA. ............................... Figure II-25 Main effects plot for metric 𝑀𝐼𝐷 of NSGA-II and AMOSA. ............................. Figure II-26 Main effects plot for metric 𝐷𝑀 of NSGA-II and AMOSA. ............................... Figure II-27 Main effects plot for metric 𝑁𝑃𝑆 of NSGA-II and AMOSA .............................. Figure II-28 Mass-customized products and part variants in this large numerical example. ... Figure II-29 The initial layout in this large numerical example. .............................................Figure II-30 Machine 3's reconfiguration time and cost. ......................................................... Figure II-31 Processing and setup parameters of each part variant in large numerical example. ................................................................................................................................................

Figure III- 1

 1 Figure III-1 The hazardous waste discharge and GHG emission formulated in this study. .. Figure III-2 Modified Pareto efficiency. ................................................................................ Figure III-3 The main procedures of NSGA-III used in this study. .......................................

Figure IV- 1

 1 Figure IV-1 The basic principle of reinforcement learning ................................................... Figure IV-2 The main procedures of Q-learning. .................................................................. Figure IV-3 Markov decision process of process planning and scheduling for MC in an RMS. ................................................................................................................................................ Figure IV-4. The flowchart of the solution approach. ............................................................

Figure 2

 2 Figure 2 Research structure of basic and further studies. ......................................................

1 .

 1 Customization (flexibility limited to part family): System or machine flexibility limited to a single product family, thereby obtaining customized flexibility. 2. Convertibility (design for functionality changes): The ability to easily transform the functionality of existing systems and machines to suit new production requirements.

Figure I- 1

 1 Figure I-1 Illustration of the concept of part family [190]. Each part variant follows an operation precedence graph as depicted in Figure I-2 (𝑂𝑝 represents Operation 1, and so on; the operation at an arrow front must be performed before the operation at an arrow end). All the operations in an operation precedence graph must be performed when a part belonging to this part variant is processed. There is at least one operation in an operation precedence graph. An operation indicates a typical process to create a certain

Figure I- 2

 2 Figure I-2 Mass-customized products decomposition. More operations and less precedence relationships among operations in an operation precedence graph increase the number of possible operation sequences to process a part belonging to the corresponding part variant. For example, the operation precedence graph of Part variant 2 in Figure I-2 only has four operations and two simple precedence constraints that Operation 2 and Operation 3 should be performed before Operation 4. For every part belonging to this part variant, there are eight possible operation sequences to choose, as follows: 1. 𝑂𝑝 →𝑂𝑝 →𝑂𝑝 →𝑂𝑝 ; 2. 𝑂𝑝 →𝑂𝑝 →𝑂𝑝 →𝑂𝑝 ; 3. 𝑂𝑝 →𝑂𝑝 →𝑂𝑝 →𝑂𝑝 ; 4. 𝑂𝑝 →𝑂𝑝 →𝑂𝑝 →𝑂𝑝 ; 5. 𝑂𝑝 →𝑂𝑝 →𝑂𝑝 →𝑂𝑝 ; 6. 𝑂𝑝 →𝑂𝑝 →𝑂𝑝 →𝑂𝑝 ; 7. 𝑂𝑝 →𝑂𝑝 →𝑂𝑝 →𝑂𝑝 ; 8. 𝑂𝑝 →𝑂𝑝 →𝑂𝑝 →𝑂𝑝 .

  competitiveness by saving energy, reducing resource consumption and reducing pollution, while ensuring on-time delivery. Customer expectations of distinct delivery times and individual attitudes toward delays are important for MC production, but these are related to the scheduling problem. Here comes the literature review for research on scheduling of RMS in the following content.

  Naderi and Azab generalized the scheduling problem of a reconfigurable assembly system and proposed an encoding scheme, hypermutation, and local search mechanisms in regard to two different artificial immune algorithms [247]. Fan et al. integrated the auxiliary modules selection into a FJSP problem with machine reconfigurations to minimize the total weighted tardiness and tested it with a real-world case [248]. Few researchers investigated RMS scheduling for flow line. Considering the close coupling between configuration generation and scheduling of multiple products for reconfigurable flow line, Dou, Li, and Su first established a new multi-objective mixed integer programming model to minimize total cost (including capital cost and reconfiguration cost) and

  assignment, and allows a freer browsing of the search space. W. Zhang, Zheng, and Ahmad considered an integrated process planning and scheduling problem for remanufacturing systems incorporating parallel disassembly workstations, a flexible job-shop-type reprocessing shop, and parallel reassembly workstations by determining the allocation/sequence of end-of-life products on the disassembly/reassembly shops and making decisions on the process path selection, operation sequencing, workstation allocation, and selection for reprocessing jobs[START_REF] Zhang | The integrated process planning and scheduling of flexible job-shop-type remanufacturing systems using improved artificial bee colony algorithm[END_REF].Li et al. considered dynamic cell formation and hybrid production with job and flow shop in the objective function for production planning in RMS and presented a scheduling algorithm of production planning in the virtual cell of RMS using GA for instable batches[START_REF] Li | Production planning in virtual cell of reconfiguration manufacturing system using genetic algorithm[END_REF]. J. Yang et al. proposed a reconfigurable assembly system model and constructed three individual cost functions corresponding to the workload balance, reconfiguration/assembly cost, and lead time to determine the assembly sequence and the deployment of assembly stations[START_REF] Yang | Multiple-objective optimization of a reconfigurable assembly system via equipment selection and sequence planning[END_REF]. Due to the outbreak of the COVID-19 pandemic, the manufacturing sector has been experiencing unprecedented issues, including severe fluctuation in demand, restrictions on the availability and utilization of the workforce, and governmental regulations. Vahedi-Nouri et al. investigated a joint workforce planning and production scheduling problem during the COVID-19 pandemic by leveraging the adaptability and flexibility of an RMS[START_REF] Vahedi-Nouri | Workforce planning and production scheduling in a reconfigurable manufacturing system facing the COVID-19 pandemic[END_REF].In a nutshell, the majority of research on scheduling in RMS considers machine reconfiguration in job-shop, since an RMS is designed for processing a particular family of part variants following different operation precedence graphs. In an RMS, there are alternative pairs of machines and configurations to perform every operation. Decisions for process planning of each part (determining the operation sequence to process parts and selecting machineconfiguration pairs to perform operations in the determined operation sequences) is highly interconnected and interdependent on the scheduling. Therefore, many studies integrated process planning with scheduling for the operation management in RMS. As foregoing, layout can affect scheduling optimization and machine layout in RMS is more flexible, thus layout reconfiguration should also be considered in research on scheduling in RMS.

  et al. proposed a revised electromagnetism-like mechanism for the layout design of RMSs utilizing automated guided vehicle (AGV). The allocation relation of workstation to site is decided, so as to minimize the total material handling cost, including the AGV travel cost and the workstation reconfiguration cost [265]. Azevedo et al. studied the reconfigurable multi-facility layout problem, in which the location of departments (within a group of facilities & inside each facility itself) is optimized to minimize the costs of material handling inside facilities and between facilities. They also studied the relayout to minimize the "unsuitability" of department positions and locations and to maximize the adjacency between departments [266]. Haddou-Benderbal and Benyoucef addressed the product family evolution for the RMS design under performance metrics and optimized the best machine layout, respecting both the constraints imposed by the generated process plans and those depicting the available location in the shop floor where machines can be placed [267]. Gašpar et al. presented the design of passive Stewart platforms suitable for constructing a

Figure I- 3

 3 Figure I-3 Schematic of RMS research perspectives [44]. The outline for optimization in this research field is a standard three-step research process: build mathematical models→develop solution approaches→validate proposed model and solution. This work adopted this standard research process. Each of the following three chapters (Chapter II, Chapter III, and Chapter IV) starts by introducing the developed or updated mathematical model, then describes the proposed or used solution approaches, and finally presents the numerical experiments for validating the proposed model and solution

Figure II- 1

 1 Figure II-1 Schematic diagram of jobs. Process planning and scheduling arrange parts and their operations performed among machines with configurations in an RMS. WIP transportation time and cost in a job are not only determined by the distance between two machines, but also by the type of the part variant the corresponding part belonging to. As different products probably have multiple identical parts, or even one product may have parts belonging to the same part variant, it is necessary to identify their processing jobs with Identifiers (IDs) during manufacturing. A job ID is labeled by the corresponding product (the part decomposed from), the corresponding part variant (the part belonging to) and the number (the part indicated by a positive integer no greater than the number of parts belonging to the corresponding part variant from the corresponding product; parts and integers are randomly mapped in order to distinguish identical parts belonging to the same part variant from a product), as shown in Figure II-1. All jobs can be done in this RMS with certain machines and configurations, of which their reconfigurations are determined in the production

Figure II- 2

 2 Figure II-2 Schematic diagram of operations in jobs. Factories of the future link all operations performance parameters to generate the optimal production plan. Orchestrated operations are the foundation for making the planning and scheduling clear. One operation appears only once in the operation sequence of a part. To distinguish identical operations used in the production of different parts (especially those from the same product and belonging to the same part variant), they are identified with a job ID, as shown in Figure II-2. Parts belonging to the same part variant follow the same operation precedence graph. If two operations follow the same operation precedence graph and for processing parts from the same part variant, the parameters of setting up and performing these

Figure II- 3

 3 Figure II-3 Open-field machine layout in a rectangular workshop. Each machine has an initial position. Machines can change their initial positions to reconfigure a new layout for a specific MC production task. When machines are in the reconfigured layout and start MC production, their positions cannot be changed. The optimal layout is influenced by the process planning and scheduling results, which in turn, are influenced by the changed layout. Whatever the new machine layout becomes, machine blocks must not overlap each other. Machines can be placed anywhere in the workshop (The position of the machine is flexible in terms of X and Y coordinates). The position of the machine can only be changed by translation, nor rotation, as shown in Figure II-4:

Figure II- 5 ,

 5 the mass-customized ordered products are first gathered and decomposed into parts to generate a job list. The production information about the involved part variants and the job list from manufacturers, the initial state and reconfiguration information about machines and layout from RMS, and individual requirements from customers are necessary inputs to determine the optimal MC production planning. To be more specific, the new RMS machine layout, the processing sequence for each job, and the selected machine and configuration together with the start time to perform every process are outputs in this optimization problem.

Figure II- 5

 5 Figure II-5 Input and output for MC production planning in RMS.

5 ) 7 ) 8 ) 9 )

 5789 Only machine reconfiguration and layout reconfiguration are considered in this RMS. Meaning, adding new machines, or adding new possible configurations on a machine are not considered. 6) Only WIP transportation and WIP holding are considered in the material handling activities. WIPs are transported separately, and spaces for holding WIPs are sufficient. The raw materials and material handling equipment are always available. No preemptions and breakdowns are considered in this problem. 10) Parts and the final mass-customized products are qualified without any defect during production in RMS.II.3 -Mathematical modelBased on the presence of nonlinear functions in the objectives and constraints, mathematical models can be classified as linear and nonlinear programming; specifically, mathematical models containing nonlinear functions are called nonlinear programming mathematical models. According to the type of decision variables, a mathematical model in which all decision variables are integers is integer programming, and a mathematical model in which some decision variables are integers is mixed integer programming.In this study, a mixed integer nonlinear programming mathematical model is built to minimize the tardiness penalty associated with the mass-customized products' due date and their real completion time, as well as the total cost covering operation setup and processing, WIP transportation and holding, and machine and layout reconfiguration. It is a new mathematical model integrating process planning, scheduling and layout optimization in RMS for MC. Notations See below for the notation of indices, parameters and decision variables in the mathematical model in this study. II.3.1.1 Indices Notations for indices in all mathematical models are as follows.

𝐼 1 , 2 ,

 12 … , 𝑖, … , 𝑖 , … |𝐼| , Set of mass-customized products. 𝐷 , Due date of the mass-customized product 𝑖. 𝑊 , Penalty cost of the mass-customized product 𝑖 for tardiness per time unit. 𝐽 , , Number of parts belonging to part variant 𝑣 in mass-customized product 𝑖. Notations for the parameters of the open-field workshop are as follows. 𝐵𝑋, Length of the open-field workshop on the X coordinate. 𝐵𝑌, Length of the open-field workshop on the Y coordinate. Notations the for parameters of machines and configurations are as follows. 𝑀 1, 2, … , 𝑚, … , 𝑚 , … |𝑀| , Set of machines. 𝑋 , Initial position of machine 𝑚 on the X coordinate. 𝑌 , Initial position of machine 𝑚 on the Y coordinate. 𝑆𝑋 , Security distance of machine 𝑚 on the X coordinate. 𝑆𝑌 , Security distance of machine 𝑚 on the Y coordinate. 𝐿𝐶 , Layout cost of moving machine 𝑚 per distance unit. 𝐺 1, 2, … , 𝑔, … , 𝑔 , … |𝐺 | , Set of configurations on machine 𝑚. 𝑅𝑇 , , , Machine reconfiguration time from configuration 𝑔 to configuration 𝑔 on machine 𝑚. 𝑅𝐶 , , , Machine reconfiguration cost from configuration 𝑔 to configuration 𝑔 on machine 𝑚. II.3.1.3 Decision variables The decision variables below are divided into independent decision variables and auxiliary decision variables. The values of each independent decision variable are taken in the range of a domain. The values of auxiliary decision variables are unique, by calculating the values of the independent decision variables. The purpose of setting auxiliary decision variables is to facilitate the expression of the objective functions and some constraints.

  Position of machine 𝑚 on the X coordinate 𝑦 Position of machine 𝑚 on the Y coordinate 𝜌 , , , The 𝑞 th operation in the operation sequence of job 𝑗 for part variant 𝑣 in product 𝑖 𝛼 , , , Machine to perform operation 𝜌 , , , 𝜑 , , , Configuration on machine 𝛼 , , , to perform operation 𝜌 , , , 𝛽 , , , Beginning time to process operation 𝜌 , , , ∆𝑥 , Distance between machine 𝑚 and machine 𝑚 on the X coordinate ∆𝑦 , Distance between machine 𝑚 and machine 𝑚 on the Y coordinate 𝑐 , , , Completion time of operation 𝜌 , , , 𝑇 Tardiness of product 𝑖 B , Beginning time set for processes from the 𝑙 th ordinal position on machine 𝑚 𝑖 , Product of the 𝑙 th process on machine 𝑚 𝑣 , Part variant of the 𝑙 th process on machine 𝑚 𝑗 , Job of the 𝑙 th process on machine 𝑚 𝑞 , Ordinal position of the 𝑙 th process on machine 𝑚 in the processing sequence of job 𝑗 , 𝜌 , Operation of the 𝑙 th process on machine 𝑚 𝛽 , Beginning time to process operation 𝜌 , 𝜑 , Configuration to process operation 𝜌 ,II.3.2.1 The first objective function on tardiness penaltyThis objective function is to minimize the total penalty cost for delayed masscustomized products as presented in equation II-1. The tardiness time of each product is weighted by the corresponding penalty cost coefficient. Compared with minimizing the makespan or the total tardiness time of all the mass-customized products, the fact that customers differ in their tolerance of waiting time for delays is reflected in this objective function. The joint optimization based on such a consideration facilitates the rational organization of production activities while ensuring customer satisfaction.

Figure II- 7 .

 7 Figure II-7.

Figure II- 7

 7 Figure II-7 Finite feasible positions for machine 𝑚. The exhaustive search first checks all the combinations of all machine' finite positions and then finds all feasible layouts that are subject to all layout-related constraints in the mathematical model, as shown in Figure II-8 (a).Figure II-8 (b) illustrates the overlap of

Figure II- 8 Figure II- 8

 88 Figure II-8 Schematic diagram of the layout solution. For a small-scale example, it is possible to get all feasible operation sequences to process each part variant and all machine with configuration settings to perform each operation in a very short computation time, and save them without taking up too much memory, as shown in Figure II-9. Since the type of each part that makes up every mass-customized product is known, it is possible to get all combinations of parts operation sequences (determined by the

Figure II- 9

 9 Figure II-9 Schematic diagram of a very small example. The exhaustive search performs a permutation work for the processing sequences on every machine and a combination work on the feasible processing sequences that have been permuted for all machines. The complexity of this permutation and combination work increases exponentially with the number of total operations to be processed. Even if there are only two customized products, two parts to process, two part variants, two operations (totally four operations to process two parts), two machine and two configuration illustrated in Figure II-9,

  Figure II-10, if there is no time interval on machine 𝑚, the holding cost will decrease and operation 𝑂𝑝 will complete before due date. The beginning time of operation 𝑂𝑝 in this schematic diagram is definitely not the best.

Figure II- 10

 10 Figure II-10 Schematic diagram of a scheduling example for two operations on a machine. Thereby, it is favorable to begin the operation as soon as a WIP arrives at an idle machine or immediately after the completion of the previous operation (plus machine reconfiguration and setup when these two activities are inevitable) if the WIP arrives at an occupied machine, as shown in Figure II-11.

Figure II- 13

 13 Figure II-13 The main Non-dominated procedure in exhaustive search.

Figure II- 15

 15 Figure II-15 The crossover operator in the adopted NSGA-II.

Figure II- 16

 16 Figure II-16 The mutation operator in the adopted NSGA-II. II.4.2.2 AMOSA

  Pareto-optimal solutions to the constructed mathematical model. The main steps of its implementation in this study are shown in Figure II-17.As for the adapted NSGAII, the approximate Pareto-optimal solutions are collected in a fixed maximum size archive. The constraint handling is conducted after a new solution is generated by the perturbation operator. The procedures for the AMOSA perturbation are the same as for the mutation operator. New solution generation occurs during iterations when the temperature arrives at a new stage. The temperature drops to different stages according to the cooling rate. When it reaches the minimum temperature, the whole procedure to search approximate Pareto-optimal solutions stops.

Figure II- 17

 17 Figure II-17 The main procedures of AMOSA used in this study. II.4.2.3 Constraint handling Some solutions might be infeasible by the crossover and mutation operators in NSGA-II and the stochastic perturbation in AMOSA. Infeasible solutions adversely affect the performance of approximation algorithms. Biedrzycki et al. [311] overview the majority of

Figure II- 18

 18 Figure II-18 Constraint handling procedure.

  Figure II-19 presents the mass-customized products and part variants in the small numerical example. There are two products. Every product has only one part. The operation precedence graph of each part variant is given. For ∀𝑒 ∈ 𝑂𝑃, 𝑂𝑝 represents Operation 𝑒. This figure also presents the parameters about:

Figure II- 19

 19 Figure II-19 Mass-customized products and part variants in this small numerical example.

Figure II- 20

 20 Figure II-20 The initial layout in this small numerical example. There are two configurations on Machine 1and three configurations on Machine 2. The reconfiguration time and cost on each machine are given in Figure II-21.

  Figure II-22 Processing and setup parameters of each part variant in small numerical example. Since all the values of parameters about machines and the workshop are integers, the type of independent decision variables 𝑥 and 𝑦 are altered from continuous variables to integers in the exhaustive search. As a consequence, for this small numerical example, there are 96 possible positions for machine 1 and 60 possible positions for machine 2. The exhaustive search examines every combination of values possible for all independent decision variables. There are 11 exact Pareto-optimal solutions (the last one are illustrated in Figure II-12) obtained by the exhaustive search for this small numerical example, as shown in Table II-1. 𝑓 and 𝑓 represent the first and second objective values respectively. 𝑆 represents the set of solutions.

Figure II- 23

 23 Figure II-23 Main effects plot for computation time of NSGA-II and AMOSA.

Figure II- 24

 24 Figure II-24 shows the main effects plots for the metric 𝑄𝑀 of NSGA-II and AMOSA.

Figure II- 24

 24 Figure II-24 Main effects plot for metric 𝑄𝑀 of NSGA-II and AMOSA.

  Figure II-25 shows the main effects plots for the metric 𝑀𝐼𝐷 of NSGA-II and AMOSA.

Figure II- 25

 25 Figure II-25 Main effects plot for metric 𝑀𝐼𝐷 of NSGA-II and AMOSA.

Figure II- 26

 26 Figure II-26 shows the main effects plots for the metric 𝐷𝑀 of NSGA-II and AMOSA.

  Figure II-26 Main effects plot for metric 𝐷𝑀 of NSGA-II and AMOSA.

Figure II- 27

 27 Figure II-27 shows the main effects plots for NSGA-II and AMOSA performance measured by the metric 𝑁𝑃𝑆. From the first subfigure, increasing population size and reducing mutation probability are positive for NSGA-II to obtain more approximate Pareto-optimalsolutions in a run, while generation limit has no impact on it. Reducing iteration number and increasing cooling rate are good for AMOSA to obtain more approximate Pareto-optimal solutions in a run. But compared to the NSGA-II, no matter how these two parameters are modified, the value of this metric for AMOSA cannot catch up with that for NSGA-II.

Figure II- 27

 27 Figure II-27 Main effects plot for metric 𝑁𝑃𝑆 of NSGA-II and AMOSACombining all the above findings, small mutation probability is more likely to benefit NSGA-II performance, while the population size and generation limit have different performance measured by different metrics, therefore these two parameters require appropriate tuning. Iteration number has limited impact on several aspects of AMOSA, while small cooling rate is definitely favorable to improve AMOSA performance.

  (a) Main effects plot for DM of NSGA-II (b) Main effects plot for DM of AMOSA (a) Main effects plot for NPS of NSGA-II (b) Main effects plot for NPS of AMOSA Large numerical example A large numerical example with ten mass-customized products, five kinds of operations, three part variants, and three machines is used to evaluate the performance of the adopted resolutions. Figure II-28 gives the due date, tardiness penalty of unit time for every product. Three operation precedence graphs for part variants and the corresponding WIP transporting time and cost of unit distance as well as the holding cost of unit time are given in this figure.

Figure II- 28

 28 Figure II-28 Mass-customized products and part variants in this large numerical example. A new machine is added in this large numerical example. Its security distance and the layout reconfiguration cost are given in Figure II-29. These parameters for the other two

Figure II- 29

 29 Figure II-29 The initial layout in this large numerical example. The added machie has two configurations. The relative reconfiguration time and cost on this machine are given in Figure II-30.

Figure II- 30

 30 Figure II-30 Machine 3's reconfiguration time and cost. The part variants in this large numerical example are absolutely different to those in the small numerical examples. Although there are also Operation 1, Operation 2 and Operation 3 that can be processed on the same two machines (Machine 1 and Machine 2), their processing time and cost, setup time and cost have been changed, as shown in Figure II-31.

  repairing infeasible independent decision variables 𝛽 , , , finds the minimum value to start each process. NSGA-II and AMOSA combined with constraint handling techniques are effective to guarantee the viability of production plans. Numerical experiments used two examples in different scales to test the accuracy and other performances of NSGA-II and AMOSA with the constraint handling techniques, which are also compared with the performance of those without the constraint handling techniques.

Figure III- 1

 1 Figure III-1 The hazardous waste discharge and GHG emission formulated in this study.

  indices and the independent decision variables of the mathematical model in this study are the same as those of the mathematical model presented in Chapter II. Notations for added parameters and auxiliary decision variables in this mathematical model are introduced below. III.3.1.1 Parameters 𝐸𝐹 , Energy consumption for transporting a WIP belonging to part variant 𝑣 per distance unit. 𝐸𝑃 , , , , Energy consumption of performing operation 𝑒 for a WIP belonging to part variant 𝑣 on machine 𝑚 with configuration 𝑔. 𝐻𝑊 , , , , Hazardous waste discharged from performing operation 𝑒 for a WIP belonging to part variant 𝑣 on machine 𝑚 with configuration 𝑔. 𝐸𝑆 , , , , Energy consumption of setup operation 𝑒 for a WIP belonging to part variant 𝑣 on machine 𝑚 with configuration 𝑔. 𝐸𝐿 , Energy consumption for moving machine 𝑚 in distance unit. 𝐸𝑅 , , , Energy consumption of reconfiguration from configuration 𝑔 to configuration 𝑔 on machine 𝑚, if 𝑔 𝑔 , 𝐸𝑅 , , 0.

Figure III- 2

 2 Figure III-2 Modified Pareto efficiency.

Figure III- 3

 3 Figure III-3 The main procedures of NSGA-III used in this study.The performance of NSGA-III is assessed by effectivity and efficiency. Based on the fact that this is a three-objective minimization problem, and most of the approximate Paretooptimal solutions are dominated by the exact Pareto-optimal solutions, or at best equal to the exact Pareto-optimal solutions, a metric 𝑒𝑓𝑓 to assess the effectivity of NSGA-III for the numerical example is defined as the following equation:

  efficacy of the modified Pareto efficiency and the effectivity and efficiency of NSGA-III. As depicted in Figure III-4, there are two products in this small numerical example, each composed of an individual part belonging to a particular part variant. The due date and the tardiness penalty per time unit for each product are given in this figure. In addition, parameters about WIP transport time (𝐹𝑇 ), transport cost (𝐹𝐶 ), transport energy consumption (𝐸𝐹 ), and holding cost belonging to each part variant (𝐻𝐶 ) are displayed in this figure.

Figure III- 4

 4 Figure III-4 Mass-customized products and part variants in this small numerical example.The average hazardous waste and GHG emissions limit per time unit (𝑈𝐻𝑊 and 𝑈𝐺𝐸) are designated to 1 and 7. The factor to appraise the GHG emissions by the amount of energy

Figure III- 5

 5 Figure III-5 The initial layout in this small numerical example.

(a) Machine 1 '

 1 Figure III-6 Machine (Machine 1 and Machine 2) reconfiguration time and cost. Parameters about processing time (𝑃𝑇 , , , ), processing cost (𝑃𝐶 , , , ), processing energy consumption (𝐸𝑃 , , , ), hazardous waste (𝐻𝑊 , , , ), setup time (𝑆𝑇 , , , ), setup cost (𝑆𝐶 , , , ), and setup energy consumption (𝐸𝑆 , , , ) of performing an operation 𝑒 on a certain machine 𝑚 with a certain configuration 𝑔 to processing a part belonging to part variant 𝑣 are presented in Figure III-7.

  Figure III-7 Processing and setup parameters of each part variant in small numerical example.

123

  Compared with the exact Pareto-optimal solutions obtained by the exhaustive search with the modified Pareto efficiency to the tri-objective mathematical model concerning hazardous waste discharge and GHG emissions, there are four solutions eliminated by the environmental indicators. Thus, considering environmental sustainability has a positive impact on the optimization of mass-customized production management in RMS. The modified Pareto efficiency is advanced in gathering economically profitable and environmentally friendly solutions. However, the computation time for exhaustive search with the modified Pareto efficiency to obtain the exact Pareto-optimal solutions to this small numerical example is 2093.28125 seconds, which is longer than the computation used by the approximate solution approach. In order to verify this assertion, the performance of the introduced approximate solution approach, NSGA-III with constraint handling techniques and unconstrained archive is tested in the numerical experiments described in the following context.

The

  Figure III-8 Pareto charts of standardized effects about effectivity and efficiency. The main effects plots in Figure III-9 reveal that the number of the reference points and the mutation probability at a high level are obviously beneficial to improve the effectivity, the number of the reference points and the generation limit at a low level are evidently beneficial to improve the efficiency of this approach to solve the problem in this study.

( a )

 a Figure III-9 Main effects plot about effectivity and efficiency. The contributions of this study are the updated mathematical model with ecological concern and the modified Pareto efficiency. Numerical experiments using NSGA-III to solve a large numerical example is omitted since all contributions has been validated in the numerical experiment solving a small numerical example. The mathematical model is validated by the

  Since there are three objectives in this mathematical model, the general Pareto efficiency gives rise to more Pareto-optimal solutions in Pareto front and increases the probability for both exact solution approaches and approximate solution approaches to obtain more exact and approximate Pareto-optimal solutions, which leads to overchoice for decision makers to determine the final MC production plan implemented in RMS. This study modified the general Pareto efficiency by sensibly devising the Pareto-optimal solutions sorting principle in two steps by the economic and environmental indicators converted from the proposed multiobjectives. This constitutes the second contribution of the research work presented in this thesis.Results in the numerical experiments show that the modified Pareto efficiency retains the Pareto-optimal solutions conforming to the bi-objective mathematical model in the previous study and screens out those better performing with the added objective in this study.The focus of the second study is considering environmental sustainability for MC in RMS. Therefore we don't need to change both the mathematical model and solution approaches in a study. NSGA-III is an advanced version of NSGA-II especially when the number of objectives is more than two. We have compared several methods (NSGA-II and AMOSA) in the first study. As NSGA-II performs well in the previous study, this study adopted NSGA-III to solve the formulated tri-objective mathematical model. Results in the numerical experiments

Figure IV- 2

 2 Figure IV-2 The main procedures of Q-learning. There is a reward to evaluate the action and the Q-value of it is updated usually by the Bellman equation, as shown in formula IV-1. In this equation, 𝑄 𝑠 , 𝑎 represents the current Q-value of the chosen action 𝑎 in the current state 𝑠 . 𝛼 is a parameter called learning rate,

Figure IV- 3

 3 Figure IV-3 Markov decision process of process planning and scheduling for MC in an RMS. As shown in Figure IV-3, this study employs a central agent for the decision-making described above. At any moment when a machine in the RMS finishes an operation and becomes idle, the agent should make an action for this idle machine based on the current state.

2 Constraints

 2 This study has nearly the same bi-objective mixed integer nonlinear programming mathematical model introduced in Chapter II. The indices of the mathematical model in this study are the same as those of the mathematical model presented in Chapter II. The major difference is that machine layout cannot be reconfigured in this study, thus parameters related to the open-field workshop's width and length (𝐵𝑋 and 𝐵𝑌), the security distance of any machines 𝑚 ( 𝑆𝑋 and 𝑆𝑌 ) and the layout reconfiguration cost per distance ( 𝐿𝐶 ) are removed from the objectives and constraints, as well as the decision variables related to the reconfigured machine layout (𝑥 and 𝑦 ). All the others described in notations subsection in Chapter II are kept in this study. For the modification above, there is no layout reconfiguration cost in the second objective function. The modified objective function is expressed as below. The modified second objective function without layout reconfiguration cost There are two objectives in this mathematical model, whose first objective function is the same of those in the previous two studies minimizing the total tardiness penalty. The second objective function in this study is the total cost including the total machine reconfiguration cost (𝑀𝑅𝐶), the total setup cost (𝑇𝑆𝐶), the total processing cost (𝑇𝑃𝐶), the total WIP transportation cost (𝑊𝐹𝐶 ), and the total WIP holding cost (𝑊𝐻𝐶 ), which are the same to those in the mathematical model introduced in Chapter II, whereas the reconfiguration cost of machine layout (𝐿𝑅𝐶) is not considered in this following modified second objective function: 𝑀𝑖𝑛 𝑀𝑅𝐶 𝑇𝑆𝐶 𝑇𝑃𝐶 𝑊𝐹𝐶 𝑊𝐻𝐶 IV-Constraints in this mathematical model are almost the same to those introduced in Chapter II, except those related to layout reconfiguration. The definition of layout reconfiguration cost (II-4), the formula to ensure no machine overlapping (II-16), as well as the domain of the independent decision variables on reconfigured machine positions (II-23) are removed from the 'Constraints' subsection in Chapter II. All the other constraints are kept in this study.

  the completion time (𝑐 , , , ) of the last operation just completed before any operation (𝑒) to be selected in the operation sequence of the corresponding part identified by part variant 𝑣, job ID 𝑗, and the determined mass-customized product 𝑖 * are given. And considering the state of machine 𝑚 * , the beginning time (𝛽 * , , , ) of any operation to be performed on the determined machine 𝑚 * with configuration 𝑔 is calculated by the following expression:𝛽 * ,, , 𝑚𝑎𝑥 𝑐 , , , 𝐹𝑇 |𝑥 * 𝑥 | |𝑦 * 𝑦 | , 𝑐𝑡 * 𝑆𝑇 , , * , , 𝑐𝑡 * 0 0, 𝑐𝑡 * 0, 𝑣 𝑣 * ∧ 𝑒 𝑒 * ∧ 𝑔 𝑔 * 𝑅𝑇 * , * , 𝑆𝑇 , , * , , 𝑐𝑡 * 0, 𝑣 𝑣 * ∨ 𝑒 𝑒 * ∨ 𝑔 𝑔 * IV-7

  (the old Q-values 1, 0 and the new Q-values 1, 0 ) or (the old Q-values 1, 0 and the new Q-values 0.5, 0.5 ) or (the old Q-values 0.5, 0.5 and the new Q-values 1, 0 ): The updated Q-values 1, 0 ; If (the old Q-values 0, 1 and the new Q-values 0, 1 ) or (the old Q-values 0, 1 and the new Q-values 0.5, 0.5 ) or (the old Q-values 0.5, 0.5 and the new Q-values 0, 1 ): The updated Q-values 0, 1 ; Else:

Figure IV- 4 .

 4 Figure IV-4. The flowchart of the solution approach.

  This study uses the same large numerical example in Chapter II. Parameters of masscustomized products, machines initial layout in RMS and their reconfiguration as well as part variants processing and setup are shown in Figure II-28, Figure II-29, Figure II-30 and Figure II-31 respectively. The performance of the proposed Q-learning based approximate solution approach is still compared with that of the improved NSGA-II with the constraint handling techniques introduced in Chapter II. There are two levels of the parameter settings for the improved NSGA-II with the constraint handling techniques, of which the one having largeparameter values run three times to obtain three groups of the approximate Pareto-optimal solutions to the large numerical example. Another one with tuned parameters run ten times to obtain ten set of approximate Pareto-optimal solutions to the large numerical example. The Qlearning based approximate solution approach also run ten times. Four metrics, Quality Metric (𝑄𝑀), Mean Ideal Distance (𝑀𝐼𝐷), Diversification Metric (𝐷𝑀), Number of Pareto-optimal Solutions (𝑁𝑃𝑆) introduced in Chapter II as well as the mean value of two objectives (𝑓 and 𝑓 ) and of the computation time (𝑡 ̅ ) are adopted to evaluate the performance of these solution approaches.

  one of the two objective functions in the bi-objective mathematical model at each assignment.Similar to the reward in the Bellman equation (formula IV-1), the updated Q-values are immediately obtained when the state of an assignment is given. The upgraded Q-values refer to the former Q-values, that is similar to the current Q-value in Bellman equation. Besides, the upgrade procedure takes two objective values into consideration after all the assignments are finished, that is similar to the learning rate parameter (𝛼) in Bellman equation adjusting the Qvalue of a state with considering the final results influenced by other states. Therefore, even though the Q-learning based reinforcement learning method introduced in this study does not use the Bellman equation, the idea of learning is unchanged.The results of the numerical experiments for the examples introduced in Chapter IIshowed that the Q-learning based approximate solution approach introduced in this chapter performs better than the improved NSGA-II with the constraint handling techniques when the scale of the problem changes. Without modifying any parameter values (like 𝜖 that determines the probability of randomly selecting actions), it consumes smaller computation time to obtain Pareto-optimal solutions with smaller objective values, which matches the minimization biobjective optimization model in this study. As the new solutions are not generated by changing some values of the independent decision variables, e.g. crossover and mutation operators, there are no infeasible solutions during searching. Thus, the constraint handling techniques are not necessary to be embedded in this solution approach.



  The joint optimization mathematical model for process planning, scheduling and layout for MC in RMS: the research presented in this thesis fills the research gap of concurrent optimization of process planning, scheduling and layout for MC in RMS, which is a real challenge when manufacturers want to exploit the reconfigurability of RMS to implement MC. The proposed constraint handling technique to repair the infeasible solutions created by the heuristics: Handling infeasible solutions in heuristics is quite a challenge and impacts their performance. For scheduling problems, repairing infeasible solutions is complex. The proposed method repairs efficiently infeasible solutions. This method is universal to any job-shop scheduling problem in which operation precedence are considered in a manufacturing system with several machines. The modified Pareto efficiency: it is problem-specific for the mathematical model concerning MC production in an RMS with sustainable environmental concern. But it also provides a new insight on how to balance many indicators in multi-objective optimization so as to avoid decision-making difficulties. The proposed Q-learning based reinforcement learning method: the proposed method is innovative for the optimization of MC production planning in RMS and cleverly balances the conflict of multi-objectives in optimization. Its good performance shows that machine learning has great potential to solve this operation research problem.

  two solutions are the same to those presented in the above Run 1. (Run 8) All two solutions are the same to those presented in the above Run 1. (Run 9) All three solutions are the same to those presented in the above Run 6.(Run 10) 1

Table I -

 I 1. Summary of selected articles referred in the literature review. ............................... Table II-1. Exact Pareto-optimal solutions for the small numerical example. ......................... Table II-2. Solution quality metrics of NSGA-II and AMOSA. .............................................. Table II-3. DOE for the small numerical experiment. ............................................................. Table II-4. Mean value of performance metrics in the numerical experiment. ...................... Table III-1. The comparison of the introduced study in Chapter II with that in this chapter. Table

  -1 sums up the problems, objectives, models and solution approaches of the referred papers in this literature review. All 54 publications are journal articles from four databases (ScienceDirect, IEEE Xplore, SpringerLink, and Taylor and Francis Online). They were selected because their contents are perfectly aligned with the research themes of planning, scheduling and layout optimization of RMS. and it is more appropriate in the case that no unidirectional flow movement might be feasible[START_REF] Li | Production planning in virtual cell of reconfiguration manufacturing system using genetic algorithm[END_REF]. However, there is only one article considering the open-field layout. This research tries to fill these two identified gaps by concurrently optimizing the process planning, job-shop scheduling and open-field layout in an RMS for an MC production task.For the solution approaches, it is without surprise that more than half of the studies did not use exact solution approaches to obtain the exact Pareto-optimal solutions to the examples during proposal validation. The main reason is that most of these optimization problems are NP-hard. Besides, some mathematical models are nonlinear. Most research directly used approximate solution approaches in the numerical experiments or case studies for validation.In most cases there are problem-specific modifications in these approximate optimization algorithms. Evolutionary algorithms (EAs) are the most popular solution approaches to solve the problems in this research field. There are 33 articles, that is 61.11% (calculated by33 54

	When it comes to the objectives, 39 articles, that is 72.22% (calculated by 39 54
	0.722222) of the references used cost-oriented objective functions, 24 of which used time-
	oriented objective functions, that is 44.44% (calculated by 24 54 0.444444 ) of the
	references, and 15 of which used both cost and time oriented objective functions, that is 31.25%
	(calculated by 15 48 0.3125) of the references that concern these two common economic
	indicators. Other objectives such as throughput, reconfiguration effort (e.g. changeovers in tool
	For the mathematical model, 41 articles, nearly 75.93% (calculated by 41 54
	0.759259) of the references have integer decision variables in their mathematical models. To
	select configurations and/or machines in the planning problem, integer decision variables must
	be adopted in the mathematical model. To determine the exact processing time of operations,
	mixed integer programming is widely adopted to solve the scheduling related problem. In the
	light of most layout involved studies treating this problem as RMS machine allocation, the

From Table

I

-1, there are 37 articles, nearly 68.52% (calculated by 37 54 0.685185) of the references examining the planning optimization of RMS, 3 of which integrate the layout problem, that is nearly 8.11% (calculated by 3 37 0.081081 ) of the references which examine the planning optimization of RMS, and another 6 integrating the scheduling problem, that is nearly 16.22% (calculated by 6 37 0.162162) of the references which examine the planning optimization of RMS. 18 articles, nearly 33.33% (calculated by 18 54 0.333333) of the references studied the scheduling optimization of RMS, 1 of which is integrated with the layout problem, that is nearly 5.56% (calculated by 1 18 0.055556) of the references which studied the scheduling optimization of RMS. 9 articles, nearly 16.67% (calculated by 9 54 0.166667) of the references studied the layout optimization of RMS. From above, there is an obvious research gap: no study simultaneously considers planning, scheduling and layout optimization of RMS, especially for MC. However, based on the discussion in section I.2, it is possible and significant to entail the layout reconfiguration in a short planning horizon to generate an optimal MC production schedule. Layout reconfiguration is available when the concept of RMS is proposed. The integrated planning and scheduling optimization has been widely proven beneficial. A major reason is because the planning problem ensures that the production quantities assigned to all periods are fulfilled on time by solving a detailed scheduling problem for every time period

[START_REF] Chu | Integrated planning and scheduling under production uncertainties: Bi-level model formulation and hybrid solution method[END_REF]

. It is also beneficial to integrate the layout optimization, since the reconfigured layout impacts both planning and scheduling decisions, as well as the makespan and the overall cost. An open field layout is more suited for large part families simultaneously producing different parts and continuously launching new parts onto the stream, and configuration), energy consumption and designed criterions related to environmental sustainability are occasionally considered in optimization. As "mass" in the concept of MC emphasizes producing customized products at the low cost close to that of MP, cost reduction is a key objective in developing production plans for MC. In addition to setup cost, production cost and material handling cost, reconfiguration cost is newly added to the mathematical models for production optimization in RMS. Similar to the above, time is another key objective for MC production optimization, as on-time delivery is important to maintain customer satisfaction.

Accordingly, this research conducted multi-objective optimization for MC production planning in RMS by considering cost and time oriented objective functions in the mathematical models.

In recent years, there has been an increasing number of RMS operation research concerning sustainable manufacturing, with close to one-sixth of the studies in this literature review have objectives of minimizing hazardous wastes and GHG emissions. To echo the call for sustainability, environmental indicators are also considered in the derived mathematical model. presence of integer decision variables in the mathematical model is inevitable. This research also built mixed integer mathematical models as the machine and configuration selection as well as sequences are determined by integer decision variables, while the beginning time to perform each operation and machine positions of the open-field layout are continuous decision variables. 0.611111) of the references using EA to solve the problem in this literature review. This research used exact solution approaches to solve the small-scale numerical examples to validate the mathematical model. The approximate solution approaches are also adopted to solve the large-scale numerical examples for which the exact solution approaches cannot obtain the optimal solutions in an acceptable computation time. EAs are first adopted and improved by problem-specific constraint handling techniques.

  Based on the literature review presented in this chapter and the research gaps highlighted above, a bi-objective mathematical model for simultaneously optimizing process planning, scheduling and RMS layout for producing multiple multi-unit mass-customized products is first introduced in Chapter II. Then an environment sustainability objective function added in the formulated problem is introduced in Chapter III. In order to fundamentally change the performance of the heuristics used in Chapter II and Chapter III for the research problem, an innovative approximate solution approach derived from reinforcement learning is introduced in Chapter IV. All the studies introduced in this thesis really fill the research gaps summarized in this literature review. Table I-1. Summary of selected articles referred in the literature review.

	Authors [214] [215] [216] [234] [217] [233]	Problems Planning Scheduling Layout √ √ √ √ √	Objectives Time √ √ maximize throughput and Cost √ √ √ √ √	Model IDEF0 ILP 0-1 NLP 0-1 NLP 0-1 NLP	Exact PG oriented approach	Solution approach Approximate GA and TS A GA based approach A GA based approach NSGA-II & NSGA-III & WGA & RWGA & TOPSIS
	[204]	√	√	minimize total buffer	MILP		NSGA-II
				capacity
	[205]	√		√	0-1 LP	Gurobi
	[208]	√		√	MINLP		A GA-based approach
	[209] [210] minimize the energy [267] √ maximize expected benifit √ √ √	ILP NLP	A self-proposed method	A two-phase-based approach combining AMOSA and an exhaustive search-based heuristic
	[268]		√	A defined criterion function	NLP	NLP solver fmincon in MATLAB	Artive set algorithm & SQP & interior point algorithm
	[270]		√	√	NLP		CGA
							cumulative
	distribution function, A2C advantage actor-critic, IG iterated greedy, QAP quadratic assignment problem, REM revised electromagnetism-like
	mechanism, SQP sequential quadratic programming, MATLAB (a software for programming and numeric computing), CGA chaotic genetic
	algorithm.

TS tabu search, ILP integer linear programming, PG precedence graph, NLP nonlinear programming, TOPSIS technique for order performance by similarity to ideal solution, MILP mixed integer linear programming, GAMS General algebraic modeling system (a software for mathematical optimization), WGA weighted genetic algorithm, RSUPP repetitive single-unit process plan, LSSUPP local search on single-unit process plan, ABILS archive-based iterated local search, MOPSO multi-objective particle swarm optimization, CPLEX (an optimization software package), MBFA modified brute-force algorithm, SPEA-II strength Pareto evolutionary algorithm, RWGA random weighted genetic algorithm, Gurobi (a commercial optimization solver for linear programming), MINLP mixed integer nonlinear programming, LR Lagrangian relaxation, SNAM social network analysis based method, RIM reference ideal method, EO equilibrium optimizer, AIA artificial immune algorithm, LINGO (a software program used for solving simultaneous linear and nonlinear equations and inequalities), BM Brownian motion, GCDF Gaussian (All abbreviations in this table are explained in the order they appear from top to bottom, left to right).

  Transportation time for moving a WIP of part variant 𝑣 per distance unit. 𝐹𝐶 , Transportation cost for moving a WIP of part variant 𝑣 per distance unit. 𝐻𝐶 , Holding cost for holding a WIP of part variant 𝑣 per time unit.

	II.3.1.2 Parameters
	Notations for parameters of part variants are as follows.
	𝑉		1, 2, … , 𝑣, … , 𝑣 , … |𝑉| , Set of part variants.
	𝑉𝑃		𝑒, … , 𝑒 , Set of operations to process part variant 𝑣.
	𝐾 ,		… , 𝑒 , … , Set of operations precedent to operation 𝑒 when processing part
	variant 𝑣.
	𝐹𝑇 , Notations for parameters of operations are as follows.
	𝑂𝑃	1, 2, … , 𝑒, … , 𝑒 , … |𝑂𝑃| , Set of operations.
	𝑀𝐺 , ,	1 0	𝑖𝑓 𝑜𝑝𝑒𝑟𝑎𝑡𝑖𝑜𝑛 𝑒 𝑖𝑠 𝑓𝑒𝑎𝑠𝑖𝑏𝑙𝑒 𝑜𝑛 𝑚𝑎𝑐ℎ𝑖𝑛𝑒 𝑚 𝑤𝑖𝑡ℎ 𝑐𝑜𝑛𝑓𝑖𝑔𝑢𝑟𝑎𝑡𝑖𝑜𝑛 𝑔 𝑖𝑓 𝑛𝑜𝑡
	𝑃𝑇
	𝑖, 𝑖 𝑣, 𝑣	Indices of mass-customized products Indices of part variants
	𝑗, 𝑗 𝑞, 𝑞 𝑙, 𝑙	Indices of jobs Indices of ordinal positions in the processing sequence of jobs Indices of ordinal positions in the processing sequence on machines
	Indices of operations 𝑚, 𝑚 Indices of machines 𝑒, 𝑒
	𝑔, 𝑔	Indices of configurations

, , , , Processing time of operation 𝑒 for part variant 𝑣 on machine 𝑚 with configuration 𝑔. 𝑃𝐶 , , , , Processing cost of operation 𝑒 for part variant 𝑣 on machine 𝑚 with configuration 𝑔. 𝑆𝑇 , , , , Setup time of operation 𝑒 for part variant 𝑣 on machine 𝑚 with configuration 𝑔. 𝑆𝐶 , , , , Setup cost of operation 𝑒 for part variant 𝑣 on machine 𝑚 with configuration 𝑔.

  The total machines reconfiguration cost is calculated by equation II-5. Machine reconfiguration occurs when two consecutive processes should be performed with different configurations. If two consecutive processes on one machine are performed with the same configuration, the values of the reconfiguration time and reconfiguration cost are equal to 0.

	𝑀𝑅𝐶 ∑	| |	∑ 𝑅𝐶 , , , ,	II-5
	Total setup cost
	𝑇𝑆𝐶 ∑	| |	𝑆𝐶 , , , , , ,	∑ 𝑆𝐶 , , , , , ,
				∀𝑣 ,	𝑣 , ∨ ∀𝜌 ,	𝜌 , ∨ ∀𝜑 ,	𝜑 ,

  3.1 Auxiliary decision variables definitionsThe setup and machine reconfiguration depend on operations' processing sequence on machines and the configurations to perform them, which are implicitly identified by the independent decision variables 𝜌 , , , , 𝛼 , , , , 𝜑 , , , and 𝛽 , , , . To define auxiliary decision variables 𝑖 , , 𝑣 , , 𝑗 , , and 𝑞 , related to the first operation 𝜌 , performed on machines and the corresponding configuration 𝜑 , to perform it, constraint II-12 collects the 𝛽 , , , values of all processes that can be performed on each machine into a set B , .The equations in constraint II-13 express that the first operations performed on machines have the minimum values of the beginning time in every set B , . By knowing the indices of the first operation, the corresponding operation, the selected configuration and the beginning time to perform it are determined via independent decision variables.The equations in constraint II-15 define all the remaining auxiliary decision variables of 𝑖 , , 𝑣 , , 𝑗 , , 𝑞 , , 𝜌 , , 𝜑 , and 𝛽 , (except those of the first operations) in the processing sequence on machines. Since it is impossible for each machine to perform multiple operations at the same time, the first equation in this constraint is a bi-objective function, but not linear.Constraint II-16 ensures no machine overlapping in the reconfigured layout. The distance between any two machines on the X or Y coordinates is not less than the sum of the security distances of these two machines.II.3.3.3 Constraints on independent decision variables about process planningConstraint II-17 expresses the relationship between values of the independent decision variables 𝜌 , , , , 𝛼 , , , , and 𝜑 , , , , indicating that each operation is able to be performed by the selected machine and configuration.Constraint II-20 points out that the first operation processed on a machine cannot start before the setup time required is finished. In other words, the beginning time to perform the first operation on each machine (𝛽 , ) should not be earlier than the completion time of its setup activity (𝑆𝑇 , , , , , , ).According to constraint II-21, if for two consecutive operations processed on a machine presenting the same operation with the same configuration for the same part variant, the beginning time of the second operation (𝛽 , ) cannot be earlier than the completion time of the first operation. This completion time of the first operation is equal to its beginning time on this machine (𝛽 , ) added to its corresponding processing time (𝑃𝑇 , , , , , , ).If the two consecutive operations processed on a machine are different, constraint II-22 restricts the start time of the subsequent operation (𝛽 , ) from being earlier than the sum of the completion time of the previous operation ( 𝛽 , 𝑃𝑇 , , , , , , ) and the machine reconfiguration time (𝑅𝑇 , , , , ) and setup time (𝑆𝑇 , , , ,, , ).Constraint II-24 defines the domains of the independent decision variables 𝜌 , , , .Constraint II-25 defines that the independent decision variables 𝑥 , 𝑦 and 𝛽 , , , are 𝜑 , , , ∈ 𝐺 , , , ∀𝑖 ∈ 𝐼, ∀𝑣 ∈ 𝑉, ∀𝑗 ∈ 1, … , 𝐽 , , ∀𝑞 ∈ 1, … , |𝑉𝑃 | II-27 Constraint II-27 defines the domains of the independent decision variables 𝜑 , , , . The mathematical model has three continuous independent decision variables and three integer decision variables, hence it is a mixed integer programming problem. As there are two constraints with the argument of the minimum in this model, which are nonlinear, it is a mixed integer nonlinear programming mathematical model. The verification that the constraints are sufficient but not too restrictive to obtain an optimal solution with certain values for each independent decision variable is essential by solving examples corresponding to this mathematical model. This mathematical model is original and thus due to the lack of benchmark instances and the opportunity to access to realistic parameters from the factories with RMS and implementing MC, numerical examples are used for validation. The solution approaches to solve the numerical examples are presented in the following section.

	𝛽 ,	𝛽 ,	𝑃𝑇 , , , , , ,
		∀𝑚 ∈ 𝑀, ∀𝑙 ∈ 𝑁 , 𝑣 ,	𝑣 , ∧ 𝜌 ,	𝜌 , ∧ 𝜑 ,	𝜑 ,	II-21
	𝑖 , , 𝑣 , , 𝑗 , , 𝑞 ,	𝑎𝑟𝑔𝑚𝑖𝑛 , , ,	𝐵 , ,
	, 1 𝜑 , , , , , , , , 𝜌 𝑚,1 𝜌 𝑖 𝑚,1 ,𝑣 𝑚,1 ,𝑗 𝑚,1 ,𝑞 𝑚,1 𝜑 , 𝑀𝐺 , , , , , , , , , , , 𝛽 , 𝛽 , , , , , , , ∀𝑖 ∈ 𝐼, ∀𝑣 ∈ 𝑉, ∀𝑗 ∈ 1, … , 𝐽 , , ∀𝑞 ∈ 1, … , |𝑉𝑃 |	∨ 𝑚, 𝑚 ∈ 𝑀 II-13 II-17
	𝛽 ,	𝛽 ,	𝑃𝑇 , , , , , , 𝑅𝑇 , , , ,	𝑆𝑇 , , , , , ,
	II.3.3.4 Constraints on independent decision variables on scheduling ∀𝑚 ∈ 𝑀, ∀𝑙 ∈ 𝑁 , 𝑣 , 𝑣 , ∨ 𝜌 , 𝜌 , ∨ 𝜑 ,	𝜑 ,	II-22
	𝛽 , , ,	𝑐 , , ,	𝐹𝑇	∆𝑥 , , , , , , ,	∆𝑦 , , , , , , ,
	𝐵 ,	𝛽 , , ,
	∆𝑥 , 𝑐 , , , ∀𝑚 ∈ 𝑀, ∀𝑙 ∈ 𝑁 , ∀𝛽 , , , |𝛼 , , , |𝑥 𝑥 |, ∆𝑦 , |𝑦 𝛽 , , , 𝑃𝑇 , , , , , , , , , , , , II.3.3.5 Domains of independent decision variables 𝑦 | 𝑚 ∧ 𝛽 , , , 𝑆𝑋 𝑥 𝐵𝑋 𝑆𝑋 , 𝑆𝑌 𝑦 𝐵𝑌 𝑆𝑌 Constraint II-23 specifies that all RMS machines are enclosed within the boundaries of ∨ 𝑚, 𝑚 ∈ 𝑀 II-23 the open-field workshop after layout reconfiguration. 𝛽 𝑖 , , 𝑣 , , 𝑗 , , 𝑞 , 𝑎𝑟𝑔𝑚𝑖𝑛 𝛽 , , , 𝑐 , , , 𝜌 , , , ∈ 𝑉𝑃 𝐵 , , , , , 𝜌 , 𝜌 , , , , , , , , ∀𝑖 ∈ 𝐼, ∀𝑣 ∈ 𝑉, ∀𝑗 ∈ 1, … , 𝐽 , , ∀𝑞 ∈ 1, … , |𝑉𝑃 | II-24
	𝜑 ,	𝜑 , , , , , , , ,
	𝐵 , 𝛽 , 𝑥 , 𝑦 , 𝛽 , , , ∈ 𝑅 𝛽 , , , , , , , 𝛽 , , , ∀𝑚 ∈ 𝑀, ∀𝛽 , , , |𝛼 , , , ∀𝑚 ∈ 𝑀, ∀𝑖 ∈ 𝐼, ∀𝑣 ∈ 𝑉, ∀𝑗 ∈ 1, … , 𝐽 , , ∀𝑞 ∈ 1, … , |𝑉𝑃 | ∨ 𝑚, 𝑚 ∈ 𝑀, , ∀𝑙 ∈ 𝑁 𝑚, ∀𝑖 ∈ 𝐼, ∀𝑣 ∈ 𝑉, ∀𝑗 ∈ 1, … , 𝐽 , , ∀𝑞 ∈ 1, … , |𝑉𝑃 | II-15 II-25
	𝛽 , continuous variables. 𝑆𝑇 , , , , , ,			∨ 𝑚, 𝑚 ∈ 𝑀 II-20 II-12
	Therefore, this mathematical model is nonlinear programming. 𝛼 , , , ∈ 𝑀
	II.3.3.2 Constrains on layout
	∆𝑥 ,	𝑆𝑋	𝑆𝑋 ∨ ∆𝑦 ,	𝑆𝑌	𝑆𝑌	∨ 𝑚, 𝑚 ∈ 𝑀 II-16

∨ 𝑚, 𝑚 ∈ 𝑀 II-10

These two equations presented in II-10 define the auxiliary decision variables of ∆𝑥 , and ∆𝑦 , by calculating the distances between every two machines on the X and Y coordinate, respectively.

∀𝑖 ∈ 𝐼, ∀𝑣 ∈ 𝑉, ∀𝑗 ∈ 1, … , 𝐽 , II-11

Equation II-11 defines the auxiliary decision variable of 𝑐 , , , , which is the completion time of the 𝑞 th operation in the operation sequence of job 𝑗 for part variant 𝑣 in product 𝑖. , , ∀𝑖 ∈ 𝐼, ∀𝑣 ∈ 𝑉, ∀𝑗 ∈

1, … , 𝐽 , , ∀𝑞 ∈ 1, … , |𝑉𝑃 | II-14

As for above, the two constraints in II-14 are used to identify operations and their performing configurations in processing sequences on machines. This constraint first defines the auxiliary decision variables of the reduced sets B , . Every time when 𝛽 , are identified, they are removed from the sets B , .

∀𝑖 ∈ 𝐼, ∀𝑣 ∈ 𝑉, ∀𝑗 ∈ 1, … 𝐽 , , ∀𝑞 ∈ 1, … , |𝑉𝑃 | 1 II-18

Constraint II-18 implies that the time to perform a process cannot be earlier than the sum of the previous process's completion time and the WIP transportation time between these two consecutive processes in the processing sequence of a job, regardless of whether these two consecutive processes have a precedent relationship specified by some operation precedence graph.

∀𝑖 ∈ 𝐼, ∀𝑣 ∈ 𝑉, ∀𝑗 ∈ 1, … 𝐽 , , ∀𝑞 ∈ 1, … , |𝑉𝑃 | 1 , ∀𝜌 , , , ∈ 𝐾 , , , ,

II-19

Constraint II-19 guarantees that an operation beginning time cannot be earlier than the completion of those operations that are precedent to this operation in the corresponding operation precedence graph, such that the processing of each part satisfies the operation sequence requirements of the corresponding part variant. ∀𝑖 ∈ 𝐼, ∀𝑣 ∈ 𝑉, ∀𝑗 ∈ 1, … , 𝐽 , , ∀𝑞 ∈ 1, … , |𝑉𝑃 |

II-26

Constraint II-26 defines the domains of the independent decision variables 𝛼 , , , .

  𝑃 , , →𝑃 , , →𝑃 , , →𝑃 , , ; 10. 𝑃 , , →𝑃 , , →𝑃 , , →𝑃 , , ; 11. 𝑃 , , →𝑃 , , →𝑃 , , →𝑃 , , ; 12. 𝑃 , , →𝑃 , , →𝑃 , , →𝑃 , , .

there are 88 combinations of processing sequence on machines, which are too many to depict in figures here. Just consider that if all operations are performed on machine 𝑚 with configuration 𝑔, there are 12 processing sequences by permutation, as below (𝑃 , , means performing Operation 𝑂𝑝 for the part, which is from Product 𝑖 and belongs to Part variant 𝑣):

1. 𝑃 ,

, →𝑃 , , →𝑃 , , →𝑃 , , ; 2. 𝑃 , , →𝑃 , , →𝑃 , , →𝑃 , , ; 3. 𝑃 , , →𝑃 , , →𝑃 , , →𝑃 , , ; 4. 𝑃 , , →𝑃 , , →𝑃 , , →𝑃 , , ; 5. 𝑃 , , →𝑃 , , →𝑃 , , →𝑃 , , ; 6. 𝑃 , , →𝑃 , , →𝑃 , , →𝑃 , , ; 7. 𝑃 , , →𝑃 , , →𝑃 , , →𝑃 , , ; 8. 𝑃 , , →𝑃 , , →𝑃 , , →𝑃 , , ; 9.

Table II -

 II 1. Exact Pareto-optimal solutions for the small numerical example.In equation II-28 , 𝑛 represents the number of solutions in each run, while 𝑛 represents the number of the exact Pareto-optimal solutions obtained in each run. This equation defines a metric to evaluate what percentage of each run of an approximate solution approach leads to the exact Pareto-optimal solutions. This gives a measure of the accuracy of the NSGA-II and AMOSA in this study.

						𝑂𝑝	𝑀	𝐶	18	𝑂𝑝	𝑀	𝐶	48
						𝑂𝑝	𝑀	𝐶	3	𝑂𝑝	𝑀	𝐶	25
	6 20	90	2 7	8 3	𝑂𝑝	𝑀	𝐶	8	𝑂𝑝	𝑀	𝐶	41
						𝑂𝑝	𝑀	𝐶	18	𝑂𝑝	𝑀	𝐶	48
						𝑂𝑝	𝑀	𝐶	3	𝑂𝑝	𝑀	𝐶	32
	7 20	90	2 7	8 3	𝑂𝑝	𝑀	𝐶	10	𝑂𝑝	𝑀	𝐶	36
						𝑂𝑝	𝑀	𝐶	18	𝑂𝑝	𝑀	𝐶	48
						𝑂𝑝	𝑀	𝐶	3	𝑂𝑝	𝑀	𝐶	25
	8 20	90	2 7	8 3	𝑂𝑝	𝑀	𝐶	10	𝑂𝑝	𝑀	𝐶	41
						𝑂𝑝	𝑀	𝐶	18	𝑂𝑝	𝑀	𝐶	48
						𝑂𝑝	𝑀	𝐶	3	𝑂𝑝	𝑀	𝐶	3
	9	8	91	2 7	8 3	𝑂𝑝	𝑀	𝐶	8	𝑂𝑝	𝑀	𝐶	14
						𝑂𝑝	𝑀	𝐶	18	𝑂𝑝	𝑀	𝐶	32
						𝑂𝑝	𝑀	𝐶	3	𝑂𝑝	𝑀	𝐶	3
	10 8	91	2 7	8 3	𝑂𝑝	𝑀	𝐶	10	𝑂𝑝	𝑀	𝐶	14
						𝑂𝑝	𝑀	𝐶	18	𝑂𝑝	𝑀	𝐶	32
						𝑂𝑝	𝑀	𝐶	3	𝑂𝑝	𝑀	𝐶	10
	11 2	103 8 6	8 3	𝑂𝑝	𝑀	𝐶	8	𝑂𝑝	𝑀	𝐶	22
						𝑂𝑝	𝑀	𝐶	15	𝑂𝑝	𝑀	𝐶	34
				Layout		Process planning and Scheduling
	𝑆	𝑓	𝑓	𝑀	𝑀	𝑖 1, 𝑣 1, 𝑗 1	𝑖 2, 𝑣 2, 𝑗 1
				𝑥 𝑦	𝑥 𝑦	𝜌	𝛼	𝜑	𝛽	𝜌	𝛼	𝜑	𝛽
	1 30	84	2 7	8 3	𝑂𝑝 𝑂𝑝 𝑝	𝑀 𝑀	𝐶 𝐶	3 15	𝑂𝑝 𝑂𝑝	𝑀 𝑀	𝐶 𝐶	10 32 II-28
						𝑂𝑝	𝑀	𝐶	25	𝑂𝑝	𝑀	𝐶	44
						𝑂𝑝	𝑀	𝐶	10	𝑂𝑝	𝑀	𝐶	5
	2 30	84	2 7	8 3	𝑂𝑝	𝑀	𝐶	15	𝑂𝑝	𝑀	𝐶	32
						𝑂𝑝	𝑀	𝐶	25	𝑂𝑝	𝑀	𝐶	44
						𝑂𝑝	𝑀	𝐶	3	𝑂𝑝	𝑀	𝐶	10
	3 30	84	2 7	8 3	𝑂𝑝	𝑀	𝐶	17	𝑂𝑝	𝑀	𝐶	32
						𝑂𝑝	𝑀	𝐶	25	𝑂𝑝	𝑀	𝐶	44
						𝑂𝑝	𝑀	𝐶	10	𝑂𝑝	𝑀	𝐶	5
	4 30	84	2 7	8 3	𝑂𝑝	𝑀	𝐶	17	𝑂𝑝	𝑀	𝐶	32
						𝑂𝑝	𝑀	𝐶	25	𝑂𝑝	𝑀	𝐶	44
	5 20	90	2 7	8 3	𝑂𝑝 𝑂𝑝	𝑀 𝑀	𝐶 𝐶	3 8	𝑂𝑝 𝑂𝑝	𝑀 𝑀	𝐶 𝐶	32 36

The computation time of the exhaustive search for the small numerical example is 2088.92 seconds. By setting the appropriate parameters of the improved NSGA-II and AMOSA that combined with constraint handling techniques and other modifications introduced in the previous section, their computation times (in Table

II-

3) are less by more than 97%. Compared with the exact Pareto-optimal solutions obtained by the exhaustive search, the quality of the approximate Pareto-optimal solutions obtained by NSGA-II and AMOSA is evaluated by defined metrics: (1) accuracy rate (𝑝 ) expressed by equation (II-28), (2) precision rate (𝑝 ) expressed by equation (II-28), (3) deviation of the first objective (𝑑𝑣 ) equation (II-30), and deviation of the second objective (𝑑𝑣 ) equation (II-31).The higher 𝑝 and 𝑝 , and the lower 𝑑𝑣 and 𝑑𝑣 are preferable. 𝑝 II-29 In equation II-28 , 𝑛 represents the total number of the exact Pareto-optimal solutions to the problem. This equation defines a metric to evaluate what percentage of exact

Table II -

 II 2. Solution quality metrics of NSGA-II and AMOSA.

  𝑛 is the number of approximate Pareto-optimal solutions obtained in a run. 𝑓1is the maximum value of the first objective among those solutions, while 𝑓1

	In this equation,
	Methods	Parameters			Run	𝑝	𝑝	𝑑𝑣	𝑑𝑣
							1	0.91 0.91 0.91% 1.18%
	𝑝	40, 𝑝	0.1, 𝑔	1000	2	0.56 0.45 1.28% 2.49%
							3	0.73 0.73 1.36% 0.46%
							1	1	0.91 4.30% 0.78%
	𝑝	80, 𝑝	0.1, 𝑔	1000	2	0.73 0.73 0.91% 1.01%
	NSGA-II						3 1	0.67 0.73 2.99% 4.81% 1 0.73 2.30% 0.67%
	𝑝	40, 𝑝		0.05, 𝑔	1000	2	0.57 0.73 6.21%	12%
							3	0.8	0.73 4.30% 0.44%
							1	0.91 0.91 0.46% 0.15%
	𝑝	40, 𝑝	0.1, 𝑔	2000	2	1	0.91 4.30% 0.78%
							3	0.8	0.73 4.30% 0.67%
							1	0.75 0.27 10.66% 4.96%
		𝑖	100, 𝑐	0.99	2	0.75 0.27 10.66% 5.22%
							3	1	0.36 9.14% 0.10%
							1	0.75 0.27 10.66% 5.15%
	AMOSA	𝑖	200, 𝑐	0.99	2	0.75 0.27 13.84% 4.71%
							3	1	0.27 1.24% 4.78%
							1	0.75 0.27 10.66% 4.39%
		𝑖	100, 𝑐	0.95	2	1	0.27 1.24% 4.78%
							3	1	0.18 2.11% 0.95%
							,	,	II-32

Table II -

 II [START_REF] Tareq | Additive manufacturing and the COVID-19 challenges: An in-depth study[END_REF]. DOE for the small numerical experiment.

	Methods Parameters Run 𝑡/𝑠	𝑄𝑀 𝑀𝐼𝐷 𝐷𝑀 𝑁𝑃𝑆	(𝑓 , 𝑓 )
	𝑝	40,	16.75	1	0.69 48.81 11	(30, 84),
	𝑝	0.1,
	𝑔	1000
	NSGA-II

constraint handling techniques highly improve the performance of NSGA-II and AMOSA.

  First of all, the objectives' mean values of approximate Pareto-optimal solutions obtained by NSGA-II with the constraint handling techniques are obviously smaller than those obtained by NSGA-II without the constraint handling techniques, which is the same as AMOSA. Additionally, both solution approaches obtain more solutions with the constraint handling techniques.

Table II-4, the

The computation time is not significantly reduced by not using constraint handling techniques

  . 𝑀𝐼𝐷 of NSGA-II is better when constraint handling techniques are used, but the opposite is true for AMOSA. Similarly, 𝐷𝑀 of AMOSA is better when constraint handling techniques are used, while the opposite is true for NSGA-II. Therefore, it is hard to say if the constraint handling techniques improve the relative distance of approximate Pareto-optimal solutions, as well as their diversity. Table II-4. Mean value of performance metrics in the numerical experiment. 102 of metrics in the above table are given in appendices Table B. The numerical experiments for this large numerical example validate the superior of proposed constraint handling techniques and improved NSGA-II.

	Methods	Parameters	𝑓	𝑓	𝑡 ̅ /𝑠	𝑄𝑀 𝑀𝐼𝐷	𝐷𝑀	𝑁𝑃𝑆
	NSGA-II
	with the proposed handling constraint	𝑔 𝑝 𝑝	1000 40, 0.1,	4075.64 2616.88 551.34 0.1 0.89 453.93 6.3
	techniques
	NSGA-II
	without the proposed handling constraint	𝑝 𝑔 𝑝	40, 1000 0.1,	12536.92 6660.3 548.78	0	0.47 983.39 1.8
	techniques
	AMOSA with
	the proposed constraint handling	𝑖 𝑐	100, 0.99	13873.53 6116.94 1743.40 0	0.84 4067.24 4.1
	techniques
	AMOSA
	without the proposed constraint	𝑖 𝑐	100, 0.99	18589.95 7947.79 1750.69 0	1	2305.23	2
	handling
	techniques
	On the other hand, NSGA-II always performs better than AMOSA regarding the
	computation time. There is a set of approximate Pareto-optimal solutions obtained by NSGA-
	II with the proposed constraint handling techniques that dominates any other solutions, which

are obtained either from other runs by the same approach, or by NSGA-II without the constraint handling techniques, or by AMOSA with/without the constraint handling techniques. Details of objective values obtained by NSGA-II and AMOSA in each run to calculate the mean value

Table III -

 III 1. The comparison of the introduced study in Chapter II with that in this chapter.

	Problems	Objectives	Approximate
	Process		solution
	planning		approach

Table III -

 III [START_REF] Tareq | Additive manufacturing and the COVID-19 challenges: An in-depth study[END_REF]. The optimal layout in the exact Pareto-optimal solutions for the small numerical

	Solutions First Objective Function Second Objective Function Third Objective Function
	Solution 1	3	70	0.9811320754716981
	Solution 2	10	69	0.8662131519274376
	Solution 3	10	69	0.8662131519274376
	example
	Solutions Machines	Position on the X-Coordinate	Position on the Y-Coordinate
	Solution 1	Machine 1 Machine 2	3 9	3 7
	Solution 2	Machine 1 Machine 2	3 9	3 7
	Solution 3	Machine 1 Machine 2	3 9	3 7

  -5, TableIII-6 and Table III-7. Table III-5. The values of the multi-objective functions in the exact Pareto-optimal solutions for no environmental indicator model Table III-6. The optimal layout in the exact Pareto-optimal solutions for no environmental

	Solutions	Variables Independent Decision	Processes
	indicator model

Table III -

 III [START_REF] Lasi | Industry 4.0[END_REF]. The values of the multi-objective functions in the approximate Pareto-optimal

			Machine 2		8				7
	Solution 2	Machine 1 Machine 2		3 8				3 7
	Solution 3	Machine 1 Machine 2		3 8				3 7
	Table III-10. The optimal production scheme in the approximate Pareto-optimal solutions for
	the small numerical example
	Solutions	Independent Decision Variables	Processes (𝑖, 𝑣, 𝑗, 𝑞) 1, 1, 1, 1 1, 1, 1, 2 1, 1, 1, 3 2, 2, 1, 1 2, 2, 1, 2 2, 2, 1, 3
			𝜌 , , ,		0	1	2	1	0	2
	Solution		𝛼 , , ,		1	1	1	0	0	0
	1		𝜑 , , ,		1	2	0	0	1	0
			𝛽 , , ,		3	27	45	3	13	24
			𝜌 , , ,		0	1	2	0	1	2
	Solution		𝛼 , , ,		0	0	0	0	0	0
	2		𝜑 , , ,		1	0	0	1	0	0
			𝛽 , , ,		5	44	52	13	22	33
			𝜌 , , ,		0	2	1	0	1	2
	Solution		𝛼 , , ,
	3
	solutions for the small numerical example
	Solutions	First Objective Function		Second Objective Function	Third Objective Function
	Solution 1		6		74		1.0242587601078168
	Solution 2		36		65		0.910411622276029
	Solution 3		36		65		0.910411622276029
	Table III-9. The optimal layout in the approximate Pareto-optimal solutions for the small
	numerical example
	Solutions Machines	Position on the X-Coordinate	Position on the Y-Coordinate
	Solution 1 Machine 1		3				3

Table III -

 III [START_REF] Leitold | Flood risk reduction and climate change adaptation of manufacturing firms: Global knowledge gaps and lessons from Ho Chi Minh City[END_REF]. It requires eight runs necessary for a two-level full factorial design with three factors. The number of replicates for the corner points was three. Thus, there were 24 runs altogether in this factor analysis.

	Table III-11. Two levels for each factor in factor analysis
	Solutions	Low Level	High Level
	Number of reference point	6	15
	Mutation probability	0.05	0.1
	Generation limit	1000	2000
	125

  in the next state. 𝑟 𝛾 max 𝑄 𝑠 , 𝑎 𝑄 𝑠 , 𝑎 is the temporal difference of the estimated new Q-value, 𝑟 𝛾 max 𝑄 𝑠 , 𝑎 , and the current Q-value, 𝑄 𝑠 , 𝑎 .

	𝑄	𝑠 , 𝑎 ← 𝑄 𝑠 , 𝑎	𝛼	𝑟 𝛾 𝑚𝑎𝑥 𝑄 𝑠 , 𝑎	𝑄 𝑠 , 𝑎	IV-1

  The pending operations in job 𝑖, 𝑣, 𝑗 are collected in Set 𝑇𝑂_𝐷𝑂 , , . Concerning two objectives of optimizing time and cost respectively, two policies 𝑃 and 𝑃 are designed to find 𝑣 * , 𝑗 * , 𝑒 * , 𝑔 * :

	𝑃 :	𝑣 * , 𝑗 * , 𝑒 * , 𝑔 *	arg min ∀ ∈ ,∀ ∈ * , ,∀ ∈ _	* , , , ∈ *	𝛽 𝑖 * ,𝑣,𝑗,𝑒 𝑃𝑇 𝑒,𝑣,𝑚 * ,𝑔	IV-5
	𝑃 : 𝑣 * , 𝑗 * , 𝑒 * , 𝑔 *	arg min ∀ ∈ ,∀ ∈ * , ,∀ ∈ _	* , , , ∈ *	𝑓 𝑣, 𝑗, 𝑒, 𝑔	𝑊𝐹𝐶 , , * ,
	𝑊𝐻𝐶 , , * ,
		𝑆𝐶 , , * ,	𝑃𝐶 , , * , , 𝑐𝑡 * 0
			𝑃𝐶 , , * , , 𝑐𝑡 * 0, 𝑣 𝑣 * ∧ 𝑒 𝑒 * ∧ 𝑔 𝑔 *
	𝑅𝐶 * , * ,	𝑆𝐶 , , * ,	𝑃𝐶 , ,

* , , 𝑐𝑡 * 0, 𝑣 𝑣 * ∨ 𝑒 𝑒 * ∨ 𝑔 𝑔 *

  These two Q-tables have 𝑛 rows and two columns, for saving the updated and upgraded Q-values of the two policies 𝑃 and 𝑃 at each assignment. An entire 𝑛-assignment procedure is an episode in this study. In each episode, a new Q-table and a solution 𝑠 to the formulated problem are generated. The new Q-table helps to upgrade the old Q-table during iteration. Here comes the pseudocode for upgrading the old In the above, the principles to judge the superiority have been introduced by comparing the growth rate of the time and cost corresponding to the actions selected by each policy. The 𝑣 * , 𝑗 * , 𝑒 * , 𝑔 * is arbitrarily selected from Set 𝐴 if the updated Q-values 1, 0 . If the updated Q-values 0, 1 , the 𝑣 * , 𝑗 * , 𝑒 * , 𝑔 * is arbitrarily selected from Set 𝐵. If the updated Q-values 0.5, 0.5 , the 𝑣 * , 𝑗 * , 𝑒 * , 𝑔 * is arbitrarily selected from Set 𝐴⋃𝐵. The ε-greedy policy is adopted to decide whether performing the above 𝑣

Q-table:

At the end of each episode: * , 𝑗 * , 𝑒 * , 𝑔 * selected principle. A random number 𝑟𝑛 is generated from the unit interval 0, 1 to control the probability of the ϵ-greedy policy occurring.

The flowchart of the proposed Q-learning based approximate solution approach is illustrated in Figure

IV-4

. The old Q-table is initialized by selecting all the 𝑣 * , 𝑗 * , 𝑒 * , 𝑔 * in the first episode using Policy 𝑃 exclusively. The new Q-table is initialized by selecting all the 𝑣 * , 𝑗 * , 𝑒 * , 𝑔 * in the second episode using Policy 𝑃 exclusively. The old Q-table is first

Table IV -

 IV [START_REF] Tareq | Additive manufacturing and the COVID-19 challenges: An in-depth study[END_REF]. Numerical experiments for the large numerical example in this study.

As the metric values in the above table shown, the

objective values of the approximate Pareto-optimal solutions obtained by the proposed Q-learning based approximate solution

	Methods	Parameters	𝑓	𝑓	𝑡 ̅ /𝑠	𝑄𝑀 𝑀𝐼𝐷	𝐷𝑀	𝑁𝑃𝑆
	The proposed
	Q-learning
	based approximate	𝜀 0.8	4958.85 2640.39	9.92	0.19 0.78	1080. 60	5.4
	solution
	approach
	NSGA-II with handling constraint	𝑝 𝑔 𝑝	40, 1000 0.1,	5640.99 3410.34 533.18 0.44 0.79 1310.68 9.67
	techniques introduced in Chapter II	𝑔 𝑝 𝑝	50 10, 0.1,	13685.96 6230.54	7.03	0.1 0.77 2123.01 3.5

approach are generally smaller than those of the approximate Pareto-optimal solutions obtained by the improved NSGA-II with the constraint handling techniques, no matter the level of the parameter setting.

  In fact, all the approximate Pareto-optimal solutions are dominated by some approximate solution obtained by the proposed Q-learning based approximate solution approach. The original metric values of this large numerical experiment are given in Table D in the Appendices. Increasing the population size and the generation limit help to have more approximate Pareto-optimal solutions with smaller objective values, but it is time-consuming. While decreasing the parameter values of the improved NSGA-II with the constraint handling techniques is less time-consuming, but it results in the worst approximate

Table A .

 A Approximate Pareto-optimal solutions for the small numerical example (in chapter II). These ten solutions are the same to the first ten exact Pareto-optimal solutions presented in TableII-1 in Chapter II. These five solutions are the same to the first three, fifth and sixth exact Paretooptimal solutions presented in TableII-1 in Chapter II. These eight solutions are the same to the first eight exact Pareto-optimal solutions presented in TableII-1 in Chapter II. These ten solutions are the same to the first ten exact Pareto-optimal solutions presented in TableII-1 in Chapter II. These eight solutions are the same to the first eight exact Pareto-optimal solutions presented in TableII-1 in Chapter II. These eight solutions are the same to the first eight exact Pareto-optimal solutions presented in TableII-1 in Chapter II. These three solutions are the same to those presented in the above Run 3. These eight solutions are the same to the first eight exact Pareto-optimal solutions presented in TableII-1 in Chapter II. These eight solutions are the same to the first eight exact Pareto-optimal solutions presented in TableII-1 in Chapter II. These eight solutions are the same to the first eight exact Pareto-optimal solutions presented in Table II-1 in Chapter II. These ten solutions are the same to the first ten exact Pareto-optimal solutions presented in TableII-1 in Chapter II. These ten solutions are the same to the first ten exact Pareto-optimal solutions presented in TableII-1 in Chapter II. (Run 3)1-8 These eight solutions are the same to the first eight exact Pareto-optimal solutions presented in TableII-1 in Chapter II. These three solutions are the same to the first, fifth and ninth exact Paretooptimal solutions presented in TableII-1 in Chapter II. These three solutions are the same to the first, fifth and ninth exact Paretooptimal solutions presented in TableII-1 in Chapter II. These four solutions are the same to the first, fifth, ninth and tenth exact Pareto-optimal solutions presented in TableII-1 in Chapter II. These three solutions are the same to the first, fifth and ninth exact Paretooptimal solutions presented in TableII-1 in Chapter II. These three solutions are the same to the first, fifth and ninth exact Paretooptimal solutions presented in TableII-1 in Chapter II. These three solutions are the same to the first, fifth and ninth exact Paretooptimal solutions presented in TableII-1 in Chapter II. These three solutions are the same to the first, fifth and ninth exact Paretooptimal solutions presented in TableII-1 in Chapter II. These three solutions are the same to the first, fifth and ninth exact Paretooptimal solutions presented in Table II-1 in Chapter II. These three solutions are the same to the first and ninth exact Pareto-optimal solutions presented in TableII-1 in Chapter II.

	NSGA-II with constraint handling techniques: 𝑝 40, 𝑝 0.1, 𝑔 1000 (Run1) 𝑆 𝑓 𝑓 Layout Process planning and Scheduling 𝑀 𝑀 𝑖 1, 𝑣 1, 𝑗 1 𝑖 2, 𝑣 2, 𝑗 1 𝑥 𝑦 𝑥 𝑦 𝜌 𝛼 𝜑 𝛽 𝜌 𝛼 𝜑 1-10 11 6 126.5 4 5 8 5 𝑂𝑝 𝑀 𝐶 3 𝑂𝑝 𝑀 𝐶 𝑂𝑝 𝑀 𝐶 8 𝑂𝑝 𝑀 𝐶 𝑂𝑝 𝑀 𝐶 19 𝑂𝑝 𝑀 𝐶 (Run 2) 1-5 6 18 96 5 6 5 6 𝑂𝑝 𝑀 𝐶 4 𝑂𝑝 𝑀 𝐶 𝑂𝑝 𝑀 𝐶 11 𝑂𝑝 𝑀 𝐶 𝑂𝑝 𝑀 𝐶 21 𝑂𝑝 𝑀 𝐶 7 15 96.5 5 6 5 6 𝑂𝑝 𝑀 𝐶 4 𝑂𝑝 𝑀 𝐶 𝑂𝑝 𝑀 𝐶 12 𝑂𝑝 𝑀 𝐶 𝑂𝑝 𝑀 𝐶 20 𝑂𝑝 𝑀 𝐶 8 12 109.2 5 6 5 6 𝑂𝑝 𝑀 𝐶 4 𝑂𝑝 𝑀 𝑂𝑝 𝑀 𝐶 11 𝑂𝑝 𝑀 𝐶 𝑂𝑝 𝑀 𝐶 19 𝑂𝑝 𝑀 𝐶 9 8 110 5 6 5 6 𝑂𝑝 𝑀 𝐶 4 𝑂𝑝 𝑀 𝐶 𝑂𝑝 𝑀 𝐶 11 𝑂𝑝 𝑀 𝐶 𝑂𝑝 𝑀 𝐶 20 𝑂𝑝 𝑀 (Run 3) 1-8 9 8 92 1 7 8 3 𝑂𝑝 𝑀 𝐶 3 𝑂𝑝 𝑀 𝐶 𝑂𝑝 𝑀 𝐶 10 𝑂𝑝 𝑀 𝐶 𝑂𝑝 𝑀 𝐶 18 𝑂𝑝 𝑀 𝐶 10 8 92 1 7 8 3 𝑂𝑝 𝑀 𝐶 3 𝑂𝑝 𝑀 𝐶 𝑂𝑝 𝑀 𝐶 8 𝑂𝑝 𝑀 𝑂𝑝 𝑀 𝐶 18 𝑂𝑝 𝑀 𝐶 11 8 92 2 8 8 3 𝑂𝑝 𝑀 𝐶 3 𝑂𝑝 𝑀 𝐶 𝑂𝑝 𝑀 𝐶 8 𝑂𝑝 𝑀 𝐶 𝑂𝑝 𝑀 𝐶 18 𝑂𝑝 𝑀 𝐶 𝑝 80, 𝑝 0.1, 𝑔 1000 1-10 (Run 2) 14 8 100 4 4 7 4 𝑂𝑝 𝑀 𝐶 8 𝑂𝑝 𝑀 𝐶 𝑂𝑝 𝑀 𝐶 18 𝑂𝑝 𝑀 1-8 (Run 1) (Run 3) 𝑖 200, 𝑐 0.99 𝐶 1-4 (Run 1) 12 8 100 3 3 6 3 𝑂𝑝 𝑀 𝐶 10 𝑂𝑝 𝑀 𝐶 𝑂𝑝 𝑀 𝐶 18 𝑂𝑝 𝑀 13 8 100 3 3 6 3 𝑂𝑝 𝑀 𝐶 3 𝑂𝑝 𝑀 𝐶 𝑂𝑝 𝑀 𝐶 8 𝑂𝑝 𝑀 𝑂𝑝 𝑀 𝐶 18 𝑂𝑝 𝑀 𝑂𝑝 𝑀 𝐶 3 𝑂𝑝 𝑀 𝐶 (Run 3) 𝐶 𝑂𝑝 𝑀 𝐶 19 𝑂𝑝 𝑀 𝐶 𝐶 4 6 131 8 8 8 3 𝑂𝑝 𝑀 𝐶 8 𝑂𝑝 𝑀 𝐶 𝑂𝑝 𝑀 𝐶 3 𝑂𝑝 𝑀 𝐶 𝐶 1-3 𝐶 𝑂𝑝 𝑀 𝐶 3 𝑂𝑝 𝑀 𝐶 (Run 2) 𝑂𝑝 𝑀 𝐶 20 𝑂𝑝 𝑀 𝐶 10 15 95 7 7 8 4 𝑂𝑝 𝑀 𝐶 4 𝑂𝑝 𝑀 𝑂𝑝 𝑀 𝐶 12 𝑂𝑝 𝑀 𝑂𝑝 𝑀 𝐶 20 𝑂𝑝 𝑀 11 15 95 7 7 7 3 𝑂𝑝 𝑀 𝐶 4 𝑂𝑝 𝑀 𝑂𝑝 𝑀 𝐶 12 𝑂𝑝 𝑀 𝑂𝑝 𝑀 𝐶 20 𝑂𝑝 𝑀 𝐶 𝑂𝑝 𝑀 𝐶 19 𝑂𝑝 𝑀 𝐶 𝐶 4 6 129 6 6 8 3 𝑂𝑝 𝑀 𝐶 8 𝑂𝑝 𝑀 𝐶 𝐶 𝑂𝑝 𝑀 𝐶 3 𝑂𝑝 𝑀 𝐶 𝐶 1-3 𝐶 (Run 1) 𝐶 𝑖 100, 𝑐 0.99 𝐶 (Run 2) 1-8 9 15 95 6 6 7 3 𝑂𝑝 𝑀 𝐶 4 𝑂𝑝 𝑀 𝑂𝑝 𝑀 𝐶 12 𝑂𝑝 𝑀 𝐶 𝑂𝑝 𝑀 𝐶 18 𝑂𝑝 𝑀 𝐶 AMOSA with constraint handling techniques: 1-2 𝐶 𝑂𝑝 𝑀 𝐶 18 𝑂𝑝 𝑀 𝐶 10 8 92 3 7 8 3 𝑂𝑝 𝑀 𝐶 3 𝑂𝑝 𝑀 𝐶 𝑂𝑝 𝑀 𝐶 8 𝑂𝑝 𝑀 𝐶 (Run 3) 1-3 𝐶 𝑆 𝑓 Layout Process planning and Scheduling 𝑀 𝑀 𝑖 1, 𝑣 1, 𝑗 1 𝑖 2, 𝑣 2, 𝑗 1 Layout Process planning and Scheduling Methods Parameters Run 𝑡/𝑠 𝑄𝑀 𝑀𝐼𝐷 𝐷𝑀 𝑁𝑃𝑆 1-3 1 1735.63 0 1.00 1586.32 2 𝑓 𝑥 𝑦 𝑥 𝑦 𝜌 𝛼 𝜑 𝛽 𝜌 𝛼 𝜑 1-8 9 8 92 2 6 8 3 𝑂𝑝 𝑀 𝐶 3 𝑂𝑝 𝑀 𝐶 𝑂𝑝 𝑀 𝐶 10 𝑂𝑝 𝑀 𝐶 𝑂𝑝 𝑀 𝐶 18 𝑂𝑝 𝑀 𝐶 10 8 92 2 6 8 3 𝑂𝑝 𝑀 𝐶 3 𝑂𝑝 𝑀 𝐶 𝑂𝑝 𝑀 𝐶 8 𝑂𝑝 𝑀 𝐶 𝑂𝑝 𝑀 𝐶 18 𝑂𝑝 𝑀 𝐶 11 6 121 7 7 6 3 𝑂𝑝 𝑀 𝐶 3 𝑂𝑝 𝑀 𝐶 𝑂𝑝 𝑀 𝐶 8 𝑂𝑝 𝑀 𝐶 𝑂𝑝 𝑀 𝐶 19 𝑂𝑝 𝑀 𝐶 (Run 3) 1-8 9 8 92 2 8 8 3 𝑂𝑝 𝑀 𝐶 3 𝑂𝑝 𝑀 𝐶 𝑂𝑝 𝑀 𝐶 10 𝑂𝑝 𝑀 𝐶 𝑂𝑝 𝑀 𝐶 18 𝑂𝑝 𝑀 𝐶 10-12 𝑝 40, 𝑝 0.05, 𝑔 1000 (Run 1) 1-8 𝑆 𝑓 𝑓 𝑀 𝑀 𝑖 1, 𝑣 1, 𝑗 1 Layout Process planning and Scheduling 2 1759.30 0 0.91 6141.77 4 (16908, 5293), (10935, 6722.9), (13044, 6328.5), (13040, 6359.6) 𝛽 3 5 5 5 3 3 3 3 3 3 3 5 5 3 5 3 (𝑓 , 𝑓 ) (18464, 5450.2), (16880, 5536) 𝛽 3 3 3 3 𝑖 2, 𝑣 2, 𝑗 1 𝑥 𝑦 𝑥 𝑦 𝜌 𝛼 𝜑 𝛽 𝜌 𝛼 𝜑 𝛽 9 8 94 2 4 8 3 𝑂𝑝 𝑀 𝐶 3 𝑂𝑝 𝑀 𝐶 3 𝑂𝑝 𝑀 𝐶 10 𝑂𝑝 𝑀 𝐶 𝑂𝑝 𝑀 𝐶 18 𝑂𝑝 𝑀 𝐶 10 8 94 2 4 8 3 𝑂𝑝 𝑀 𝐶 3 𝑂𝑝 𝑀 𝐶 3 𝑂𝑝 𝑀 𝐶 8 𝑂𝑝 𝑀 𝐶 𝑂𝑝 𝑀 𝐶 18 𝑂𝑝 𝑀 𝐶 𝑝 40, 𝑝 0.1, 𝑔 2000 (Run 1) 1-10 11 4 106 5 6 5 3 𝑂𝑝 𝑀 𝐶 3 𝑂𝑝 𝑀 𝐶 𝑂𝑝 𝑀 𝐶 8 𝑂𝑝 𝑀 𝑆 𝑓 𝑓 𝑀 𝑀 𝑖 1, 𝑣 1, 𝑗 1 𝑖 2, 𝑣 2, 𝑗 1 𝑥 𝑦 𝑥 𝑦 𝜌 𝛼 𝜑 𝛽 𝜌 𝛼 𝜑 𝛽 4 6 130.5 2 7 6 7 𝑂𝑝 𝑀 𝐶 3 𝑂𝑝 𝑀 𝐶 3 𝑂𝑝 𝑀 𝐶 8 𝑂𝑝 𝑀 𝐶 𝑂𝑝 𝑀 𝐶 19 𝑂𝑝 𝑀 3 1737.88 0 0.65 6269.82 (13904, 5617.5), (15150, 5584), (18020, 5547.3), (11823, 6500.1), (13289, 7 6065.3), (13443, 5828.9), (12814, 6155.3) AMOSA 4 1727.97 0 0.85 1213.04 3 (12960, 6248.6), (12437, 7343.1), (12707, 6450.8) with the proposed 𝑖 100, 5 1736.20 0 0.75 5054.64 (16371, 5316.6), (14352, 5861), (14382, 5793.8), (11600, 6986), (12404, 6 6038.6), (11737, 6519.2) 𝐶 (Run 2) 1-3 4 2 127 8 6 8 3 𝑂𝑝 𝑀 𝐶 3 𝑂𝑝 𝑀 constraint 𝑐 0.99 6 1747.31 0 0.70 2527.17 (12344, 5934.7), (13991, 5723.4), (12701, 5761), (11991, 6108.2), (11563, 5 6424.4) handling techniques 7 1758.91 0 0.92 5389.29 (10451, 6683.9), (10367, 6700.8), (11752, 6214.6), (15605, 5432.8), (15024, 5 5771.5) 𝐶 𝑂𝑝 𝑀 𝐶 8 𝑂𝑝 𝑀 8 1745.45 0 0.77 3518.17 4 (10651, 5779.9), (10584, 6189.2), (14047, 5568.6), (13056, 5630.6) 𝐶 𝑂𝑝 𝑀 𝐶 15 𝑂𝑝 𝑀 9 1733.92 0 0.89 6701.85 3 (11837, 6870.8), (18501, 6159.5), (14561, 6544.1) 𝐶 10 1751.45 0 1.00 2270.35 2 (16269, 6346), (13999, 6385.8) (Run 3) 1 1760.31 0 1.00 1127.77 (15860, 7565), (14830, 8024.3) 2 𝐶 𝑂𝑝 𝑀 𝐶 17 𝑂𝑝 𝑀 𝐶 (Run 2) 1-10 9 8 92 3 7 8 3 𝑂𝑝 𝑀 𝐶 3 𝑂𝑝 𝑀 𝐶 3 𝑂𝑝 𝑀 𝐶 10 𝑂𝑝 𝑀 𝐶 1-3 𝑖 200, 𝑐 0.95 (Run 1) 1-3 4 6 124.5 8 7 8 3 𝑂𝑝 𝑀 𝐶 3 𝑂𝑝 𝑀 𝐶 3 𝑂𝑝 𝑀 𝐶 8 𝑂𝑝 𝑀 𝐶 𝑂𝑝 𝑀 𝐶 19 𝑂𝑝 𝑀 2 1747.05 0 1.00 5343.29 2 (13957, 7945.9), (19288, 7583.7) AMOSA 3 1755.22 0 1.00 3059.19 2 (26422, 9002.3), (23382, 9344.4) without 4 1738.73 0 1.00 1014.62 2 (16975, 7242.4), (16022, 7590.6) the 𝑖 100, 5 1765.17 0 1.00 599.24 2 (19956, 8080.6), (20180, 7524.8) proposed 𝑐 0.99 6 1731.59 0 1.00 4673.46 2 (19898, 7507.4), (15228, 7687.2) constraint 7 1745.39 0 1.00 1564.03 2 (14240, 7096.3), (12693, 7326.5) handling 8 1768.78 0 1.00 2107.38 2 (23344, 8718.9), (21237, 8758.9) techniques 9 1733.34 0 1.00 946.33 2 (23392, 8660.6), (22446, 8685.7) 𝐶 (Run 2) 10 1761.34 0 1.00 2617.01 2 (14916, 7309), (17533, 7301.2)

Table C .

 C Approximate Pareto-optimal solutions for the small numerical example (in chapter This solution is the same to the ninth exact Pareto-optimal solution presented in TableII-1 in Chapter II. (Run 2) All two solutions are the same to those presented in the above Run 1. (Run 3) 1 This solution is the same to the first solution presented in the above Run 1. 2 This solution is the same to the tenth exact Pareto-optimal solution presented in Table II-1 in Chapter II. (Run 4) All two solutions are the same to those presented in the above Run 1. (Run 5) All two solutions are the same to those presented in the above Run 3. (Run 6) 1This solution is the same to the first solution presented in the above Run 1.

	IV).
	Q-learning based approximate solution approach:
	𝑝	40, 𝑝	0.1, 𝑔	1000
					(Run1)
					Process planning and Scheduling
	𝑆 𝑓	𝑓	𝑖 1, 𝑣 1, 𝑗 1		𝑖 2, 𝑣 2, 𝑗 1
			𝜌	𝛼	𝜑	𝛽	𝜌	𝛼	𝜑	𝛽
			𝑂𝑝	𝑀	𝐶	4	𝑂𝑝	𝑀	𝐶	5
	1 36	90	𝑂𝑝	𝑀	𝐶	14	𝑂𝑝	𝑀	𝐶	9
			𝑂𝑝	𝑀	𝐶	34	𝑂𝑝	𝑀	𝐶	21
	2
			𝑂𝑝	𝑀	𝐶	4	𝑂𝑝	𝑀	𝐶	5
	2 32	92	𝑂𝑝	𝑀	𝐶	20	𝑂𝑝	𝑀	𝐶	9
			𝑂𝑝	𝑀	𝐶	28	𝑂𝑝	𝑀	𝐶	21
			𝑂𝑝	𝑀	𝐶	3	𝑂𝑝	𝑀	𝐶	2
	3 8	95	𝑂𝑝	𝑀	𝐶	10	𝑂𝑝	𝑀	𝐶	20
			𝑂𝑝	𝑀	𝐶	18	𝑂𝑝	𝑀	𝐶	36
	(Run 7) All

  -1 in Chapter II. NSGA-II with the constraint handling techniques introduced in Chapter II :

					Process planning and Scheduling
	𝑆 𝑓	𝑓	𝑖 1, 𝑣 1, 𝑗 1			𝑖 2, 𝑣 2, 𝑗 1
			𝜌	𝛼	𝜑	𝛽	𝜌	𝛼	𝜑	𝛽
			𝑂𝑝	𝑀	𝐶	4	𝑂𝑝	𝑀	𝐶	1
	3 35	104	𝑂𝑝	𝑀	𝐶	18	𝑂𝑝	𝑀	𝐶
			𝑂𝑝	𝑀	𝐶	26	𝑂𝑝	𝑀	𝐶
			𝑂𝑝	𝑀	𝐶	3	𝑂𝑝	𝑀	𝐶	2
	4 17	126	𝑂𝑝	𝑀	𝐶	10	𝑂𝑝	𝑀	𝐶
			𝑂𝑝	𝑀	𝐶	18	𝑂𝑝	𝑀	𝐶
			𝑂𝑝	𝑀	𝐶	3	𝑂𝑝	𝑀	𝐶	2
	5 17	126	𝑂𝑝	𝑀	𝐶	8	𝑂𝑝	𝑀	𝐶
			𝑂𝑝	𝑀	𝐶	18	𝑂𝑝	𝑀	𝐶
					(Run 2)
	𝑝	10, 𝑝	0.1, 𝑔	50
					(Run 1)
			𝑂𝑝	𝑀	𝐶	3	𝑂𝑝	𝑀	𝐶	7
	1 38	90	𝑂𝑝	𝑀	𝐶	19	𝑂𝑝	𝑀	𝐶	41
			𝑂𝑝	𝑀	𝐶	27	𝑂𝑝	𝑀	𝐶	48
			𝑂𝑝	𝑀	𝐶	3	𝑂𝑝	𝑀	𝐶	7
	2 38	90	𝑂𝑝	𝑀	𝐶	17	𝑂𝑝	𝑀	𝐶	41
			𝑂𝑝	𝑀	𝐶	27	𝑂𝑝	𝑀	𝐶	48
					190

  -1 in Chapter II. These two solutions are the same to the last two solutions presented in the above Run 1. These two solutions are the same to the first two solutions presented in the above Run1. 3 This solution is the same to the third solution presented in the above Run 6.

			𝑂𝑝	𝑀	𝐶	3	𝑂𝑝	𝑀	𝐶
	2 22	110	𝑂𝑝	𝑀	𝐶	15	𝑂𝑝	𝑀	𝐶
			𝑂𝑝	𝑀	𝐶	23	𝑂𝑝	𝑀	𝐶
	3-4 (Run 3)
	1-2
			𝑂𝑝	𝑀	𝐶	3	𝑂𝑝	𝑀	𝐶	2
	4 8	95	𝑂𝑝	𝑀	𝐶	8	𝑂𝑝	𝑀	𝐶
			𝑂𝑝	𝑀	𝐶	18	𝑂𝑝	𝑀	𝐶
					(Run 4)
	1-3

  -1 in Chapter II.

			𝑂𝑝	𝑀	𝐶	4	𝑂𝑝	𝑀	𝐶
	4 26	105	𝑂𝑝	𝑀	𝐶	18	𝑂𝑝	𝑀	𝐶
			𝑂𝑝	𝑀	𝐶	26	𝑂𝑝	𝑀	𝐶
			𝑂𝑝	𝑀	𝐶	3	𝑂𝑝	𝑀	𝐶	3
	5 21	126	𝑂𝑝	𝑀	𝐶	10	𝑂𝑝	𝑀	𝐶
			𝑂𝑝	𝑀	𝐶	18	𝑂𝑝	𝑀	𝐶
			𝑂𝑝	𝑀	𝐶	3	𝑂𝑝	𝑀	𝐶	3
	6 21	126	𝑂𝑝	𝑀	𝐶	8	𝑂𝑝	𝑀	𝐶
			𝑂𝑝	𝑀	𝐶	18	𝑂𝑝	𝑀	𝐶
					(Run 5)
	1-2

  -1 in Chapter II. (Run 6) 1This solution is the same to the first solution presented in the Run 1 by Q-learning based approximate solution approach.2This solution is the same to the SECOND solution presented in the Run 6 by Q-learning based approximate solution approach. These two solutions are the same to the last two solutions presented in the above Run 1. These three solutions are the same to the ninth and the tenth exact Pareto-optimal solutions presented in TableII-1 in Chapter II. (Run 8)

					Process planning and Scheduling
	𝑆 𝑓	𝑓	𝑖 1, 𝑣 1, 𝑗 1			𝑖 2, 𝑣 2, 𝑗 1
			𝜌	𝛼	𝜑	𝛽	𝜌	𝛼	𝜑	𝛽
			𝑂𝑝	𝑀	𝐶	3	𝑂𝑝	𝑀	𝐶	3
	3 22	110	𝑂𝑝	𝑀	𝐶	15	𝑂𝑝	𝑀	𝐶	14
			𝑂𝑝	𝑀	𝐶	23	𝑂𝑝	𝑀	𝐶	26
	4-5 (Run 7)
	1-2

  -1 in Chapter II.

			𝑂𝑝	𝑀	𝐶	3	𝑂𝑝	𝑀	𝐶	7
	3 22	116	𝑂𝑝	𝑀	𝐶	15	𝑂𝑝	𝑀	𝐶	23
			𝑂𝑝	𝑀	𝐶	23	𝑂𝑝	𝑀	𝐶	30
	4

[START_REF] Bensmaine | A non-dominated sorting genetic algorithm based approach for optimal machines selection in reconfigurable manufacturing environment[END_REF] √ √ √ ILP an adapted NSGA-II[220] √ √ ILP & MILP GAMS [222] √ √ NLP NSGA-II & TOPSIS [223] √ minimize changeovers in

[START_REF] Khezri | Towards a sustainable reconfigurable manufacturing system (SRMS): multi-objective based approaches for process plan generation problem[END_REF] √ √ √

[START_REF] Wan | Integrated scheduling and self-reconfiguration for assembly job shop in knowledgeable manufacturing[END_REF] √ √ √ MINLP

√ √ √ √ NSGA-III

√ √ √ √ Q-learning based reinforcement learning

(4) 𝑓 𝑓 and 𝑓 𝑓 and 𝑓 𝑓 ; (5) 𝑓 𝑓 and 𝑓 𝑓 and 𝑓 𝑓 ; (6) 𝑓 𝑓 and 𝑓 𝑓 and 𝑓 𝑓 ; (7) 𝑓 𝑓 and 𝑓 𝑓 and 𝑓 𝑓 ; (8) 𝑓 𝑓 and 𝑓 𝑓 and 𝑓 𝑓 ; (9) 𝑓 𝑓 and 𝑓 𝑓 and 𝑓 𝑓 ; (10) 𝑓 𝑓 and 𝑓 𝑓 and 𝑓 𝑓 ; (11) 𝑓 𝑓 and 𝑓 𝑓 and 𝑓 𝑓 ; (12) 𝑓 𝑓 and 𝑓 𝑓 and 𝑓 𝑓 ; (13) 𝑓 𝑓 and 𝑓 𝑓 and 𝑓 𝑓 ; (14) 𝑓 𝑓 and 𝑓 𝑓 and 𝑓 𝑓 ; (15) 𝑓 𝑓 and 𝑓 𝑓 and 𝑓 𝑓 ; (16) 𝑓 𝑓 and 𝑓 𝑓 and 𝑓 𝑓 ; (17) 𝑓 𝑓 and 𝑓 𝑓 and 𝑓 𝑓 ; (18) 𝑓 𝑓 and 𝑓 𝑓 and 𝑓 𝑓 ; (19) 𝑓 𝑓 and 𝑓 𝑓 and 𝑓 𝑓 ; (20) 𝑓 𝑓 and 𝑓 𝑓 and 𝑓 𝑓 ; (21) 𝑓 𝑓 and 𝑓 𝑓 and 𝑓 𝑓 ; (22) 𝑓 𝑓 and 𝑓 𝑓 and 𝑓 𝑓 ; (23) 𝑓 𝑓 and 𝑓 𝑓 and 𝑓 𝑓 ; (24) 𝑓 𝑓 and 𝑓 𝑓 and 𝑓 𝑓 ; (25) 𝑓 𝑓 and 𝑓 𝑓 and 𝑓 𝑓 ; (26) 𝑓 𝑓 and 𝑓 𝑓 and 𝑓 𝑓 ; (27) 𝑓 𝑓 and 𝑓 𝑓 and 𝑓 𝑓 .

Solution 𝑢 and solution 𝑣 are non-dominated to each other when they match one of the thirteen conditions (3), ( 6), ( 7), ( 8), ( 9), ( 12), ( 14), ( 16), ( 19), ( 20), ( 21), ( 22), (25), i.e., there is close to one in two probability for two solutions to be non-dominated to each other. Therefore, the general Pareto efficiency brings about numerous Pareto-optimal solutions in the Pareto front. Seeing that the number of feasible solutions in the solution space will not change by the sorting methods, a strict Pareto efficiency narrows the Pareto front and decreases the collected Pareto-optimal solutions.

non-dominated to each other when they one of the following expressions:

(1) 𝑓 𝑓 𝑓 𝑓 and 𝑓 𝑓 ;

(2) 𝑓 𝑓 𝑓 𝑓 and 𝑓 𝑓 ;

(3) 𝑓 𝑓 𝑓 𝑓 and 𝑓 𝑓 .

To be more specific, two solutions must match one of the following expressions:

(1) 𝑓 𝑓 and 𝑓 𝑓 and 𝑓 𝑓 ;

(2) 𝑓 𝑓 and 𝑓 𝑓 and 𝑓 𝑓 ;

(3) 𝑓 𝑓 and 𝑓 𝑓 and 𝑓 𝑓 𝑓 𝑓 and 𝑓 𝑓 ; (4) 𝑓 𝑓 and 𝑓 𝑓 and 𝑓 𝑓 𝑓 𝑓 and 𝑓 𝑓 ; (5) 𝑓 𝑓 and 𝑓 𝑓 and 𝑓 𝑓 𝑓 𝑓 and 𝑓 𝑓 ; (6) 𝑓 𝑓 and 𝑓 𝑓 and 𝑓 𝑓 ; (7) 𝑓 𝑓 and 𝑓 𝑓 and 𝑓 𝑓 ; (8) 𝑓 𝑓 and 𝑓 𝑓 and 𝑓 𝑓 ; (9) 𝑓 𝑓 and 𝑓 𝑓 and 𝑓 𝑓 𝑓 𝑓 and 𝑓 𝑓 ; (10) 𝑓 𝑓 and 𝑓 𝑓 and 𝑓 𝑓 𝑓 𝑓 and 𝑓 𝑓 ; (11) 𝑓 𝑓 and 𝑓 𝑓 and 𝑓 𝑓 𝑓 𝑓 and 𝑓 𝑓 ; (12) 𝑓 𝑓 and 𝑓 𝑓 and 𝑓 𝑓 ; (13) 𝑓 𝑓 and 𝑓 𝑓 and 𝑓 𝑓 .

Based on the above, only six of the thirteen conditions for two solutions being nondominated by general Pareto efficiency in a three-objective optimization become tighter through this sorting method, reducing the probability to one third for two solutions to be nondominated to each other. This study proposed a modified Pareto efficiency lowering the probability to less than one in five for two solutions to be non-dominated to each other. As shown in Figure III-2, it is divided into two steps. In the first step, solutions are sorted by the values of the first objective function and the second objective function. In the second step, all the non-dominated solutions from the previous step are further sorted by the economic indicator and the environmental indicator (𝑓 ).

the modified layout and operation sequence as well as the machine with configuration will influence the beginning time to perform an operation. This repair sequence insures that the unfeasible solution requires one iteration to be repaired and be transformed into a feasible one. Second, the independent decision variable 𝜌 , , , for the 𝑞 th process in the processing sequence of job 𝑗 for part variant 𝑣 in product 𝑖, 𝛼 , , , for the selected machine to perform operation 𝜌 , , , , and 𝜑 , , , for the selected configuration on machine 𝛼 , , , to perform operation 𝜌 , , , are respectively checked. If they are infeasible, they are repaired by picking a random value from their domains. The repair assignment will be repeated until the variable becomes feasible. This method is both simple and effective, thanks to the fact that there are not many feasible values in the domains of these independent decision variables.

Finally, the values of the independent decision variables 𝛽 , , , are checked. The main difficulty, is that they are mutually influential. If the start time of a process is not constrained and is modified, the start time of the subsequent processes not only in the corresponding job's processing sequence but also on the performed machine will probably become infeasible and ought to be modified. This creates a chain reaction. Hence, the difficulty in repairing this set of decision variables lies in finding a suitable order to modify the infeasible values. Our study put forward a greedy-based repair method. The pseudocode for this method is as follow:

For ∀𝑖 ∈ 𝐼, ∀𝑣 ∈ 𝑉, ∀𝑗 ∈ 1,2, … , 𝐽 , , ∀𝑞 ∈ 1,2, … , |𝑉𝑃 | :

Set 𝛽 , , , 𝑐 , , , 𝐹𝑇 ∆𝑥 , , , , , , , ∆𝑦 , , , , , , , ; Sort all 𝛽 , , , in ascending order on machines to form the auxiliary decision variables 𝛽 , , save all auxiliary decision variables 𝛽 , in set 𝑃; While 𝑃 ∅: Find the minimum value in 𝑃 and save the corresponding performed machine (𝑚 * ), the ordinal of the processing sequence on the performed machine (𝑙 * ), the performed configuration (𝑔 * ), the processed operation (𝑒 * ), the product (𝑖 * ), the part variant (𝑣 * ), the job (𝑗 * ), and the ordinal position in the processing sequence of the corresponding job (𝑞 * );

Pareto-optimal solutions are obtained in each run of an approximate solution approach. Thus, on the basis of the precision of NSGA-II and AMOSA, there is a measure of the breadth of distribution of the exact Pareto-optimal solutions obtained.

𝑑𝑣 ∑ ∑

∑ ∑

II-30

The numerator of this equation is the difference between the mean value of the first objective values of all approximate Pareto-optimal solutions obtained in each run by an approximate solution approach and the mean value of the first objective values of the exact Pareto-optimal solutions. To normalize this deviation, it is divided by the sum of these two mean values. This metric evaluates the effectiveness of the approximate Pareto-optimal solutions obtained by NSGA-II and AMOSA via the first objective value. The metric for the second objective value follows the same idea in the equation below.

II-31

In order to explore the effect of three parameters, population size (denoted by 𝑝 ), mutation probability (denoted by 𝑝 ), and generation limit (denoted by 𝑔 ) on NSGA-II performance, as well as two parameters, iteration number (denoted by 𝑖 ) and cooling rate (denoted by 𝑐 ) on AMOSA performance, two different values are given to each parameter to set multiple groups for comparison. These two approximate solution approaches with the parameter setting in each group have been run three times. For this small numerical example, most approximate Pareto-optimal solutions obtained by NSGA-II and AMOSA are exact Pareto-optimal solutions, thus there is no need to presented them again. Only those approximate Pareto-optimal solutions, whose values of objective functions and of independent decision variables are different from any exact Pareto-optimal solution, are present in appendices Table B.

Table II instructive to boot parameters tuning.

The major limitation of this study is the absence of a real application for such a problem.

Also, it is important to address the sustainability of RMS. RMS appears to be eco-friendly while coping with rapidly changing market demands, but lacks discussion regarding sustainability or environment-friendly functions within it [START_REF] Kurniadi | Development of Multi-Disciplinary Green-BOM to Maintain Sustainability in Reconfigurable Manufacturing Systems[END_REF]. Therefore, and due to the increasing urgency and importance of sustainable design and implementation, the next chapter introduces a study considering environmental indicators on the basis of this research.

three new operations: 1) normalization of objective vectors and the supplied reference points so as to have both sets within a single range; 2) association of every population member (solution from the combined parent and offspring population) with a particular reference point based on a proximity measure; and 3) niching of accepted population members in order to ensure a diverse set of solutions [START_REF] Jain | An Evolutionary Many-Objective Optimization Algorithm Using Reference-Point Based Nondominated Sorting Approach, Part II: Handling Constraints and Extending to an Adaptive Approach[END_REF]. The total number of the reference points (𝐻) in this three-objective problem is defined by the division parameter for each objective axis (𝑝):

The main procedures of NSGA-III keep the same as those of NSGA-II illustrated in The determination of reference points on a hyper-plane in this study is determining the number of the reference points based on the above equation. Adaptive normalization of population members refers to evenly creating the determined number of reference points on the three-dimensional unit simplex because this study does not have any preference information.

Association of each population member with a reference point is to calculate the perpendicular distance of each population member from each of the reference lines that are defined by joining the reference point with the origin. Niche count for reference points is counting the number of population members that are associated with each reference point because a reference point may have one or more population members.

The reinforcement learning method can be used to solve this finite Markov decision process. A difficulty in this study is that there are two objectives in this research problem. In general cases, there is only one objective, thus the reward suggesting the feedback of an action decided by an agent in the reinforcement learning method is consistent with that objective. This study is a bi-objective optimization as it keeps the majority of the statements in the problem formulation in Chapter II, as shown in Table IV- The new Q-values 0, 1 ; Else:

The new Q-values 0.5, 0.5 ; Get the old Q-values of this assignment ordinal from Q-table; the same to the optimal layout in the first ten exact Pareto-optimal solutions to the small numerical example illustrated in Chapter II, the exact Pareto-optimal solutions to the small numerical example in this study are the same to those listed in Table II-1 (except the last exact Pareto-optimal solutions with different optimal layout to the others).

The computation time for the exhaustive search to obtain these 10 Pareto-optimal solutions is 0.609375 seconds. Compared with the computation time considering layout reconfiguration (2088.92 seconds), it seems that layout reconfiguration complicates the research problem a lot.

To evaluate the accuracy of the approximate Pareto-optimal solutions obtained by the proposed Q-learning based reinforcement learning, four metrics, accuracy rate (𝑝 ), precision rate (𝑝 ), deviation of the first objective (𝑑𝑣 ) and deviation of the second objective (𝑑𝑣 ) defined in Chapter II are used by comparing two objective values of the approximate Paretooptimal solutions with those of the exact Pareto-optimal solutions. To compare the performance of the novel method introduced in this chapter, the improved NSGA-II with the proposed constraint handling techniques introduced in Chapter II is also used to solve this small numerical example. The reason for comparing the developed Q-learning method only with the improved NSGA-II is that in the first study, we found both NSGA-II and AMOSA requiring parameter tuning to improve their performance, and the improved NSGA-II performs better than AMOSA.

As shown in Table IV-2, the proposed Q-learning based approximate solution approach obtained the approximate Pareto-optimal solutions immediately, as the computation time for each run is mostly equal to 0. Compared with the exact Pareto-optimal solutions, the first eight exact Pareto-optimal solutions of which all the six operations in each solution are planned to be performed on the same machine cannot be obtained by this approach. This implies that the proposed Q-learning based approximate solution approach could balance the workload on machines in the RMS. To evaluate the performance of this approach, the improved NSGA-II with the constraint handling techniques introduced in Chapter II is also adopted in this study to [START_REF] Smets | Shouldn't customers control customized product development?[END_REF][START_REF] Yassine | Investigating the role of IT in customized product design[END_REF], [START_REF] Smets | Shouldn't customers control customized product development?[END_REF][START_REF] Yassine | Investigating the role of IT in customized product design[END_REF], [START_REF] Smets | Shouldn't customers control customized product development?[END_REF][START_REF] Yassine | Investigating the role of IT in customized product design[END_REF], [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Lasi | Industry 4.0[END_REF][START_REF] Elmaraghy | Product variety management[END_REF], [START_REF] Lasi | Industry 4.0[END_REF][START_REF] Elmaraghy | Product variety management[END_REF] 2 15.97 1 1 0 0 [START_REF] Smets | Shouldn't customers control customized product development?[END_REF][START_REF] Yassine | Investigating the role of IT in customized product design[END_REF], [START_REF] Smets | Shouldn't customers control customized product development?[END_REF][START_REF] Yassine | Investigating the role of IT in customized product design[END_REF], [START_REF] Smets | Shouldn't customers control customized product development?[END_REF][START_REF] Yassine | Investigating the role of IT in customized product design[END_REF], [START_REF] Smets | Shouldn't customers control customized product development?[END_REF][START_REF] Yassine | Investigating the role of IT in customized product design[END_REF], [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Lasi | Industry 4.0[END_REF][START_REF] Elmaraghy | Product variety management[END_REF], [START_REF] Lasi | Industry 4.0[END_REF][START_REF] Elmaraghy | Product variety management[END_REF] 3 15.25 1 1 0 0 [START_REF] Smets | Shouldn't customers control customized product development?[END_REF][START_REF] Yassine | Investigating the role of IT in customized product design[END_REF], [START_REF] Smets | Shouldn't customers control customized product development?[END_REF][START_REF] Yassine | Investigating the role of IT in customized product design[END_REF], [START_REF] Smets | Shouldn't customers control customized product development?[END_REF][START_REF] Yassine | Investigating the role of IT in customized product design[END_REF], [START_REF] Smets | Shouldn't customers control customized product development?[END_REF][START_REF] Yassine | Investigating the role of IT in customized product design[END_REF], [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Lasi | Industry 4.0[END_REF][START_REF] Elmaraghy | Product variety management[END_REF], [START_REF] Lasi | Industry 4.0[END_REF][START_REF] Elmaraghy | Product variety management[END_REF] 𝑝 10, 𝑝 0.1, 𝑔 50 1 0.17 0 0 14.62% 9.95% [START_REF] Stump | Integrating lean and other strategies for mass customization manufacturing: A case study[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Stump | Integrating lean and other strategies for mass customization manufacturing: A case study[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Da Silveira | Mass customization: Literature review and research directions[END_REF][START_REF] Chong | A review of digital manufacturing-based hybrid additive manufacturing processes[END_REF], [START_REF] Andersen | Engineering education in changeable and reconfigurable manufacturing: Using problem-based learning in a learning factory environment[END_REF][START_REF] Singh | Design of cellular manufacturing systems: An invited review[END_REF], (17, 126) 2 0.16 0.25 0.1 0.23% 11.89% [START_REF] Smets | Shouldn't customers control customized product development?[END_REF][START_REF] Yassine | Investigating the role of IT in customized product design[END_REF], [START_REF] Adan | Combining make to order and make to stock[END_REF][START_REF] Lyly-Yrjänäinen | Effects of combining product-centric control and direct digital manufacturing: The case of preparing customized hose assembly kits[END_REF], [START_REF] Andersen | Engineering education in changeable and reconfigurable manufacturing: Using problem-based learning in a learning factory environment[END_REF][START_REF] Singh | Design of cellular manufacturing systems: An invited review[END_REF], [START_REF] Andersen | Engineering education in changeable and reconfigurable manufacturing: Using problem-based learning in a learning factory environment[END_REF][START_REF] Singh | Design of cellular manufacturing systems: An invited review[END_REF] All the values of the independent decision variables in this study for the approximate Pareto-optimal solutions presented in the above table are in given in Table C.

3 0.17 0 0 3.14% 2.61% [START_REF] Stump | Integrating lean and other strategies for mass customization manufacturing: A case study[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Stump | Integrating lean and other strategies for mass customization manufacturing: A case study[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Lasi | Industry 4.0[END_REF][START_REF] Kwon | How online self-customization creates identification: Antecedents and consequences of consumer-customized product identification and the role of product involvement[END_REF], [START_REF] Lasi | Industry 4.0[END_REF][START_REF] Kwon | How online self-customization creates identification: Antecedents and consequences of consumer-customized product identification and the role of product involvement[END_REF] 4 0.20 0.5 0.3 9.87% 7.24% [START_REF] Smets | Shouldn't customers control customized product development?[END_REF][START_REF] Yassine | Investigating the role of IT in customized product design[END_REF], [START_REF] Smets | Shouldn't customers control customized product development?[END_REF][START_REF] Yassine | Investigating the role of IT in customized product design[END_REF], [START_REF] Smets | Shouldn't customers control customized product development?[END_REF][START_REF] Yassine | Investigating the role of IT in customized product design[END_REF], [START_REF] Li | Customer satisfaction evaluation method for customized product development using Entropy weight and Analytic Hierarchy Process[END_REF][START_REF] Gerrikagoitia | Digital manufacturing platforms in the Industry 4.0 from private and public perspectives[END_REF], [START_REF] Perret | Production Scheduling of Personalized Fashion Goods in a Mass Customization Environment[END_REF][START_REF] Singh | Design of cellular manufacturing systems: An invited review[END_REF] [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Andersen | Engineering education in changeable and reconfigurable manufacturing: Using problem-based learning in a learning factory environment[END_REF][START_REF] Singh | Design of cellular manufacturing systems: An invited review[END_REF] 9 0.17 0.5 0.2 7.73% 10.32% [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Mohapatra | Significance of digital technology in manufacturing sectors: Examination of key factors during Covid-19[END_REF][START_REF] Hoftijzer | Implementing 'Design for Do-It-Yourself' in Design Education[END_REF], [START_REF] Andersen | Engineering education in changeable and reconfigurable manufacturing: Using problem-based learning in a learning factory environment[END_REF][START_REF] Singh | Design of cellular manufacturing systems: An invited review[END_REF], [START_REF] Andersen | Engineering education in changeable and reconfigurable manufacturing: Using problem-based learning in a learning factory environment[END_REF][START_REF] Singh | Design of cellular manufacturing systems: An invited review[END_REF] 10 0.20 0.5 0.2 6.80% 7.72% [START_REF] Smets | Shouldn't customers control customized product development?[END_REF][START_REF] Yassine | Investigating the role of IT in customized product design[END_REF], [START_REF] Smets | Shouldn't customers control customized product development?[END_REF][START_REF] Yassine | Investigating the role of IT in customized product design[END_REF], [START_REF] Adan | Combining make to order and make to stock[END_REF][START_REF] Lee | Predictive manufacturing system -Trends of next-generation production systems[END_REF], [START_REF] Andersen | Engineering education in changeable and reconfigurable manufacturing: Using problem-based learning in a learning factory environment[END_REF][START_REF] Singh | Design of cellular manufacturing systems: An invited review[END_REF] sequence, are developed based on numerical algorithms, mathematical modeling and programming, decision tables and trees, etc., as well as on possessed and stored manufacturing know-how, is better able to handle product variety [START_REF] Azab | Sequential process planning: A hybrid optimal macrolevel approach[END_REF]. Scheduling, especially FJSP, has been proved NP-hard, thus developing approximate solution approaches is necessary to solve the built mathematical models.

The aim of this research is to make a comprehensive decision from joint optimization in order to better perform MC in an RMS. The state of the art about MC, RMS, and implementing MC in RMS is first summarized in Chapter I, including a literature review on planning, scheduling and layout optimization in RMS. Observing that there is no study integrating planning, scheduling and layout optimization for MC in an RMS, the research in this thesis built the multi-objective mathematical model and developed some solution approaches to solve this integrated problem, since it allows the reconfigurability of configurations on machines and their layout in an RMS to be fully exploited. Coped with the small quantity and high diversity of mass-customized products, as well as the process variety and the flexibility of RMS, it is impossible to achieve MC at the low cost of MP by optimizing process planning and scheduling in two separate steps.