Tarun Kumar

Ravi Kalaisrasan

Magnus Wiktorsson

Blockchain-based secured collaborative model for supply chain resource sharing and visibility

Keywords: Blockchain, Resource Sharing, Supply chain collaboration, Visibility

Globalization, escalating competition, and demand for sustainable practices have required supply chain and production managers to consider various capabilities and value creation strategies for the customers. Rapid technological advancement in the current production environment calls for integrative and collaborative efforts for effective resource utilization and better visibility to gain competitive advantages. However, privacy risks and trust have always been a significant barrier for organizations' efforts towards supply chain integration. Supply chain stakeholders fear these collaborate practices might weaken their bargaining power, accelerate risk of data manipulation and result in loss of information advantages. Addressing these issues, the study proposes a Blockchainbased collaborative model for production visibility and resource sharing. It demonstrates the framework for stakeholders' interaction over a central procurement system backed with blockchain technology. The study further lays down the notion of production capacity backed smart contract rules. These smart contracts will run on the proposed blockchain network to reduce the possibilities of fraudulent transactions and capacity overbooking-leading to illegitimate subcontracting. The overall network will stimulate visibility and develop a technologybased trust among partners which ensuring sustainability by effective utilization of resource.

Introduction: Supply chain collaboration

Recent studies highlight the importance and demand for sustainable initiatives in the production and supply chain environment [START_REF] Grzybowska | Literature Review on Sustainable Logistics and Sustainable Production for Industry 4.0[END_REF]. Constant pressure from customers, governments, and stakeholders' groups has stimulated firms to e fficiently incorporate sustainability issues into their strategic and operational planning [START_REF] Chen | Supply chain collaboration for sustainability: A literature review and future research agenda[END_REF]. Nevertheless, these initiatives can only lead to sustainable supply and production when all the individual firms in the supply network collaborate and integrate to possess or develop the necessary resources as requisites for realizing sustainable production and supply chain. In this way, they can facilitate environmentally and ethically sound organizational and supply chain behaviors [START_REF] Grzybowska | Literature Review on Sustainable Logistics and Sustainable Production for Industry 4.0[END_REF][START_REF] Chen | Supply chain collaboration for sustainability: A literature review and future research agenda[END_REF][START_REF] Soosay | A decade of supply chain collaboration and directions for future research[END_REF]. Numerous studies in the past have advocated the benefits of collaborative alliances and inter-organizational resource sharing [START_REF] Soosay | A decade of supply chain collaboration and directions for future research[END_REF][START_REF] Duong | Supply chain collaboration in the presence of disruptions: a literature review[END_REF]. Collaboration has multiple advantage: ranging from network expansion to better insights and visibility in production units, from accessing new talent, skills / techniques and processes to potentially increased productivity, better utilization of resources, faster growth and increased global reach [START_REF] Chen | Supply chain collaboration for sustainability: A literature review and future research agenda[END_REF][START_REF] Soosay | A decade of supply chain collaboration and directions for future research[END_REF]. These collaborations can include both material flow and information flow. Wherein information flow includes not only the process, and product transformation data but also the knowledge creation and transfer among organizations in the forms of new products, processes and technology development [START_REF] Chen | Supply chain collaboration for sustainability: A literature review and future research agenda[END_REF][START_REF] Soosay | A decade of supply chain collaboration and directions for future research[END_REF].

While it is true that many managers have attempted to form collaborative alliances with other partner organizations, such strategies are tough to implement. Usually, they fail as implementation and management of an alliance is much harder than the decision to collaborate [START_REF] Eurich | The impact of perceived privacy risks on organizations' willingness to share item-level event data across the supply chain[END_REF]. It is found that building consensus among organizations is often difficult. As collaboration involves material and information flow, partner organizations often do not trust each other and perceive a strong risk of losing strategic/competitive advantage. They believe information sharing among partners can lead to risk of manipulated data, loss of know-how and expertise, weakening of bargaining power and information disadvantage [START_REF] Eurich | The impact of perceived privacy risks on organizations' willingness to share item-level event data across the supply chain[END_REF]. At the same time, collaboration leads to a revelation of the whole supply network and easier tracing of unethical practices, which happens to be a major concern for dishonest partners.

In fact, lack of collaboration among partners has made the supply chain quite unsustainable. There is lack of transparency in supply chain, ineffective utilization of resources, issues of capacity overbooking -leading to illegitimate subcontracting and inefficient production planning and synchronization. Hence, there is a need for further development of digital collaborative platforms that can facilitate information sharing and develop mutual trust among partners without risking their competitive and information advantage.

In this direction, the paper proposes blockchain as a potential technology that can address the above challenges. Several studies in past has advocated blockchain application for supply chain digitalization, traceability and visibility enhancement, improving forecasting, demand based supply planning, fraud prevention, reducing negative impact on environment [START_REF] Korpela | Digital Supply Chain Transformation toward Blockchain Integration[END_REF][START_REF] Dujak | Blockchain Applications in Supply Chain[END_REF][START_REF] Saberi | Blockchain technology and its relationships to sustainable supply chain management[END_REF]. Nevertheless, [START_REF] Queiroz | Blockchain adoption challenges in supply chain: An pirical investigation of the main drivers in India and the USA[END_REF] identifies blockchain for supply chain integration as a potential field lacking substantial research contribution. Besides, it is important to consider and examine the blockchain operations based on its applications in supply chain requirements [START_REF] Umeh | Blockchain Double Bubble or Double Trouble?[END_REF]. Therefore, based on literature and understanding of blockchain technology, the study introduce a work-in-progress solution contributing to improve supply chain collaboration at operational level. It follows a pearl growth search strategy to explore literature (initiating from few sources to identify and understand appropriate terminologies that could be used in search queries) [START_REF] Rowley | Conducting a literature review[END_REF]. Thereafter, the paper primarily focuses on information flow between actors and illustrates a blockchain-based collaborative network that can ensure privacy of individual partner, promote visibility in the network and ultimately develop a technology-based trust among partners. Finally, smart contact rules for the proposed system has been explained that can lead to efficient resource sharing and reduce fraudulent transactions. The potential effects and consequence of the proposed solution are explained in the discussion section followed by conclusions and future research areas.

Proposed collaborate network platform

Based on literature, the proposed collaborate platform is presented by introducing general aspects on blockchain technology, followed by an example where two key features are described: Supply chain partner integration and Capacity based smart contract.

Blockchain technology

Blockchain is defined as: A shared, immutable ledger that facilitates the process of recording transactions and tracking assets in a business network. An asset can be tangible (a house, a car, cash, land) or intangible (intellectual property, patents, copyrights, branding). Virtually anything of value can be tracked and traded on a blockchain network, reducing risk and cutting costs for all involved [START_REF] Gupta | Blockchain for Dummies[END_REF].

Blockchain can be public (open to anyone and everyone with access to internet [START_REF] Gupta | Blockchain for Dummies[END_REF]. Anyone can join the network and perform transactions) and private (only authorized participants can join the network and their rights are defined by network accessibility rules) [START_REF] Swan | Blockchain: Blueprint for a New Economy[END_REF]. Due to text limitation, the paper explains four key features of blockchain technology that makes it unique and secure. These features play a vital role in the proposed collaborative framework. Subsequently, a detailed overview of the technology and its application in logistics and manufacturing industry can be found in [START_REF] Venkatesh | System architecture for blockchain based transparency of supply chain social sustainability[END_REF][START_REF] Francisco | The Supply Chain Has No Clothes: Technology Adoption of Blockchain for Supply Chain Transparency[END_REF][START_REF] Lee | A blockchain enabled Cyber-Physical System architecture for Industry 4.0 manufacturing systems[END_REF][START_REF] Liu | Industrial blockchain based framework for product lifecycle management in industry 4.0[END_REF].

First is distributed network, which makes it ideal for multi-tier supply chain applications where the partners are usually located at large geographical distances with limited amount of trust. It encourages partners with differences in governance rule to share essential information on a shared platform. Second, smart contract, which contains transaction rules agreed among partners, written and stored as computer programs over the network. It ensures validity of transaction and trust that that everyone is playing fairly. Third, data immutability, which is achieved through cryptographic function and block linking mechanism. It ensures traceability and accountability of each transaction. Fourth, is consensus, which is the process of reaching an (automatic) agreement among all the involved partners for each transaction by verifying it with each individual copy of shared ledger. It helps in reducing fraudulent and inaccurate transactions and overcome the Byzantine fault problem [START_REF] Gupta | Blockchain for Dummies[END_REF][START_REF] Swan | Blockchain: Blueprint for a New Economy[END_REF][START_REF] Abeyratne | Blockchain ready manufacturing supply chain using distributed ledger[END_REF].

Blockchain-based collaborative model : Supply chain partner integration

For effective communication, a simplified example of supply chain of two suppliers (A and B) and one buyer (focal firm) the framework explains partner interactions over the network. Buyer is downstream manufacturing unit in the supply chain that procures the same kind of intermediate products from supplier A and B. It can be noted that the described network can be complicated with certain modifications and following similar network rules to fit in real case examples and/or more complex supply chain networks. Production at buyer site is highly dependent on flow of intermediate products from suppliers. In usual situations, both suppliers compete to secure more orders to keep their production running and gain more profit. This lead to over-booking of capacity, irregular utilization of resources and illegitimate subcontracting. Additional, sharing information lead to security and privacy issue.

Hence, in the proposed framework, as shown in Fig. 1, the individual suppliers can be connected over different channels over a same blockchain network managed by a focal firm, in this example, the buyer. Each channel is a subnet/subnetwork of the global blockchain network, having their separate smart contract and a shared ledger that runs transaction rules as agreement between buyer and respective supplier. Individual channels also have their own-shared ledger to recode different transactions between the buyer and supplier and ensures traceability and visibility. However, on the global network scale, each channel is connected to central procurement unit that only records and maintains shareable real-time capacity data. Thus, the central procurement unit ledger can be referred for order procurement to ensure fair and efficient utilization of resources. It can be further observed from Fig 1 that each production unit (buyer and suppliers) have an internal traceability system that is connected with sensors at each work station, tracking and recording material flow at each stage of the production system. This enables better visibility within the production unit and is helpful to identify bottleneck. Nevertheless, as shown, and if administered properly, it is not necessary to link each of these data collection points directly to a blockchain network. Direct linking can make the network cumbersome.

Characteristics of the Blockchain-based collaborative model

The proposed collaborative model utilizes the four key features of Blockchain technology, the distributed network, smart contract, data immutability and consensus. The distributed network and capacity based smart contract is explained in the previous sections. Data immutability is ensured by the cryptographic SHA function, Merkle tree and linking the chain of hashes with the newly formed block, as in case of bitcoin. A detail about the explanation about the data immutability feature can be found in [START_REF] Gupta | Blockchain for Dummies[END_REF][START_REF] Swan | Blockchain: Blueprint for a New Economy[END_REF]. As the proposed model is a permissioned network, actors can only access the network based on their accessibility condition and consensus algorithm need not to be as complex as the proof-of-work in case of bitcoin. Proof-of-stake or proof-of-elapsed time can be used to test and achieve consensus among the actors. Additionally, being a permissioned network, not all actor can perform all type of task on the blockchain. For instance, an actor with read only permission can only access the distributed ledger to check the transactions, whereas an actor with write permission can read as well as record a transaction on the ledger. Similarly, only limited actors will have the permission to actually perform transactions on the blockchain. This characteristic makes the system secure but also eliminate possibility of unwanted transactions in a distributed network.

Potential effects of the Blockchain-based collaborative model

The hypothesis behind the proposed blockchain-based collaborative model is that the distributed database technology for secure data responds well to the increasing complexity of global supply chains, with high requirements on security, privacy and trust amongst actors, while still enabling high visibility and resource utilization for individual actors in the network. By the technology for secure and transparent collaboration, would the resilience and sustainability of the system as a whole increase. These performance benefits are however still left to prove, as well as specific challenges in implementing and operating the collaborative model. By relating the blockchain-based collaborative model to the five performance dimensions of the SCOR (Supply Chain Operations Reference) model [START_REF]New SCOR Model for the Modern Supply Chain | APICS[END_REF], performance benefits and challenges could be envisioned as briefly listed in Table 1. Collaboration and resource sharing helps in effective utilization of available assets and working capital (resources).Thus, helpful to attain return on investment.

Conclusions

Taking an example of buyer and suppliers' interaction, the paper explains a blockchainbased secured collaborative model for visibility and resource sharing. A network framework using channel /subnet has been demonstrated that can ensure security and at the same time privacy, in the blockchain network. Further, a capacity-based smart contract has been proposed that can ensure efficient utilization of resources and prevent fraudulent transactions. Thus assuring a technology-based trust among the partners in a multi-tier supply chain. The proposed model ensure effective utilization of resources, ensuring economical sustainability. It can be noted that this study is work-in-progress and in future works, the model will be complicated to include other stages of s upply chain. Further, a discussion with companies could enable a first step towards validation of the still theoretical model. The proposed smart contact would be developed on an example blockchain network to test with a simulated situation of the demonstrated framework. Followed by development of a consensus algorithms that would facilitate validation and broadcasting of transactions on blockchain. Beside, further research can be carried out to identify key factors that can influence the collaborative models (including those related with social and environmental sustainability), which, can be input for the smart contract and useful for multi criteria decision making for optimum order distribution. Moreover, as data sharing among partners is a major concern due to privacy issues; therefore, the data components and their accessibility and visibility to the specific actors would be identified and segregated. Further, for coherent and seamless data communication and sharing data nomenclature would be defined.

Fig. 1 .

 1 Fig. 1. A simple illustration of supply chain partners' (one focal firm as buyer and two suppliers) interactions over a blockchain-based collaborative network.

  collaborative model : Capacity based smart contract A capacity-based smart contact would be an apt solution to prevent fraudulent transactions and overbooking of capacity in the proposed collaborative model. The shared ledger on the central procurement unit will connect real-time data of the capacity of individual supplier from individual channels. These real-time data, for instance, could include information related to number of operational machine, shift hours, product SAM, line efficiency, machine downtime, operators' attendance etc. Even factors related to multi criteria decisions including factors of the triple bottom line to reach the sustainability of the supply chain. In a situation of another new demand from the buyer, it will verify and distribute order according to the available resources as shown in Fig.2.

Fig. 2 .

 2 Fig. 2. Steps for capacity-based smart contract execution

Table 1 .

 1 SCOR performance attribute and how the proposed model can influence them

	Performance at-	Metric description	Blockchain-based collaborative model
	tribute
	Reliability	Order fulfilment.	Develops a technology-based trust among the
		(customer focus)	partners and ensure fair play. Capacity based
			smart contract can facilitate effective utiliza-
			tion of resources for on-time order fulfilment.
	Responsiveness	Order fulfilment cycle	Optimum order allocation based on real time
		time.	capacity knowledge can ensure quick response
		(customer focus)	and shorter order fulfilment cycle.
	Agility	Upstream adaptability. Downstream adaptability. Value-at-Risk. (customer focus)	Secured information sharing mechanism can enhance supply chain visibility, which facili-tate swift adaptability in unforeseen situations.
	Cost	Total SCM costs.	Effective resource utilization reduces produc-
		Cost of goods sold.	tion cost.
	Asset Manage-	Cash-to-cash cycle time.
	ment Efficiency	Return on fixed assets.
		Return on working capi-
		tal.
		(internal focus)

Acknowledgement

We gratefully acknowledge the funding from Produktion2030 and Vinnova for this research, as part of the research project Production Logistic Visibility (LOVIS).


Hajime Mizuyama
email: mizuyama@ise.aoyama.ac.jp

Agent-Based Modeling and Analysis of Dynamic Slab Yard Management in a Steel Factory

Keywords: Slab Yard Management, LIFO Buffers, Reinforcement Learning

The slab yard upstream of a heating furnace in a steel factory has several LIFO buffers and a crane. In managing the yard, slabs are moved from a buffer to another and to the furnace by the crane. Which specific operation should be done and when are determined by the crane operator according to the situation being dynamically changed with arrivals of new slabs to the yard and removals of heated slabs from the furnace. This study models a mathematical aspect of the dynamic decision process for managing the yard, and analyzes the effects of some factors on its performance using a reinforcement learning agent. As a result, it reveals an interaction effect between the arrival rate of slabs and how many steps in the future the agent considers when making decision. The findings obtained by this and similar following studies will be valuable for supporting the actual decision process and for training skills for the process.

Introduction

There is a heating furnace between casting and rolling processes in a steel factory, and slabs supplied from the casting process are loaded to the rolling process after being adjusted their temperature by the furnace. The yard upstream of the furnace has several stacks, i.e. LIFO buffers, of slabs and a crane, and the slabs are moved by the crane from a stack to another or to the furnace. In the factory considered in this paper, the sequence and timing of arrivals of slabs, those of loading the slabs to the rolling process are not strictly specified in advance. Hence, it becomes a dynamic decision process to determine which slab should be moved to where and when by the crane. This decision process involves quite a few objectives and constraints, such as satisfying the due date assigned to each slab, inserting slabs of similar heating conditions as consecutively as possible, ordering slabs so that the transitions of their width and thickness should fit the requirements of the rolling process. Further, their relative importance depends on the situation. In this factory therefore the dynamic decision process for managing the slab yard is carried out by the operator according to the state of the yard and the furnace, various information exchanged and negotiated with the neighboring processes, etc. Thus, it is an important issue how to effectively support the human decision process online as well as how to efficiently train skills necessary for conducting the process offline. Some authors assumed that the initial arrangement of slabs in the stacks is given and no further slabs will arrive afterwards, and dealt with a planning problem of crane operation for loading the slabs to the furnace and rearranging them in the stacks before loading if necessary. For example, [START_REF] Tang | An Effective Heuristic Algorithm to Minimise Stack Shuffles in Selecting Steel Slabs from the Slab Yard for Heating and Rolling[END_REF][START_REF] Tang | Modelling and a Genetic Algorithm Solution for the Slab Stack Shuffling Problem when Implementing Steel Rolling Schedules[END_REF][START_REF] Singh | Modelling the Slab Stack Shuffling Problem in Developing Steel Rolling Schedules and its Solution Using Improved Parallel Genetic Algorithms[END_REF][START_REF] Tang | Modelling and a Segmented Dynamic Programming-Based Heuristic Approach for the Slab Stack Shuffling Problem[END_REF][START_REF] Cheng | A Scatter Search Algorithm for the Slab Stack Shuffling Problem[END_REF][START_REF] Tang | Models and Algorithms for Shuffling Problems in Steel Plants[END_REF] proposed how to minimize the number of rearrangement operations (or a certain generalized objective, such as a weighted sum of the number of rearrangement operations and the total moving distance of the crane) with satisfying a specified constraint on the loading sequence. Other authors considered both departures and arrivals of slabs. For instance, [START_REF] Konig | Solutions to Real-World Instances of PSPACE-Complete Stacking[END_REF] minimized the number of crane movements under a given arrival schedule and a specified loading sequence. [START_REF] Kim | A Simplified Steel Plate Stacking Problem[END_REF][START_REF] Lu | A 0-1 Integer Programming Model and Solving Strategies for the Slab Storage Problem[END_REF] minimized the number of rearrangement operations under given arrival and loading sequences. [START_REF] Rei | Heuristic Search for the Stacking Problem[END_REF][START_REF] Rei | Tree Search for the Stacking Problem[END_REF] minimized the number of crane movements under a given arrival schedule and specified loading due dates. However, all of these earlier studies capture the problem of managing the slab yard as a static optimization problem, which is essentially a different setting from the dynamic decision process to be addressed in this paper. Further, they all aim at automating and computerizing operational planning and lacked the perspective of how to support a human decisionmaking and how to train skills for it.

Why the task of managing the slab yard is deferred to a human in this factory is not only because it is a dynamic decision process with quite a few objectives and constraints. It is also and more essentially because various conditions and parameters necessary for formulating the decision-making problem underlying the task, such as the relative importance among the objectives and constraints, how to parameterize the uncertain future state, etc. are not fixed a priori and dependent on the changing situation, and hence the human's informal and intuitive estimate on them made on the fly in the shop floor is often helpful and can work favorably. Further, relevant information to the estimate can be attained not only by careful observation of the yard, but also by communicating and negotiating with the managers of neighboring processes. These proactive information collection and intervention can be naturally carried out by a human but are still quite difficult to automate. On the other hand, the decisionmaking problem itself obtained by setting the parameters and conditions with the help of fuzzy information processing by a human is still a sort of mathematical optimization problem. When considering how to support this decision-making, it will be difficult to take an approach which provides potential solutions to the human in charge, because the problem to be solved cannot be clearly captured a priori and will become complete only after being complemented by the decision maker's intuition. A promising alternative would be to provide an adequate solution framework to be taken when a human cognitively derives a solution to the problem.

Thus, this paper adopts a reinforcement learning model [START_REF] Sutton | Reinforcement Learning: An Introduction, 2nd edition[END_REF] for capturing the framework of the dynamic decision process for managing the slab yard and tries to analyze what factors affect its performance using the model. Due to the sparkling success of recent deep learning techniques and their application to reinforcement learning, (deep) reinforcement learning approaches are actively applied even to manufacturing systems. Most of such applied researches aim at enhancing and automating dynamic scheduling and dispatching [START_REF] Wang | Application of Reinforcement Learning for Agent-Based Production Scheduling[END_REF][START_REF] Gabel | Distributed Policy Search Reinforcement Learning for Job-Shop Scheduling Tasks[END_REF][START_REF] Qu | Optimized Adaptive Scheduling of a Manufacturing Process System with Multi-Skill Workforce and Multiple Machine Types: An Ontology-based, Multi-agent Reinforcement Learning Approach[END_REF][START_REF] Shahrabi | A Reinforcement Learning Approach to Parameter Estimation in Dynamic Job Shop Scheduling[END_REF][START_REF] Ou | Gantry Work Cell Scheduling through Reinforcement Learning with Knowledge-Guided Reward Setting[END_REF][START_REF] Stricker | Reinforcement Learning for Adaptive Order Dispatching in the Semiconductor Industry[END_REF][START_REF] Waschneck | Optimization of Global Production Scheduling with Deep Reinforcement Learning[END_REF][START_REF] Ou | Simulation Study on Reward Function of Reinforcement Learning in Gantry Work Cell Scheduling[END_REF][START_REF] Minguillona | Coupling of Centralized and Decentralized Scheduling for Robust Production in Agile Production Systems[END_REF][START_REF] Kuhnle | Autonomous Order Dispatching in the Semiconductor Industry Using Reinforcement Learning[END_REF][START_REF] Kuhnle | Design, Implementation and Evaluation of Reinforcement Learning for an Adaptive Order Dispatching in Job Shop Manufacturing Systems[END_REF][START_REF] Chen | Can Sophisticated Dispatching Strategy Acquired by Reinforcement Learning?: A Case Study in Dynamic Courier Dispatching System[END_REF]. This paper can be distinguished against them in that it utilizes a reinforcement learning model as a framework for analyzing how a human addresses a dynamic decision making process. In the remainder, the slab yard and its management task are modelled, and then a reinforcement leaning agent is introduced. Further, numerical experiments and their results are presented, and finally the paper is concluded with potentially fruitful future research directions.

A Simple Model of Slab Yard and Its Management

This section provides a model of the dynamic decision process for managing the slab yard, which is considerably simplified but captures essential features of the mathematical aspect of the actual process. The outline of the model is described below.

• The heating furnace and the following rolling process are grouped into a virtual single machine for simplicity, and the slab yard is captured as a set of the machine, several buffers, and a crane. • The buffers in the yard is classified into an entrance buffer, four intermediate buffers, and a loading buffer to the machine. Further, there is assumed to be an invisible queue upstream of the entrance buffer. • The entrance and intermediate buffers are LIFO, and the loading buffer and the queue are FIFO. The capacity of the buffers is set to four, except that the queue has an infinite capacity. • Slabs arrive randomly and enter into the queue. The time between consecutive arrivals follows an exponential distribution with 1/𝜆 ∈ {10, 11, 12}. Slabs in the queue are automatically moved to the entrance buffer one by one and the cycle time of this movement operation is four.

• Each slab has information on its type (∈ {1, 2, 3, 4}) and due (specified by adding a random variable from a uniform distribution [60, 720] to the arrival time). • Slabs in the loading buffer are automatically loaded to the machine one by one when the machine becomes available. The processing time of each slab is six irrespective of the type. • A setup operation is necessary before loading a slab if its type is different from that of the immediately earlier one, and its time depends heavily on whether the type is changed in an increasing direction (= 6) or a decreasing one (= 60). • The order of processing the slabs is not rigidly specified a priori, but only loosely constrained by their due dates. • The possible route of the crane is represented by a star graph, whose leaves correspond to the entrance, intermediate and loading buffers. The traveling time of each edge of the graph is the same. • Every movement cycle of the crane is to start from the center node, move to a buffer, take out a slab from the buffer, travel to another buffer via the center node, release the slab there, and come back to the center node. The cycle time of this movement is four. • A unitary bonus is given to the operator each time a slab is loaded to the machine.

• A tardiness penalty (= tardiness × 1/30) is incurred for each slab which cannot be loaded to the machine before its due date. • The objective function to be maximized is the score defined by subtracting the total tardiness penalty from the total bonus attained in a specified period of time (= 60 × 24) starting from a randomly set initial state.

Crane Operator Reinforcement Learning Agent

Actions

The crane operator needs to decide what action to take next in every cycle time. Possible actions are to wait in the center node for a cycle or to move a slab from a buffer (origin) to another (target). When "to move" is chosen, the origin and target buffers need to be specified. Neither the loading buffer nor empty buffers cannot be chosen as the origin. Hence, the origin must be selected among nonempty entrance or intermediate buffers. Similarly, the target must be selected among non-fully occupied intermediate or loading buffers, since neither the entrance nor fully occupied buffers cannot be specified as the target.

Rewards

How much reward the crane operator receives in each cycle can be defined with the score, i.e. the value calculated by subtracting the tardiness penalty from the loading bonus. More specifically, the reward is defined by the difference of the score between before and after a corresponding action is taken.

State Features

The state of the yard changes with actions taken by the crane operator, arrivals of new slabs, and accompanied events, such as moving a slab to the entrance buffer, loading one to the machine, starting and finishing a setup operation, starting and finishing processing a slab on the machine, etc. Further, a same state may be perceived differently, and it may affect the performance of the decision process. For simplicity, this paper assumes the operator perceives the state of each buffer with the features below.

The level of the buffer, the number of slab types in it, the number of times the type changes in an increasing direction when taking out slabs one by one, that in a decreasing direction, the average due date, the number of times a consecutive pair of slabs are lined up in the order of their due dates, that in the opposite order, the number of slabs whose due date is earlier than any slab in the loading buffer, their maximum depth from the top, the number of slabs which cannot satisfy their due date without rearranging their position, their maximum depth, the maximum estimated tardiness, the depth of the slab whose estimated tardiness is the longest, the sum of the expected tardiness of all slabs, the sum of the slack times of all slabs

State Value Function

The discounted sum of the rewards which the crane operator in a specific state will achieve in the future if she/he follows a certain policy from that on is called the state value. This paper approximates the state value function by a standard multi-layer neural network whose input is the state feature vector and output is the state value.

Myopic and Forward-Looking Policies

If arrivals of new slabs are ignored, the crane operator can envision the state (or afterstate) attained as the result of taking an action from a state. Similarly, the state attained by taking another action from the afterstate, the next state attained by further taking another action, etc. can also be estimated. Thus, it is quite natural to assume that the crane operator chooses an action so that the value of an afterstate be maximized. The question is which afterstate it is. In other words, of how many steps in the future an afterstate is considered?

Learning Algorithm

This paper utilizes 𝑇𝐷(𝜆) algorithm, which learns the state value function according to so called 𝜆 return [START_REF] Tesauro | Practical Issues in Temporal Difference Learning[END_REF]. This algorithm is suitable for a dynamic decision process where afterstates can be defined as described above, and is confirmed to be effective in real-life problems such as backgammon [START_REF] Tesauro | Temporal Difference Learning and TD-Gammon[END_REF].

Numerical Experiments

This section conducts numerical experiments using the mathematical models described above and analyzes what factors affect the performance of the decision process of managing the slab yard. Potential factors include how the state is perceived, how much discounting rate is used for calculating the state value, how many future steps are considered in each decision, etc., but this paper focuses on the effects of the number of forward-looking steps with changing the arrival rate of slabs. More specifically, experiments are conducted under nine different conditions obtained by combining three levels of the mean time between arrivals of slabs (∈ {10, 11, 12}) and three levels of the number of future steps considered (∈ {1, 2, 3}).

Other settings are determined according to preliminary experiments. For instance, a standard fully connected multi-layer network with two intermediate layers of 40 nodes is used as the structure of the neural network approximating the state value function. The network uses a sigmoid function as the activation function of every node. The initial value of the learning rate is set to 0.00001 and is decreased exponentially by the ratio of 0.999 in every episode. The learning process is continued up to 1000 episodes. Since the crane operator chooses an action (and a learning step is taken) for about (60 × 24)/4 =360 times in each episode, this means that the total num-ber of learning steps is about 360,000. The discounting rate for calculating the state value is set to 0.99, and the value of parameter 𝜆 for learning is set to 0.7. Fig. 1 and Fig. 2 show the results. In the figure legends, the number of future steps considered and, in the parenthesis, the mean time between arrivals (i.e. the inverse of the arrival rate) of slabs are indicated. The vertical axis of Fig. 1 shows a moving average of the mean absolute loss of each episode, that of Fig. 2 represents a moving average of the final score of each episode. A most clear result is that the smaller the loss the higher the score. It is also natural that the shorter the mean time between arrivals and the more frequently slabs arrive, the harder the problem and the lower the average score. In addition, it is observed that the average score obtained by the myopic policy with only one step forward-looking is inferior to those obtained by forward-looking policies considering two or more steps in the future irrespective of the mean time between arrivals. Looking into further details reveals that there is no noticeable difference between the policies considering two steps and three steps when the mean time between arrivals is long but, when it is decreased to 10, the superiority of considering three steps appears. These results imply the effectiveness of forward-looking policies and that it depends on the congestion of the yard how many future steps should be taken into consideration.

Conclusions

This paper mathematically modelled the dynamic decision process for managing a slab yard in a steel factory and the process of how an operator learns a policy for the decision process. It then analyzed what factors affect the performance of the decision process by conducting numerical experiments using the models. As a result, forwardlooking decisions are shown to be effective. Further, it depends on the congestion of the yard how many future steps should be taken into consideration.

The numerical experiments conducted in this paper focused mainly on the effects of the number of forward-looking steps considered. It would be also interesting and valuable to investigate the effects of other factors, such as how the state is perceived, how much discounting rate is used for calculating the state value. Further, it would be important to study how to utilize the findings obtained by this and following studies for supporting the actual decision process and for training skills for the process.

Fig. 1 .

 1 Fig. 1. Mean absolute losses.

Fig. 2 .

 2 Fig. 2. Mean game scores.


Mike Freitag

Oliver Hämmerle

Agile guideline for development of smart services in manufacturing enterprises with support of artificial intelligence

Keywords: Smart Service, Design Thinking, Business Model, Agile Service Engineering, Artificial Intelligence

The shift from product-oriented to service-oriented business requires a rethink, especially in traditional companies in the mechanical and plant engineering industry. This guideline for the development of Smart Services is intended to illustrate the complexity and thus improve their handling, supporting the planning and modelling of a Smart Service. The Design Thinking Process is the first step in understanding the challenges associated with the reorientation of the business and in designing appropriate solutions. In the second step, an important part of the rethinking process of companies is the proactive development, revision or complete redesign of their business models. The third part of this guideline is the concept of agile service engineering with support of artificial intelligence, which can be understood as a link between the design thinking process and business model development and can be used as a management tool.

Introduction

This guideline describes the step-by-step development of Smart Services, especially in manufacturing sector. The term "Smart Services" refers to data-based, individually configurable service offerings consisting of services, digital services and products that are organized and provided via integrated platforms [START_REF]Smart Service Welt -Umsetzungsempfehlungen für das Zukunftsprojekt Internet-basierte Dienste für die Wirtschaft[END_REF] It is becoming increasingly important to offer suitable Smart Services to complement the products of manufacturing enterprises. Therefor an integrated development and subsequent management of these Smart Services is of central importance. Figure 01 illustrates the essential elements of this guideline for the development of Smart Services on the basis of the 3 phases:  Design Thinking [START_REF] Plattner | Design-thinking -Understand-Improve-Apply. Landsberg am Lech[END_REF][START_REF] Brenner | Design thinking as mindset, process, and toolbox, Design thinking for innovation[END_REF][START_REF] Schallmo | Design Thinking erfolgreich anwenden. So entwickeln Sie in 7 Phasen kundenorientierte Produkte und Dienstleistungen[END_REF],  Business model [START_REF] Wirtz | Strategic development of business models: implications of the Web 2.0 for creating value on the internet[END_REF] and  Agile Service Engineering [START_REF] Stich | Digitale Dienstleistungsinnovationen Smart Services agil und kundenorientiert entwickeln[END_REF].

When a Smart Service is developed, both the three phases and the sub-processes in these phases can be repeated iteratively. Each of these three phases is already iteratively in progress. Especially in Design Thinking and agile Service Engineering, the iterative approach is a core element of the process model. The development of alternatives also plays an important role in business model modeling. In the approach presented in this paper, the possibilities of iteration are limited (dark grey arrows) due to small and medium-sized companies.

Design Thinking

Plattner et al. [START_REF] Plattner | Design-thinking -Understand-Improve-Apply. Landsberg am Lech[END_REF] propose the 6 phases understanding, observing, defining a position, finding ideas, developing a prototype and testing, which are assigned to Design Thinking and shown in Figure 01. These 6 phases are described below.

Develop new business models

Existing product with additional IoT benefits Here, products that are already established on the market and have possibly been used in the same way for some time are supplemented with IoT features, thus creating additional benefits. This variant represents the lowest level of IoT business models. This model uses the data generated by the purchase and use of a product to offer a new product or an addition to the original product. This allows completely new products to be developed or existing products to be improved. Therefor an innovative value proposition can be created, whereby previously unfamiliar customer groups can also be tapped, for example, by IoT services with agreed savings targets, Services to prevent machine/system failures, analysis and optimization solution for reducing consumption, data as a separate business, etc. "Heidelberg Prinect" is a suitable case study and is briefly described below.

Case Study »Heidelberg Prinect« "Heidelberg Prinect" offers intelligent machines that autonomously organize and run standardized print processes. This requires the harmonious interaction of six key factors: involving customers, reduce touchpoints, increase productivity and runtime, reduce waste and inventory, optimize consistency and repeatability and business intelligence. Fig. 02 illustrates the concept of Heidelberg Prinect. The "Trumpf Axoom Eco-System" represents a suitable case study [START_REF]Homepage von Trumpf[END_REF].

Develop productless Smart Services

Many enterprises use agile methods in development of productless Smart Services.

There are a number of different methods that can be used. They have some elements in common: they develop a service in small steps that are not planned in advance. To develop a new Smart Service it is necessary to use iterative cycles, this means that the results are tested and evaluated and the results of the tests are incorporated into the next interactive cycle. Furthermore, they are strongly user-centric, so the wishes and reactions of the users are considered at every stage of development.

Considerations of customer processes, the design of a digital ecosystem and the optimization of usability and user experience are also of great importance in the development of a Smart Service. Companies can use data to supplement completely productless Smart Services. Sensor data-based optimization of services, subscription for data analysis software solutions (SaaS models) and revenue share of the platform operator for applications on the platform are just some examples of productless Smart Services.

In order to develop a sustainable business model it is necessary to define a marketing strategy from a business management perspective. The determination of the clientele should be separated from the consideration of the users. Not every user group is automatically a customer group and willing to pay for the Smart Service -and even if the willingness to pay is positive, pricing is not always optimal, since in addition to relative willingness to pay indirect network effects must also be considered. This can be observed especially in Smart Services.

The market launch strategy starts with the definition of a problem or a user or customer group. Through agile development, in which the hypotheses are gradually captured and tested, a service can be developed with very high accuracy, which is also demanded by users and paid for by the clientele. At the same time, in order to prepare the actual market launch, the marketing and sales concept must be developed and implemented on the basis of the value proposition. This should be as iterative as product development. However, other factors are also decisive. Thus, it must be examined which (potential) competitors offer or will offer similar services in the future and whether they offer advantages from the point of view of the target person (price, use, access, range of functions, image, etc.).

This knowledge can be used to further develop and adapt the service or value proposition. The "DigitalClone" from Sentient Science is a suitable example for productless Smart Service [START_REF]Homepage der Sentient Science[END_REF].

Agile Service Engineering

One possible approach to manage the Smart Service is agile service engineering. This can be described by the three phases of conception, implementation and test, which in turn can be assigned to the three levels of business model management, smart service management and network management. These three levels contain a total of 12 development modules for the three phases of conception, implementation and test [START_REF] Freitag | An Approach to Standardise a Service Life Cycle Management[END_REF][START_REF] Freitag | Ein konfigurierbares Vorgehensmodell für die exportorientierte Entwicklung von technischen Dienstleistungen[END_REF][START_REF] Freitag | Smart Service Lifecycle Management[END_REF][START_REF] Wiesner | Cyber-Physical Product-Service Systems[END_REF]. Fig. 05 shows the three phases of agile service engineering with their respective 4 development modules.

Fig. 3. 3 Phases of agile service engineering [17]

The entire process is therefore not only focused on the Smart Service alone, but aims at the Smart Service as a holistic solution. As a result, the levels of business model management and network management are also considered. Each of the three phases contains four development modules.

Artificial Intelligence is part of the Smart Service Management. In a study by Fraunhofer IAO [START_REF] Bauer | Künstliche Intelligenz in der Unternehmenspraxis -Studie zu Auswirkungen auf Dienstleistung und Produktion[END_REF], companies that already have experience in using AI-applications were interviewed. The vast majority of the AI-applications were assigned to the provision of services (59 percent). Customer support (39 percent) and manufacturing (18 percent) follow at a considerable distance. If marketing and sales are added to the first two areas, an astonishingly high number of applications in which a high level of customer reference and possibly even direct customer integration can be assumed. From this it can be concluded that the use of artificial intelligence in companies does not only affect their own managers and employees, but to a large extent also the customers of the companies.

The study revealed basic AI-functionalities used by the respective applications. Data and information extraction are used most frequently (78 percent), followed by databased forecasting (59 percent) and decision support and automation (51 percent).

Case Study »Heidelberg Prinect Production Manager«

To take the example of the Heidelberg Prinect from Chapter 3, the "Prinect Production Manager" -a new usage concept in the form of a rental model of the Prinect business intelligence platform -can be used as an example of agile service engineering. The Production Manager enables print shops of all sizes to use all the integration functions of the Prinect production workflow exactly as they need it. Customers can use all the Smart Service functions at any time. However, they do not pay for the licenses for the individual components but only for the actual monthly data consumption, which is calculated in m². The user fee also includes all automatic software updates so that the workflow is always up to date. While the necessary hardware is installed on the customer's premises, parts of the selected Prinect components are cloud-based.

"The digitalization of all processes is not a question of company size. If a print shop wants to remain competitive, it needs to address the issue," says Anthony Thirlby, Head of Prinect at Heidelberg. "It's about developing a digital business model that focuses on end customer requirements in terms of quality, flexibility, productivity, availability and delivery reliability. This can only be achieved with digitized, largely autonomous processes that enable print shop operators to focus on their customers.

Summary

The shift from product-oriented to service-oriented business requires a rethink, especially in traditional companies in the manufacturing industry. This guide for the development of Smart Services supported by Artificial Intelligence is intended to illustrate the complexity and thus improve their handling, supporting the planning and modelling of a Smart Service. The Design Thinking process is important for understanding the challenges associated with the reorientation of the business and in designing appropriate solutions. Building on the Design Thinking process the proactive development, revision or complete redesign of their business models is an important part of the rethinking process of companies, that want to remain competitive and take advantage of the opportunities offered by IoT. The concept of agile service engineering as a management tool to manage the Smart Service can be understood as a link between the design thinking process and business model development.

Fig. 1 .

 1 Fig. 1. The 3 essential components in the development of Smart Services with their respective phases

Fig. 2 .

 2 Fig. 2. Concept of Smart Service "Heidelberg Prinect" [10]

The development of new business models is based on innovative, smart products and services, which will increase added value and thus significant growth for companies in the future. The creation of data-driven business models is the core of the development of strategies for IoT."Business model" should be understood as the mapping of the performance system of a company or part of a company. In a simplified form, it describes how the company creates customer value through saleable products and services, what resources it needs to do so and how the service is provided[START_REF] Stich | Digitale Dienstleistungsinnovationen Smart Services agil und kundenorientiert entwickeln[END_REF]. If the products are "smart" and can collect data, this also means that companies can offer their customers added value beyond the actual product benefits: Smart services can be created in this way. As a result, business models are no longer limited to a pure product orientation, but are also data and service driven[START_REF]Smart Service Welt -Umsetzungsempfehlungen für das Zukunftsprojekt Internet-basierte Dienste für die Wirtschaft[END_REF][START_REF]Existenzgründungsportal des BMWi[END_REF][START_REF] Dhar | Big data, data science, and analytics: The opportunity and challenge for IS research[END_REF][START_REF] Emmrich | Geschäftsmodell-Innovation durch Industrie 4.0: chancen und Risiken für den Maschinen-und Anlagenbau[END_REF] When developing new business models in IoT, there are various options for companies, especially in the relationship between existing or newly developed products and the associated Smart Services, which are briefly described below. For introduction, three possible options of business models are presented here, which are particularly suitable for SMEs and are easy to implement.

Acknowledgements. This work has been partly funded by the German Federal Ministry of Education and Research (BMBF) through the Project "SmARtPlaS (No. 02K18D112), the project "DigiLab NPO -Digitallabor für Non-Profit-Organisationen 4.0" (No. 02L18A230) and the project "VARIETY", founded by the French Goverment. The authors wish to acknowledge the Commission, the Ministry and all the project partners for their contribution.


Xuehong Gao

Xuefeng Jin

A Robust Multi-Commodity Rebalancing Process in Humanitarian Logistics

Keywords: Robust optimization, Nonlinear, Mathematical programming, Humanitarian logistics

After disasters occurred, many refugees have to suffer a lot. To relieve this detrimental situation, various commodities are distributed to the pre-determined warehouses. However, the initial multi-commodity distribution may be imperfect, which results in some warehouses having surplus commodities compared to other unmet warehouses. Hence, it is necessary to rebalance commodities among those warehouses. Because of the uncertain environment after a disaster, the demand is usually uncertain. To plan this multi-commodity rebalancing process appropriately, it is usually assumed that the collected data are uncontaminated. However, this assumption can be easily violated due to the uncertain environment or human error, which results in the biased estimation of the solution. In this study, we propose a strategy for remedying the difficulties associated with data contamination so that a set of robust decisions are obtained. Through a case study, we show that the proposed strategy facilitates effective decision-making in the multi-commodity rebalancing when the data contamination is involved.

Introduction

Based on the Emergency Event Database (EM-DAT, www.emdat.be), large-scale natural or man-made disasters had occurred frequently since the 1900s. Recently, largescale disasters such as the 2008 Wenchuan Earthquake in China, 2011 Earthquake and Tsunami in Japan, 2018 Earthquake and Tsunami in Indonesia, 2019 Forest Fire in Australia, 2019 COVID in the world, and 2019 Plague of Locusts from Kenya result in a great number of refugees. To relieve this detrimental situation, medical and living commodities should be distributed to and stocked at the pre-determined warehouses.

As the initial multi-commodity distribution happened before the disasters, it is impossible to match with the practical situation, which results in some warehouses having surplus commodities compared to other unmet warehouses. The undersupply of commodities worsens human suffering and results in an increased mortality rate. In contrast, the oversupply of commodities may also occur in some other affected areas. To make full use of any surplus commodities, it is necessary to conduct a multi-commodity rebalancing process to rebalance the commodities among the warehouses. However, the collected information about the demand may be inaccurate during the critical hours right after disasters [START_REF] Kirac | The traveling salesman problem with imperfect information with application in disaster relief tour planning[END_REF]. Also, the collected data may be contaminated due to human error. The above two factors result in a biased rebalancing strategy. Neither strategic approaches nor quantitative models to handle this multi-commodity rebalancing under data contamination. Thus, we investigate the multi-commodity rebalancing problem with data contamination and propose a robust optimization model for remedying the data contamination so that a set of robust decisions can be obtained.

The remainder of this paper is organized as follows. Section 2 reviews previous studies about commodity rebalancing (also referred to as redistribution) in humanitarian logistics, highlighting the novelty of this study. In Section 3, we present the problem description and propose a strategy for remedying the problem with data contamination. Then a mixed-integer nonlinear programming (MINP) model is formulated for the problem. Next, a linearization approach is applied to linearize the model. In Section 4, we compare different methods in a case study. Finally, Section 5 concludes this study with contributions and further directions.

Literature review

The logistics related problem in disaster response has been extensively studied by many researchers [START_REF] Pérez-Rodríguez | Inventory-allocation distribution models for postdisaster humanitarian logistics with explicit consideration of deprivation costs[END_REF][START_REF] Chen | The regional cooperation-based warehouse location problem for relief supplies[END_REF][START_REF] Gao | A Hybrid Genetic Algorithm for Multi-emergency Medical Service Center Location-allocation Problem in Disaster Response[END_REF][START_REF] Gao | A Robust Two-Stage Transit-based Evacuation Model for Large-Scale Disaster Response[END_REF][START_REF] Besiou | Humanitarian Operations: A World of Opportunity for Relevant and Impactful Research[END_REF]. However, the commodity rebalancing is also an important part in relief supply chain planning [START_REF] Gao | A bi-level stochastic optimization model for multi-commodity rebalancing under uncertainty in disaster response[END_REF][START_REF] Gao | A Stochastic Optimization Model for Commodity Rebalancing Under Traffic Congestion in Disaster Response[END_REF]. However, this commodity rebalancing has been received insufficient attention in the past. In what follows, we review the studies about commodity rebalancing in disaster response. Then the research gap is discussed and the novelty of this study is summarized.

To the best of our knowledge, the commodity redistribution was firstly proposed by Lubashevskiy [START_REF] Lubashevskiy | Resource Redistribution Method For Short-Term Recovery Of Society After Large-Scale Disasters[END_REF][START_REF] Lubashevskiy | Resource Redistribution under Lack of Information: Short-Term Recovery after Large Scale Disasters[END_REF] who implemented the required redistribution of vital resources between the affected and neighboring cities in the disaster area. However, they did not consider any uncertainty. Then Gao and Lee [START_REF] Gao | A stochastic programming model for multi-commodity redistribution planning in disaster response[END_REF] considered a multi-commodity redistribution problem under demand uncertainty. Based on the above study, Gao and Lee [START_REF] Gao | A two-stage stochastic programming model for commodity redistribution under uncertainty in disaster response[END_REF] considered the multi-commodity redistribution process happens in a multi-modal transportation system when multiple disaster areas are involved. Sarma, Das [START_REF] Sarma | Redistribution for cost minimization in disaster management under uncertainty with trapezoidal neutrosophic number[END_REF] and Sarma, Das [START_REF] Sarma | An optimal redistribution plan considering aftermath disruption in disaster management[END_REF] also introduced different mathematical models for humanitarian logistic applying the fact of redistribution of resources. After that, Gao [START_REF] Gao | A Stochastic Optimization Model for Commodity Rebalancing Under Traffic Congestion in Disaster Response[END_REF] and Gao [START_REF] Gao | A Bi-Level Stochastic Optimization Model for Commodity Rebalancing in Humanitarian Operations[END_REF] modified the terminology from "multi-commodity redistribution" to "multi-commodity rebalancing" as its appropriate and suitable description to the problem. Later Gao [START_REF] Gao | A bi-level stochastic optimization model for multi-commodity rebalancing under uncertainty in disaster response[END_REF] defined the multi-commodity rebalancing in disaster response, which is given by "The multi-commodity rebalancing process in disaster response is to rebalance the commodities from the oversupplied nodes to unmet nodes over the transportation network to satisfy the potential demand at all nodes (relief centers)". And the relief centers were divided into three groups, namely (i) complete supply relief centers, (ii) complete demand relief centers, and (iii) potential demand or supply relief centers, where the multi-commodity rebalancing process was conducted among these three groups of relief centers.

As the collected data is easily contaminated due to human error and inaccurate information right after disasters, it results in a biased rebalancing strategy. For more details on the data contamination and its effect, one should refer to Park [START_REF] Park | Determination of the joint confidence region of the optimal operating conditions in robust design by the bootstrap technique[END_REF]. However, neither strategic approaches nor quantitative models to handle this multi-commodity rebalancing under data contamination. To address this challenge and fill this research gap, a robust approach that is less sensitive to the outliers or noises is warranted. As a consequence, we investigate the multi-commodity rebalancing problem under data contamination and propose a strategy for remedying the difficulties associated with data contamination so that a set of robust decisions can be obtained.

3

Robust optimization model

Problem description

Suppose that a large-scale disaster strikes an area where a set of warehouses have been pre-identified to stock the commodities. These stocked commodities are used to support basic lives. Inevitably, some warehouses end up having surplus commodities, whereas others end up having insufficient commodities. To reduce human suffering and make full use of any surplus commodities, it is necessary to plan a multi-commodity rebalancing strategy to rebalance the commodities among the warehouses.

The complex and dynamic nature of a large-scale disaster creates a highly uncertain environment. It is difficult to determine how much of a particular commodity a warehouse will require in the future, which makes the demand uncertain. Here, the demand is classified into several independent scenarios. The uncertain demand is represented as a set of discrete stochastic quantities, and a specific realization is considered as a scenario. For each commodity type, there is a set of scenarios 𝛯. For a particular scenario 𝜉 ∈ 𝛯 , there is a probability of occurrence 𝑝(𝜉) such that 𝑝(𝜉) ≥ 0 and ∑ 𝑝(𝜉) = 1 𝜉∈𝛯 . As the inaccurate information usually exists, the demand quantity is considered as the outlier in this study. What we need to is identifying these warehouses and determine the outgoing and incoming shipments at the warehouses. Before the model is proposed, an assumption is stated in advance. Each of the warehouses is a separate unit and the weighted values of the warehouses are given.

Notations

The

Robust strategy

To copy with the uncertain demand, many researchers use expected demand [START_REF] Gao | A bi-level stochastic optimization model for multi-commodity rebalancing under uncertainty in disaster response[END_REF][START_REF] Laporte | An integer L-shaped algorithm for the capacitated vehicle routing problem with stochastic demands[END_REF][START_REF] Liao | A multi-objective supplier selection model under stochastic demand conditions[END_REF]. Given a set of discrete demand quantities 𝐷 𝑟𝑡

The above-expected demand can be considered as the weighted mean. However, the weighted mean is not a robust outlier-resistant location estimator. As the median is a robust outlier-resistant location estimator [START_REF] Hampel | Robust statistics: the approach based on influence functions[END_REF], instead of using weighted mean, we use the median to represent the demand at warehouse 𝑟, which is denoted by 𝕄(𝐷 𝑟𝑡 𝜉 , 𝑃 𝑟𝑡 𝜉 , 𝜉)

and given by

𝕄(𝐷 𝑟𝑡 𝜉 , 𝜉) = Median(𝐷 𝑟𝑡 1 , 𝐷 𝑟𝑡 2 , … , 𝐷 𝑟𝑡 𝑁 ) (2)

Robust optimization model

The problem is formulated as the following MINP model.

Min 𝛹 1 = ∑ ∑ 𝑊 𝑟 [Median(𝐷 𝑟𝑡 1 , 𝐷 𝑟𝑡 2 , … , 𝐷 𝑟𝑡 𝑁 ) -(𝑆 𝑟𝑡 + 𝑞𝑖 𝑟𝑡 -𝑞𝑜 𝑟𝑡 )] 𝑟∈ℛ 𝑡∈𝒯 (3) s.t. ∑ 𝑞𝑖 𝑟𝑡 𝑟∈ℛ = ∑ 𝑞𝑜 𝑟𝑡 𝑟∈ℛ ∀ 𝑡 ∈ 𝒯. ( 4
)
𝑞𝑜 𝑟𝑡 ≤ 𝑆 𝑡𝑟 ∀ 𝑟 ∈ ℛ, 𝑡 ∈ 𝒯. ( 5
)
𝑞𝑖 𝑟𝑡 • 𝑞𝑜 𝑟𝑡 = 0 ∀ 𝑟 ∈ ℛ, 𝑡 ∈ 𝒯. ( 6
)
𝑆 𝑟𝑡 + 𝑞𝑖 𝑟𝑡 -𝑞𝑜 𝑟𝑡 ≤ Median(𝐷 𝑟𝑡 1 , 𝐷 𝑟𝑡 2 , … , 𝐷 𝑟𝑡 𝑁 ) ∀ 𝑟 ∈ ℛ, 𝑡 ∈ 𝒯. ( 7
)
𝑞𝑖 𝑟𝑡 and 𝑞𝑜 𝑟𝑡 are nonnegative variables ∀ 𝑟 ∈ ℛ, 𝑡 ∈ 𝒯.

The objective function (3) aims to minimize the total weighted unmet demand at warehouses. Constraint (4) ensures the balance between incoming and outgoing shipments. Constraint [START_REF] Gao | A Robust Two-Stage Transit-based Evacuation Model for Large-Scale Disaster Response[END_REF] restricts that the outgoing shipment is not greater than the stock level. Constraint [START_REF] Besiou | Humanitarian Operations: A World of Opportunity for Relevant and Impactful Research[END_REF] ensures that either outgoing or incoming shipments could happen. Constraint [START_REF] Gao | A bi-level stochastic optimization model for multi-commodity rebalancing under uncertainty in disaster response[END_REF] restricts that the commodity after rebalancing is not greater than the demand. Constraint (8) defines the decision variables.

Linearization method

The above model is nonlinear due to Constraint [START_REF] Besiou | Humanitarian Operations: A World of Opportunity for Relevant and Impactful Research[END_REF]. It is significant to propose a linearization strategy for the above MINP model so that it can be solved by using CPLEX. Consequently, we introduce a big positive value 𝑀 and two auxiliary binary variables into the model. These two auxiliary binary variables are given by

𝑖 𝑟𝑡 = { 1 if 𝑞𝑖 𝑟𝑡 > 0 0 otherwise ∀ 𝑟 ∈ ℛ, 𝑡 ∈ 𝒯. (9)
𝑗 𝑟𝑡 = { 1 if 𝑞𝑜 𝑟𝑡 > 0 0 otherwise ∀ 𝑟 ∈ ℛ, 𝑡 ∈ 𝒯. (10)
Then the MINP model can be reformulated as the following mathematical model.

Min 𝛹 1 s.t.
Constraints (4), ( 5), [START_REF] Gao | A bi-level stochastic optimization model for multi-commodity rebalancing under uncertainty in disaster response[END_REF], and ( 8)

𝑖 𝑟𝑡 + 𝑗 𝑟𝑡 ≤ 1 ∀ 𝑟 ∈ ℛ, 𝑡 ∈ 𝒯. (11)
𝑞𝑖 𝑟𝑡 ≤ 𝑖 𝑟𝑡 𝑀 ∀ 𝑟 ∈ ℛ, 𝑡 ∈ 𝒯.

𝑞𝑜 𝑟𝑡 ≤ 𝑗 𝑟𝑡 𝑀 ∀ 𝑟 ∈ ℛ, 𝑡 ∈ 𝒯.

Constraint [START_REF] Gao | A stochastic programming model for multi-commodity redistribution planning in disaster response[END_REF] guarantees that either an outgoing or incoming shipment could happen. Constraints ( 12) and ( 13) restrict the quantities of incoming and outgoing shipments, respectively.

Numerical example

This study considers a numerical example that includes eleven warehouses and two commodity types based on the study of Gao [START_REF] Gao | A bi-level stochastic optimization model for multi-commodity rebalancing under uncertainty in disaster response[END_REF]. Let the outlier value be 10000, which exists in warehouses 1, 5, and 9. The proposed mathematical model is implemented in the CPLEX (Version: 12.6). All the experiments are run on a computer with an Intel(R) Core(TM) i7-7700 CPU@3.6 GHz under the Windows 10 Pro system.

To evaluate the robustness of the proposed robust optimization model against the outliers, we compare the results for the numerical example using different models. We first present the optimal solution in Fig. 1 without considering data contamination and then compare different methods and present the results in Table . 1 when the data contamination is involved. . 1, it provides an outlier resistance behavior for commodity type 1 in different models. The solution obtained using the robust optimization model is quite close to the solution through the method in Gao [START_REF] Gao | A bi-level stochastic optimization model for multi-commodity rebalancing under uncertainty in disaster response[END_REF]. After considering the outlier, the method in Gao [START_REF] Gao | A bi-level stochastic optimization model for multi-commodity rebalancing under uncertainty in disaster response[END_REF] is strongly sensitive to the outlier, whereas the proposed model is quite stable to obtain the solution that is still quite close to the true optimal solution.

Conclusion and future study

This study focused on the multi-commodity rebalancing problem under data contamination in disaster response. We proposed a robust optimization model to formulate the problem. Then we applied a linearization method for the model so that it could be solved in CPLEX. Next, the solutions were obtained to illustrate the effectiveness of the proposed method. Finally, we compared the results of different methods in a case study to verify the outperformance of the proposed method in overcoming the outlier in the multi-commodity rebalancing problem. The limitation of this study is that the data contamination only exists in the demand for commodities.

In future work, some directions are meaningful that could be explored deeply from the following two perspectives. This study only focuses on the multi-commodity rebalancing process. However, it would be interesting to consider how to transport these commodities among the warehouses. Another future consideration is to develop a more reliable multi-commodity rebalancing by considering a multi-period process.

(a) Commodity type 1 (b) Commodity type 2 Fig. 1 .

 121 Fig. 1. Commodity rebalancing strategy for two commodity types

  parameters and decision variables used in this model are shown as follows: Quantity of outgoing commodity-type 𝑡 at warehouse 𝑟. 𝑞𝑖 𝑟𝑡 Quantity of incoming commodity-type 𝑡 at warehouse 𝑟.

	𝑃 𝑟𝑡 𝜉 𝑂 𝑟𝑡 𝜉	𝜉 . Probability of occurrence for 𝐷 𝑟𝑡 Outlier demand for commodity-type 𝑡 at warehouse 𝑟 in scenario 𝜉.
	𝑁	Number of scenarios in 𝛯.
	Decision variables
	𝑞𝑜 𝑟𝑡
	Sets
	ℛ	Set of warehouses, indexed by 𝑟 ∈ ℛ, (𝑟 ≠ 𝑐).
	ℰ	Set of commodity types, indexed by 𝑡 ∈ 𝒯.
	𝛯	Set of scenarios, indexed by 𝜉 ∈ 𝛯.
	Parameters
	𝑊 𝑟	Weight of warehouse 𝑟.
	𝑆 𝑟𝑡 𝐷 𝑟𝑡 𝜉	Stock level of commodity-type 𝑡 at warehouse 𝑟. Demand of commodity-type 𝑡 at warehouse 𝑟 in scenario 𝜉.

Table 1 .

 1 Comparison of commodity rebalancing strategies.

			Without outlier			With outlier in warehouses 1, 5, and 9
	ID	Gao [7]	Robust model	Gao [7] 𝑂 11 𝑁 = 10 4	Gao [7] 𝑂 51 𝑁 = 10 4	Gao [7] 𝑂 91 𝑁 = 10 4	Robust model
		𝑞𝑜 𝑟1	𝑞𝑖 𝑟1	𝑞𝑜 𝑟1	𝑞𝑖 𝑟1	𝑞𝑜 𝑟1	𝑞𝑖 𝑟1	𝑞𝑜 𝑟1	𝑞𝑖 𝑟1	𝑞𝑜 𝑟1	𝑞𝑖 𝑟1	𝑞𝑜 𝑟1	𝑞𝑖 𝑟1
	𝑟=1	0	89.8	0	95	0	1051.3	0	89.8	0	89.8	0	95
	𝑟=2	0	134	0	139	0	95.9	0	96.1	0	75.5	0	139
	𝑟=3	10.6	0	23	0	282	0	282	0	282	0	23	0
	𝑟=4	123.7	0	129	0	274	0	274	0	274	0	129	0
	𝑟=5	0	137.3	0	133	0	137.3	0	1098.6	0	137.3	0	133
	𝑟=6	0	34.6	0	30	0	34.6	0	34.6	0	34.6	0	30
	𝑟=7	33.5	0	35	0	285	0	285	0	285	0	35	0
	𝑟=8	49.8	0	46	0	300	0	300	0	300	0	46	0
	𝑟=9	115	0	111	0	115	0	115	0	0	866.9	111	0
	𝑟=10	0.5	0	0	4	0.5	0	0.5	0	0.5	0	0	4
	𝑟=11	62.6	0	57	0	62.6	0	62.6	0	62.6	0	57	0

Acknowledgment

This research was partially supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (NRF-2017R1A2B4004169), the China-Korea cooperation program managed by the National Natural Science Foundation of China and the NRF (NRF-2018K2A9A2A06019662), and National Natural Science Foundation of China (71861167002, 5183000192).


Giuseppe Fragapane
email: giuseppe.fragapane@ntnu.no

Hans-Henrik Hvolby

Fabio Sgarbossa

Jan Ola Strandhagen

Autonomous mobile robots in hospital logistics

Keywords: Autonomous Mobile Robots, Material Handling, Hospital, Logistics

The recent advances in technology have increased flexibility in indoor mobility and human-robot collaboration, opening up new opportunities to perform material handling tasks, especially in narrow, dynamic environments, such as hospitals. Sensing devices, powerful on-board computers, artificial intelligence, and collaborative equipment allow autonomous mobile robots (AMRs) to drive and fulfill material handling activities autonomously. In hospitals, material handling activities are widely performed manually. This study investigates five innovative applications of AMRs, highlighting their benefits compared with other material handling systems applied in hospitals, and presenting research needs. The study concludes that AMRs can support and collaborate closely with hospital personnel to increase value-added time for patient care.

Introduction

Automating material handling activities can significantly improve efficiency and productivity in the healthcare sector, since hospitals, compared to other industries, still, often perform these activities manually. A large survey conducted in the US, Canada, and Germany exposed that nursing's none-value-adding and non-nursing activities (e.g., delivery and retrieval of food trays, ancillary, or housekeeping services) consume around 40% of nurses' time [START_REF] Yen | Nurses' Time Allocation and Multitasking of Nursing Activities: A Time Motion Study[END_REF]. Many of these activities could have been accomplished by other hospital personnel or advanced material handling systems [START_REF] Aiken | Nurses' reports on hospital care in five countries[END_REF][START_REF] Kudo | Japanese pro-fessional nurses spend unnecessarily long time doing nursing assistants' tasks[END_REF][START_REF] Wong | A wardbased time study of paper and electronic documentation for recording vital sign observations[END_REF]. However, finding the appropriate material handling system and level of automation for addressing hospitals' and patients' requirements is quite challenging [START_REF] Granlund | Automation in internal logistics: strategic and operational challenges[END_REF]. The common material handling systems applied in hospitals allow low flexibility in mobility. Automated guided vehicles (AGV) cannot bypass obstacles and enter wards or departments due to safety and space concerns. Automated vacuum collection sys-tems, pneumatic tube systems, and overhead transportation systems often have only fixed pick and/or delivery stations in departments. The low flexibility in mobility makes it difficult to automate material handling in the so-called last 50 meters, referring to the distance in the department to the patient, as well as in other departments.

The need for more flexibility has pushed the development of autonomous mobile robots (AMRs). Thanks to ubiquitous sensors, powerful on-board computers, artificial intelligence (AI), and simultaneous location and mapping (SLAM) technology, AMRs can understand their operating environments and navigate in facilities without the need to define and implement reference points in advance. AMRs are often small and agile, and they show their strength in high-traffic environments and low-volume transportation. Due to AMRs' characteristics, they can access narrower areas, interact with healthcare personnel and patients, and provide more services, such as assistive activities. AMRs have shown great potential in automotive, warehousing and process industry, in which they have supported increased production flexibility and productivity [START_REF] Fragapane | Increasing flexibility and productivity in Industry 4.0 production networks with autonomous mobile robots and smart intralogistics[END_REF]. The potential of AMRs' high accessibility and flexibility in providing services has not been exploited and investigated in hospital environments.

Therefore, the aim of this paper is (I) to describe innovative material handling services and applications of AMRs in hospitals, (II) to compare them with other material handling systems in hospitals, and (III) to highlight future research needs for AMRs in hospital logistics. Two case studies have been conducted and three examples from the literature are presented to achieve these objectives.

The rest of the paper is organized as follows. The next section provides background information on traditional material handling systems in hospitals. Section three introduces and explains the cases and examples in which AMR has been applied in hospitals. The fourth section compares AMRs with other material handling systems in hospitals. The study ends with recommendations for future research areas for hospital logistics and AMRs.

2

Theoretical background

Material handling systems in hospitals

Logistics and material handling are crucial parts of the healthcare industry. Depending on the characteristics of the goods and the delivery requirements, a material handling system can be assigned to these tasks. Small objects, e.g., samples being delivered to laboratories, are mainly sent via pneumatic tube systems in hospitals due to time constraints. While automated vacuum collection systems are only used for return transportation of waste or linen, overhead transportation systems are used for both incoming and return transportation of loads of up to 15 kg to different departments. For transportation of heavier and bigger goods, AGVs have demonstrated good results in distributing high-volume goods to many pick-and-place positions and traveling long distances within hospitals without disturbing the hospital traffic [START_REF] Fragapane | Material distribution and transportation in a norwegian hospital: a case study[END_REF]. The current material handling systems in hospitals are still largely dependent on human interaction for preparing, loading/unloading, and sending the items.

In contrast, AMRs have been recently introduced to hospitals, and their applicability in hospital logistics has not been exploited. AMRs possess a wide array of small and power-efficient sensing technologies, such as integrated laser scanners, 3D cameras, accelerometers, gyroscopes, etc., which allow them to digitalize an environment. Processing the sensing data with simultaneous location and mapping technology enables an AMR to create a map of its environment and calculate its position [START_REF] Bloss | Simultaneous sensing of location and mapping for autonomous robots[END_REF]. Unlike AGV systems, in which the central unit makes all routing and thus navigation decisions for all AGVs, AMRs can plan collision-free paths, make real-time decisions to avoid collisions, and so navigate in dynamic and unpredictable environments. AI techniques such as vision systems and machine learning enable the identification and classification of obstacles. These learning techniques enable AMRs to solve complex control problems in unfamiliar, real-world environments.

The main characteristics of the aforementioned material handling systems are shown in Table 1. A recent literature review in material logistics pointed out that standards and best practices for how to transport goods in hospitals barely exist [START_REF] Volland | Material logistics in hospitals: a literature review[END_REF]. To the best of the authors' knowledge, the applicability and research needs of AMRs in hospital logistics have not been addressed.

3

Cases and examples

Case 1: Sterile instrument transportation

Sterile instruments are transported in wagons from and to departments in a closedloop logistics system by an AMR. The sterile processing department is responsible for picking and sending the sterile instruments required for medical treatment from the storage area to the departments. After usage at the department level (e.g., operating room), the goods are sent back to the central sterile processing department for cleaning, inspection, and sterilization. Hospital personnel can request, send, and track transportation via a tablet. Due to the logistics setup and the reliable transportation offered by the AMR, sterile goods can be sent Just-in-Time and demand-based on demand.

There are 10 pick and delivery stations located among five different levels in the case hospital. The AMR can open doors, use elevators, enter departments, bypass obstacles, handle dynamic environments, and reach the destination pickup-anddelivery station to deliver the sterile instruments. The AMR delivers approximately 60 wagons daily and substitutes one full-time employee.

Case 2: Personalized cancer medicine transportation

In chemotherapy treatments, patients receive personalized cancer medicine that has a short lifetime and can cost several thousand euros. The medicine is produced in the pharmaceutical department in the basement of the case hospital, while the actual treatment takes place several floors above it. Therefore, high-precision delivery, reliability, safety, and security are required for the transportation of medicine between the two departments.

The case hospital uses an AMR to transport the medicine in a locked drawer from the basement through the hospital to the final destination in the department [START_REF]MIR Homepage[END_REF]. After delivery, the AMR returns to the pharmaceutical department and waits to receive the next order. The AMR reduces the healthcare personnel's responsibility and the amount of time involved in the transportation of medicine. This helps healthcare personnel increase patient care and value-added time and thus has enabled the return of the initial investment within one year.

Example 1: AMR providing telemedicine

AMRs with teleoperating and medical equipment can be controlled by a human from a long distance and perform tasks in dynamic environments [START_REF] Papadopoulos | A systematic review of enablers and barriers to the implementation of socially assistive humanoid robots in health and social care[END_REF]. Tele-operation enables a new method of using this valuable resource and eases the high risk of infectious disease transfer. Physicians and specialists can communicate with patients and perform some of their duties at a safe distance from infected patients. A telemedicine robot with autonomous navigation technology has been introduced in hospitals in the US [12]. While the AMR moves from room to room, the physician can easily connect for patient consults and access clinical documentation, patient data, and medical imaging. Digitizing and robotizing the material handling activities will provide data for and insights into new options for process improvement.

Example 2: Assistive, collaborative AMR in the hospital laboratory

AMRs with manipulators can assist workers by interacting with humans as robotic coworkers. In other industries, AMRs are used as assistive systems and can mount several heavy parts of a car body together at different stages along the car assembly line [START_REF] Angerer | Give me a hand-The potential of mobile assistive robots in automotive logistics and assembly applications[END_REF], thus increasing both productivity and quality while simultaneously reducing fatigue levels among workers. A dual-arm mobile robot has been introduced to a US hospital laboratory [START_REF] Homepage | [END_REF]. The AMR was designed to work alongside medical staff and help complete lab workers' repetitive tasks (Fig. 4). Fig. 4. Assistive, collaborative AMR in a hospital laboratory [START_REF] Homepage | [END_REF] It can sense and navigate its way around its human coworkers autonomously and simultaneously learn to find optimized routes from one location to another. The vision for using AMRs in laboratories is that they will take over a wide range of repetitive and time-consuming activities, such as preparation of medicines, loading and unloading centrifuges, pipetting and handling liquids, and picking up and sorting test tubes.

Example 3: AMR disinfecting rooms

Hospitals must clean and disinfect rooms after usage to reduce the spread of hazardous bacteria. Hospitals are strategizing to reduce the risk of such spread and simultaneously increase awareness [START_REF] Goldmann | Strategies to prevent and control the emergence and spread of antimicrobial-resistant microorganisms in hospitals: a challenge to hospital leadership[END_REF]. The development of UV-C light systems significantly supports the destruction of bacteria. UV-C light systems need 10 min of exposure time to kill 99.99% of bacteria [START_REF]UVD Robots Homepage[END_REF]. While the equipment can often be found on wheels and is moved manually from room to room, AMRs provide a platform for autonomously transporting the equipment into and within rooms (Fig. 5a). Therefore, an AMR can cover more surfaces compared to a fixed UV-C system and reduce the exposure of hospital personnel to bacteria (Fig. 5b). Further, it can collect relevant data during the disinfection process and communicate when the room is ready for usage.

Discussion and conclusion

Unlike other material handling systems applied in hospitals, AMRs utilize AI for decision making, which significantly increases their flexibility in performing material handling activities in hospitals. AI especially supports AMRs' decision making in path and motion planning. While typically, an A* algorithm is applied in path planning to find the shortest path, AI allows analysis of the dynamic environment and reacts appropriately to congestions. Machine learning techniques can suggest optimized paths based on previous deliveries. These processes increase the reliability of precision delivery, which is crucial in hospitals since small delivery deviations can have large impacts on patient treatment. For instance, missing instruments or medicine will postpone treatments or negatively impact the health of the patient. Robust and reliable material handling systems like AMRs are crucial for transferring material flows that can highly impact patient health from manual to automated.

AI can further support interaction between humans and machines, enabling assistive or collaborative tasks. Vision-based sensing, manipulators for grabbing and handling probes, and the use of machine learning allow AMRs to learn to perform a wide variety of repetitive activities. AMRs can function as robotic coworkers in laboratories, autonomously improving both specific processes and overall performance. This allows for the reduction of workload for highly trained laboratory workers.

AMRs can deliver to the point-of-use, the patient, and so cover a wide service area. For many years, mobile robots were a virtually unimaginable and practically unacceptable solution in healthcare support. People could not associate hospitals with a production environment. The increased social acceptance of AMRs allows integration into departments and wards. AMRs can deliver to the point of use, the patient, and so cover a wide service area. The integration of AMRs as transporting, collaborating, or assisting robots allows a rethinking of logistics and material handling activities in hospitals. Therefore, future research should focus on:

Fig. 1 .

 1 Fig.1. AMR in sterile instrument logistics[START_REF]MIR Homepage[END_REF]

Fig. 2 .

 2 Fig. 2.Sending and delivery of cancer medicine with secured transportation[START_REF]MIR Homepage[END_REF]

Fig. 3 .

 3 Fig. 3. AMR providing telemedicine [12]

Fig. 5a :

 5a Fig.5a: AMR with disinfection equipment[START_REF]UVD Robots Homepage[END_REF]. 5b: Left room disinfected with stationary equipment; right room disinfected by AMR (green/blue areas symbolizes disinfection range)

Table 1 .

 1 Characteristics of automated material handling systems in hospitals

		Automated	Autonomous	Automated	Overhead	Pneumatic
		guided	mobile robot	vacuum	transporta-	tube system
		vehicle		collection	tion system
				system
	Capacity load	1-500kg	1-100kg	1-50kg	1-15kg	up to 2kg
	Transportation	1-2 km/h	3-4km/h	20-70km/h	2-3km/h	10-25km/h
	speed
	Services	Transportation	Transporta-	Only return	Transportation	Transportation
	provided		tion, collabo-	transportation
			ration, assis-
			tance, etc.
	Service points	Fixed pick and	Flexible pick	Fixed pick and	Fixed pick and	Fixed pick and
		delivery points	and delivery	delivery points	delivery points	delivery points
			points
	Navigation	Fixed guided	Autonomous	Fixed tube	Fixed guided	Fixed tube
		path	in predefined	path	path	path
			zones

Acknowledgment

This research received funding from the strategic research area of NTNU Health in 2020 at NTNU. The authors would also like to express their gratitude to the Regional Development Fund in Denmark for supporting the project "Reference architecture and open standards". Finally, the authors gratefully acknowledge the case hospitals that made it possible to carry out this study.

 What manual material handling activities should AMRs assist with or take over, and when?  How should AMRs interact with hospital personnel and patients to achieve high social acceptance and safety?  How should hospital goods be transported by AMRs to achieve high reliability, productivity, safety, and quality?  How should AMRs be integrated with AGVs and other stationary automated material handling systems?  How should AMRs be designed and planned at the strategic level in hospitals?  How can the optimal number of AMRs be determined in hospital logistics?  How should path planning and motion planning be adapted for hospital environments?