Intelligence -> A.I Agent Spawner -> A

This thesis investigates the evolution of agents within a virtual world, focusing on DNA transfer between generations, identification of significant genes and the explorations of parameters influencing survival and gene significance. To address these questions, we created an artificial life simulation within Unreal Engine 5, which mirrors real-life characteristics and behaviors of animals. The methodology involved running a genetic algorithm with binary tournament, uniform crossover and n-point mutation and analyzing the collected data to determine the most significant genes in different cases. We demonstrate that parameters such as sensory capabilities, resource availability and mutation thresholds greatly influenced species' survival and their success in the virtual environment. The difference of size, "Health" and sight capabilities was crucial for the survival of deers and their interaction with the environment.

In conclusion, this study offers an insight on evolution and evolution dynamics, denoting the influence of resource availability, competition between agents, mutation thresholds and sensory capabilities, with a substantial potential if time and resources were allocated to the research. I would like to express my sincerest gratitude to a few people that helped me achieve my thesis.

Table of content

Abstract

I -Introduction

In a world where the environment is constantly changing due to various factors such as climate change, migration to the cities, industrialization, etc…, we would like to predict or at least have an idea on how the different species that live on the globe might evolve in the future. We would also like to be able to understand how species used to be in the past and how they did evolve from their past self to their current descendant. The method we will use to try to bring an answer to this problem is to simulate the evolution system by using the concept of Artificial Life itself in a graphic engine (in our case, Unreal Engine 5). In this paper, replicating life and simulate the evolutionary system would follow a modified version of the genetic algorithm based on multiple conditions :

-the agent attribute or A.A which contains multiple statistics that we will develop more in the later of the paper. -the environment. Depending on the agent species we will evolve, we want to be able to have multiple levels that simulate different types of environments, for example : a mountain range, a cave, a forest, plains, etc… We bring a new approach to the traditional genetic algorithm by implementing different species in the same level and creating a randomness of environment with the different resources available to the agent during his artificial life. The agents that exist in the level are not at all static and live depending on goals we implement in their A.I behavior. Finally, the performance evaluation index is based on the overall performance of the simulation and the gap between reality and our virtual world.

II -Creating a real world simulation

As stated in the introduction, the purpose of this paper is to create a realistic life simulation in UE5 and study the evolution process of agents inside of it. We have to consider different factors to achieve such a result. These factors include giving the agents a physical form to interact with the environment, with other agents (predator and prey alike), being able to differentiate the agents with the help of attributes, implementing sight and hearing capabilities and automatically creating and managing resources. In the following sections, we will examine each of these factors in detail to develop a comprehensive realist life simulation.

The agent

The natural world is home to a vast array of ecosystems, blooming with a wide range of flora and fauna. To simulate these ecosystems and the organisms that inhabit them in a realistic manner, we must strive to accurately replicate their biological processes. Nevertheless, this task is challenging due to the limitations of computational power. Our objective is to create virtual models of life that closely approximate reality while maintaining optimal performance. Therefore, we have to first recreate the fauna. To be able to differentiate the real life and the virtual versions, we propose to use the term "the agent" for the replicated one.

External representation

In order to interact with its environment and other agents in the system, each agent is required to possess a physical form in the virtual world. Each agent has a specific 3D model or mesh that we brought from free sources online (like Epic Market Marketplace). This saves a lot of time and resources since we do not have to go through all the steps to make the models. Compared to "Evaluating the Models and Behaviour of 3D intelligent Virtual Animals in a Predator-Prey Relationship" by D.Richards, M. J. Jacobson [START_REF] Georgiev | Evolution, Robustness and Generality of a Team of Simple Agents with Asymmetric Morphology in Predator-Prey Pursuit Problem[END_REF], we did not model the 3D assets ourselves hence a number of polygons that could limit the computational performance for the system, especially during the calculations of collisions with other meshes in the virtual world. To avoid this problem, we use a common solution in computer graphics called "Bounding Spheres" or "Sphere capsule collision" (S.C.C). The meshes we use are often complex and irregular in shapes, making it very difficult to define their collision using multiple primitive shapes such as boxes and spheres. The S.C.C wraps a model in an invisible sphere that will be used to calculate the collision in a more simple manner than with more complex collision. The figure 1 shows the difference between the simple sphere collision compared to a complex collision using the physics asset in UE5. It is true that a more complex collision would be more precise and give the agents a bigger range of action but in the case of a real-time simulation, we sacrifice that precision for computational performance.

Internal representation

The agent now possesses an external representation that allows it to interact with the environment. However, it doesn't have the complexity that comes along with life.

Each species has abilities, faculties that separate them from others.

In a more local scope, each of the members of those species is unique and has a different value on how much those abilities express themselves. An animal's DNA contains information that will determine its abilities and how much they get expressed. Due to the limits of computational power, modeling an animal's DNA as is in a real-time virtual world would be quite challenging. Nonetheless, we want our agents to include this concept of uniqueness.

A solution we found is to create a simplified version of the DNA along with attributes that depend on that structure.

The agent's DNA

DNA is a polymer of two chains in the form of a double helix, composed of base pairs, that carries genetic information for the development, functioning, growth and reproduction of all known organisms and many viruses. The length of this structure can vary between animals. It's important to note that in our machines, the measurement unit is megabyte or gigabytes not in base pairs. Nonetheless, an equivalent exists between the two units and it is 1.000.000 base pairs (bp) = 1 Mb / 1.000.000.000 bp = 1 Gb.

A video game or a simulation usually weighs between 4Gb and 100Gb. This emphasizes the fact that we need to create a simplified version of the DNA in order to create unique agents and recreate their abilities in our world. We will discuss the implementation of the simplified DNA in a later part.

The agent's attributes

Previously, we said that an animal is defined by its DNA and the abilities that the structure codes. An example of one of those abilities is the speed of the Cheetah, which has the title of "Land's fastest animal". They are able to run extremely fast (at 70 mph or 112,654 km/h) because of a combination of physical traits and genetic adaptations such as their slender, muscular body, their enlarged nostrils and lungs (more oxygen = more energy), their long tail (maintains balance and control) and adapted claws (better traction and acceleration). We formalize those abilities, those adaptations in the form of "attributes" that possess numerical values and that depend on the simplified version of the DNA mentioned in the previous part. We created different variables that encompass all that is necessary to define an agent's abilities. Table 3 showcases all the attributes an agent has, their type and their formal description.

Type of attribute Attribute Description

Integers AgentID

The unique identifier of the agent.

Integers AgentSize

The size of the agent.

Integers Energy

The amount of energy the agent currently has.

Floats AgentSpeed

The speed at which the agent can move.

Floats AgentDamage

The amount of damage the agent can inflict on hostile agents.

Floats AgentHealth

The current health of the agent.

Floats SightRange

The maximum distance at which the agent can see other agents.

Floats

SightRadius

The radius of the agent's field of view.

Floats SightOffset

The height at which the agent's field of view is offset from its heading.

Floats HearingThreshold

The minimum sound level that the agent can detect.

Floats

SurvivabilityScore

A score indicating how well could the agent theoretically survive in the environment.

Enums AgentSpecie

The specie of the agent.

Array

DNA

The genetic information that determines the agent's traits and behavior.

Table 2 : Agent Attributes : Types and short description

For an agent to be considered as created and completely operational in the simulated environment, all those attributes that were explained in the figure 3 need to be initialized. It is crucial to initialize the attributes of an agent because they will determine their abilities as well as their behavior during the course of the experiments. Not initializing them might even cause unexpected behaviors and crashes during the simulation.

To solve this problem and automate the task, we developed an algorithm that does everything needed when we create an agent. The figure 2 shows a summary of all the different steps that need to be called during the algorithm.

Step 1 : Initialize DNA //initialize the DNA structure of the Agent

Step 2 : Initialize base attributes // initializing the SightRange, Radius, Offset and HearingThreshold.

Step 2. (Footnote 1 : Every attribute depends on the agent's real life counterpart's abilities and capabilities). (Footnote 2 : Each step of the algorithm can be seen in more detail in the appendix of the thesis via the Github link of the project.)

Birth and Death simulation

In our world, birth and death are two processes that can make the number in a population of animals grow, wither or stay the same. Birth is the way new children come to life whereas death is the way life purges populations to keep their numbers stable. This cycle of birth/death is the most important part of life since it enables the ecosystems to preserve resources and maintain life as it is. The paper by D.Richards and M.J.Jacobson defines the cycle of birth/death in their simulation of Omosa with a creation of the agent as a baby (birth) that can go through the multiple stages of life and death by old age or its health points dropping to 0 in case of a fight with a predator. Our simulation takes an entirely another approach to define the cycle of birth and death. Birth is defined by the live creation of an agent inside the level (one of the environments) or spawn for short. Death is defined by the live destruction of an agent inside the level or despawn.

Simulation of birth

As stated previously, birth is an important part of ecosystems since it maintains the number of a certain population. To simulate said process in our virtual environment, we defined an invisible box called an AgentSpawner that possesses multiple characteristics. Table 3 of the birth process variables of the AgentSpawner, their type and their formal description.

Type of attribute

Attribute Description

Integers TotalToSpawn

The number of agents to spawn.

Floats MaxSpawnRange

The maximum distance at which the spawner can spawn agents.

Floats PerlinSeed

Variable used to create a randomness for the calculation of the random displacements of X and Y.

Floats MinPerlin

The lower interval for the calculation of the random displacements of X and Y.

Floats MaxPerlin

The upper interval for the calculation of the random displacements of X and Y.

TSubclassOf<AAgent> ClassToSpawn

The exact specie to spawn for an AgentSpawner.

Table 3 : AgentSpawner : Birth process variables, their type and short description

When the correct function is called, the AgentSpawner will try to spawn a number of agents as close to TotalToSpawn. Nonetheless, the number of agents might be less compared to TotalToSpawn due to a feature of UE5, which we enabled for better computational performance.

If the location of the attempted spawn already contains an agent or another physical entity (called an actor in UE5) with collision enabled, the attempted agent spawn will fail and go to the next one.

The figure 3 shows a summary of the different steps of the spawning process.

Step 1 : Check if the TotalToSpawn is higher than 0.

Step 2 : Initialize i = 0

Step 3 : Loop until i <= TotalToSpawn

Step 3.1 : Calculate a random position with the Perlin variables on X and Y

Step 3.2 : Spawn the actor It is important to note that the spawning process for the first time, the first iteration is completely different than for the rest of the simulation. The other birth simulation will be covered in the second part of the paper.

Simulation of death

In real life, death is essential to preserve the balance in ecosystems between the number of predators and prey, the availability and scarcity of resources.

An animal can die of old age or from its wounds due to a fight with another animal, trying to defend its territory or while roaming freely in the environment.

In our virtual world, we decided on some conditions to make the simulation a little simpler to make. Our goal is not to simulate and solve the predator-prey but to see how DNA would travel between the generations. We don't create sub-populations of agents, which could lead to inner fighting over prey and resources and unnecessary deaths. Instead, all agents of the same specie are "friendly" with each other and don't have default interaction between them. The despawn of the agent would happen in two cases.

The first case is that the AgentHealth attribute drops to 0. It is possible that there was a fight between predator(s) and prey(s) or if the AgentEnergy is under a threshold for a certain time.

The second case is when the simulation passes to the next generation of agents.

The figure 4 showcases a graph that sums up the cycle of birth/death for our simulation.

The Environment

The purpose of our research is to be able to create a realistic life simulation. In the real world, animals live in a variety of ecosystems, such as forests, mountains, caves, seas, etc… Each of these ecosystems contains an array of resources that are necessary for the survival and development of each species. For example, cheetahs live in grasslands, savannas and some mountainous regions. Those environments are all open landscapes with a lot of resources (food, shelter and water) where the cheetah can use its speed to its maximum potential to chase down prey.

Based on this example, we can infer that each animal has an ideal environment where it can use to its fullest its capabilities. Therefore, to mimic reality as closely as possible, we need to try to recreate to its maximum possible the vast array of environments where all of the agent species prosper.

Initial level

Mimicking reality is a process that takes a lot of time and resources, to imagine and create the environment itself.

To be able to test our agents' fundamental features while not having our environments yet ready, we created a simple game field consisting of basic shapes provided by E.G in UE5. This level does not accurately reproduce the environments in which the agent species typically live. The figure 5 shows a screenshot of the UE5 editor view of the whole initial level.

Environment design

Multiple ecosystems nurture life as we know it. In reality, here is a chance factor that will determine where the fauna and flora will end up living.

To fully mimic an actual environment, we need to recreate that chance factor and make our worlds more realistics. Furthermore, we study the evolution of a multitude of agents. Some agent species won't share the same ecosystem because their real-life counterparts could never survive there. Hence, a need to create a multitude of ecosystems in order to cover a certain percentage of the environments the animals live in.

Simulating the resources

To infuse the simulated environments with the unpredictability of life, we take inspiration from solutions implemented by game developers over the course of video game history.

A common solution in level design to create this chance factor or randomness is to use procedural foliage. It would be a good idea if we didn't want to add randomness to each restart of the simulation. We believe that making the environment more random at start might change the way DNA changes between generations of agents.

Unfortunately, there is no free procedural foliage library on the E.G marketplace, hence a need to develop a solution for ourselves.

The solution we developed is based on the AgentSpawner of part I., subsection 2, subsection. 2.1. We reused the concept of a transparent box with a physical location in the environment. Although the AgentSpawner architecture makes it only able to spawn one kind of A.S, the EnvironmentSpawner is able to spawn a vast array of special actors called EnvironmentActor. An EnvironmentActor is an spawnable or placeable actor (depending on the needs) in the level that has multiple variables required to simulate how its real counterpart would behave (while being static).

All the variables, their type and a short description of an EnvironmentActor are listed in table 4 hereafter.

Type of attribute Attribute Description

EnvironmentAttribute ActorAttributes

The variable that contains some of the variables needed for the class.

Enum ActorType

The type of resource that the actor represents (example : tree, grass...).

Enum ActorType

The status of the actor (Has resource, is regrowing).

Floats

RenewalTime Time needed for the resource to renew.

Floats PerlinSeed

Variable used to create a randomness for the calculation of the random displacements of X and Y.

Floats MinPerlin

The lower interval for the calculation of the random displacements of X and Y.

Floats MaxPerlin

The upper interval for the calculation of the random displacements of X and Y.

CapsuleComponent Capsule

The capsule that gives an EnvironmentActor a simpler collision with the environment.

Table 4 : EnvironmentActor : variables, their type and short description

All the meshes that we use for the EnvironmentActor come from the megascans collection in Quixel Bridge (free with UE5).

Figure 6 shows two screenshots of the initial level from figure 8 with EnvironmentActors (3 types of trees and 3 types of grass) spawned in the entire MaxSpawnRange.

Multiple environments

Creating different biomes from scratch is a colossal task. To solve this, we could go different ways. We used two solutions to create the different biomes based on what we wanted to achieve with them. The first biome is composed of an abstract mountain range that creates a natural barrier, protecting the agents from going into zones they shouldn't go to.

The level is divided between tinier biomes, a wide plain with a spawnable random forest, another wide plain with less plants to make it more open and finally a cave next to the 0 level of the mountain. The sub-biomes are separated with invisible walls that disable the agents from interfering in environments they shouldn't be in. This biome serves as a good training ground to observe the agents' behavior in a closed environment with 0 possibility for the agents. The other biomes were created using the height map technique. The purpose of using technique is to accurately represent reality and the different elevations of a terrain. We used online tools to obtain height map data for different real life locations that could accurately represent environments where our animals live.

To further the sense of realism in our environment, we would like to paint the surface of the environment in a procedural fashion, based on the heights of each pixel. This method, called "Landscape Auto Material" is a common method in level design, especially to generate countless worlds while keeping an artistic print that is proper to each video game. Figure 7 showcases the capabilities of the Unreal Engine 5 in terms of environmental realism.

Predator and prey system

Real-life animals need food to survive, sustain themselves and reproduce in their environments. They could have carnivore, herbivore or omnivore diets. Carnivores are animals (or plants) that have evolved and have adapted to only gain nutrients through animal tissues (muscle, fat or other soft tissues).

Herbivores are species that have adapted to be able to eat and digest plant based materials (leafs, algaes, etc…) whereas omnivores are a combination of carnivores and herbivores (for example, humans).

As we try to reproduce their behavior in virtual worlds. We recreate the diet plan of each agent species.

Our solution was inspired from the paper Evolution, Robustness and Generality of a Team of Simple Agents with Asymmetric Morphology in Predator-Prey Pursuit Problem by M. Georgiev, I. Tanev, K. Shimohara and T. Ray in 2019. Both types of agents (predator and prey) in the paper possess the capability to get information from their environment and act based on their perceptions, though in different manners. The predator agent can only see in a straight line with a high range of sight while the prey can see in all directions with a lower sight range to not give it an unfair advantage.

Agent's perceptions

The paper's agents use sensors to provide the perception information to the robots. Without a perception system to detect hostile agents or resources, an artificial life simulation would not be complete.

The simulation in the paper by D.Richards and M.J.Jacobson collected "the total population, numbers of births and deaths for both prey and predators as well as the number of predator kills and prey deaths from old age" with different settings (predator/prey awareness, flocking, herding). When the predator class isn't aware of the prey, it fails to function properly and when the prey class isn't aware of the predator, it quickly becomes extinct.

Unreal Engine 5 provides some systems to implement senses in the game but after careful consideration, we decided to implement our own sense of sight. We use "Multi Object Sphere Trace" x times per second, which consists of sweeping a view range and detecting objects in the agent's field of view. This would compensate for the short-comings of the UE5 powered systems and enable herbivore and omnivore agents to detect sources of food as well as enemies.

All agents share the same set of attributes, including SightRange, SightRadius, SightOffset and HearingThreshold for the senses configuration. Nevertheless, we define how much an agent's senses are expressed based on the real-life counterpart's known ability and a bonus coming from DNA structure. Table 5 is the DNA structure with each gene representing a defined attribute.

Table 5 : Agent's DNA structure

The architecture of the DNA bonus is a bonus that is calculated (randomly between 0 and x) and added to the value of the attribute only if the gene is equal to 1. This could either advantage or disadvantage an agent depending on how its perceptions would affect its actions.

Agent's actions and decision making

Our perception system reacts to the stimuli around it (in a limited range). The agent is an A.I powered actor in the level that has predefined actions based on what the agent's perception system would register from the environment.

If the stimuli detected comes from a hostile agent (H.A), the agent could react via two distinct ways. If the H.A is one of its predators, the agent would run away from it until reaching a safe location where the perception system doesn't detect the H.A anymore.

In the case of the H.A being a prey, the agent would chase it until reaching a radius called the CombatRadius where it could start attacking it until one of them dies.

To illustrate how this would work in the simulation, we will follow the example of our Deer and Fox since foxes are natural predators to deers. Both possess different sight senses and hearing senses. If a deer manages to detect a fox before getting detected (because of better sight), it would run away from its predator.

If a fox detects a deer nearby, it would chase the deer until being able to fight its prey in its defined combat radius.

In addition to the predator-prey pursuit and combat system, an agent would also be able to register what EnvironmentActors exist in its field of view. It would calculate automatically the closest resource and move in its direction in order to consume its content.

Figure 8 shows a graph summarizing how an agent can register perceptions and the decision making it can make.

Figure 8 : AI behavior, perceptions and decision making

To conclude this part of the essay, it is safe to say that we managed to reproduce the different parts of an ecosystem, fauna and flora as well as the animals' behavior.

The reproduction of animals in the game was made by giving agents a physical and internal representation to help them interact with the world, while simultaneously remaking the cycle of birth and death with the AgentSpawner and destruction functions. We made the environment with heightmaps and artistic techniques and recreated the unpredictability of resources location with perlin noise. Finally, we created a system to reproduce an agent's sight, hearing being too complex to add on our own for now.

III -Agent's evolution

Evolution is the process over which heritable characteristics change in a certain population during the flow of time.

The purpose of the paper is to study how our agents will evolve over the course of time and multiple generations, or tuning. Unfortunately, a problem such as this could not be solved using classical computing methods. Therefore, we follow the work of John Holland in his 1975 paper Adaptation in Natural and Artificial Systems with his well defined Genetic Algorithm.

In this part, we will view how this algorithm works, the different processes involved in it and how we remade them to fit our needs.

Overview of the genetic algorithm 1.Pseudo-code

A genetic algorithm or G.A is an algorithm that is used to tune certain traits in populations and arrive at a certain solution, best or accepted in a certain amount of time. It has the advantage of being able to look for solutions inside a wide solution space without needing a lot of information and being able to maintain a good performance.

Figure 9 shows the different steps needed to run a regular G.A.

Step 1: Initialize the first generation //in our case, we would spawn the first generation of agents

Step 2 : Evaluate the given generation based on a fitness function //we usually give a fitness value to a solution and evaluate it based on it //if the first generation is the solution, stop the algorithm and return solution

Step 3 : Selection of parents //could be realized using different methods

Step 4 : Crossover of the parents genetic representation and creation of children // also called offspring

Step 5 : Mutation of the offspring's genes //following a mutation chance

Step 6 : Check if we reached the termination condition.

Step 6.1 : If no, repeat from 2 with the Offspring population Step 6.2 : else, stop execution and print the found solution The G.A that we will develop for our simulation takes its base from the algorithm presented in figure 13 while bringing different approaches to the evaluation method and the genetic operations in Step 3, 4 and 5.

Genetic representation

The reason why some problems require G.As is because the problem is way too complex to be solved using traditional methods. As stated in the introduction, these types of algorithms are commonly wielded to find solutions in a large search space and maintain a good execution time. However, G.As have a technical limitation. We have to represent all possible solutions of our problem with a linear representation. (for example, an array or a list)

In consequence, we have to represent our DNA in a linear representation because it is the variable that we want to evolve. An agent's DNA is an array of 8 integers that can only contain 0 and 1. We could fill it with numbers from R (real numbers) instead of just the {0, 1} set. Putting real numbers inside the DNA instead of 0 and 1 could possibly cause more harm than good because of two main reasons:

-Truly random numbers are impossible to generate based on the deterministic nature of the algorithm used to create them.

-It would make the solution space go from 2^8 (256) to infinite. The real-life counterpart of DNA can only take 4 possible values, which are adenine (A), cytosine (C), guanine (G), and thymine (T). These bases form specific pairs (A with T, and G with C), which makes the possibilities even lower. A possibility would be to change the set {0, 1} to {0, 1, 2, 3} but that would be something to discuss more in the fourth part of the thesis.

Fitness function

Without a way to give a score to possible solutions, the algorithm fails to uphold its promises. It is the reason why we implement a fitness function, which is simply a function that returns how close a solution is to the optimal solution of a problem.

At the evaluation time in our simulation, we use the survivability time of alive agents as a fitness function.

The survivability score is calculated based on the formula (1). 𝑆𝑢𝑟𝑣𝑖𝑣𝑎𝑏𝑖𝑙𝑖𝑡𝑦𝑆𝑐𝑜𝑟𝑒 = (1. 5𝑓 * 𝑙𝑜𝑔2_𝑠𝑝𝑒𝑒𝑑 + 3. 𝑓 * (𝑙𝑜𝑔10_𝑠𝑖𝑧𝑒 + 𝑙𝑜𝑔10_ℎ𝑒𝑎𝑙𝑡ℎ) + (1)

5𝑓 * 𝐷𝑎𝑚𝑎𝑔𝑒) + 𝑆𝑒𝑛𝑠𝑒𝐵𝑜𝑛𝑢𝑠;

It takes into account all attributes of an agent that enable it to interact with its environment while scaling it down to give agents with a lower score a chance. The different constants {1.5f, 3.5f, 3.5} were chosen and assigned a weight to underline the importance of each value into the hypothetical chance of survival. We also added a SenseBonus to the equation of the SurvivabilityScore since it's theoretically accurate to hypothesize on the fact that better senses is equal to a survival chance for an agent. The SenseBonus is calculated by formula (2). 𝑆𝑒𝑛𝑠𝑒𝐵𝑜𝑛𝑢𝑠 = (𝑆𝑖𝑔ℎ𝑡𝑅𝑎𝑛𝑔𝑒 / 𝑆𝑖𝑔ℎ𝑡𝑅𝑎𝑑𝑖𝑢𝑠) * 𝑆𝑖𝑔ℎ𝑡𝑂𝑓𝑓𝑠𝑒𝑡 * (1. 𝑓 / 𝐻𝑒𝑎𝑟𝑖𝑛𝑔𝑇ℎ𝑟𝑒𝑠ℎ𝑜𝑙𝑑);

(2)

The ratio is used to perform the sphere trace function (𝑆𝑖𝑔ℎ𝑡𝑅𝑎𝑛𝑔𝑒 / 𝑆𝑖𝑔ℎ𝑡𝑅𝑎𝑑𝑖𝑢𝑠) described in part I, subpart 4, subpart 4.1. A high ratio indicates a wide field of vision. However, a larger radius will increase the time needed to perform the trace, leading to a lower bonus. The higher the SightOffset is, the higher the bonus will be. This takes root in the fact that height is an advantage that can lead to agents being able to detect enemies as they approach as well as find resources easily. Finally, the is such as when an agent's hearing ability is (1. 𝑓 / 𝐻𝑒𝑎𝑟𝑖𝑛𝑔𝑇ℎ𝑟𝑒𝑠ℎ𝑜𝑙𝑑) good, the higher the bonus will be.

Termination criteria

If a termination criteria is not implemented for a G.A, it will continue to run indefinitely because it wouldn't know what solution to look for in a given problem. In a problem such as ours without "conventional solutions", a termination criteria could probably be the number of generations, which could be changed between runs). The reason for such criteria is because of the intent behind our study. How genetic information would get transferred between generations and which combinations were the most interesting to keep by nature's law ? To avoid having the same solution over and over again, It could also be a good idea to implement a convergent protection. That means to stop running the algorithm if the N+1 generation's DNA set is very close to the N generation's DNA set.

Genetic operations

When new offspring are created in our world via reproduction, they undergo multiple stages and multiple processes until reaching birth. The real-life processes are vastly numbered and not the subject of this paper. However, the G.A is rendered useless if some of those processes are not reproduced in a virtual environment. They are called selection, crossover and mutation in a genetic algorithm. The goal of this second part is to focus on all the genetic operations that are essential to the GA's flow (seen in figure 13).

Binary tournament

After the evaluation is done, we have to select suitable parents for the crossover (C.R.O) step (step 4 in figure 13). In classical genetic algorithms, there are two methods to select parents. The "roulette wheel" method, which follows the "survival of the fittest" concept and consists of giving a probability of choosing a parent based on the ratio of its fitness to the sum of all fitnesses in a generation. This raises an issue of selecting only the best solutions in a set, which might lead to getting a suboptimal or not optimal solution for a given problem. The other method is called the binary tournament. It simply consists of taking two random solutions from the set and comparing the return value of their fitness function. The solution with the higher fitness will end up winning and get chosen. The paper On Evolutionary Exploration and Exploitation (1998) by A. E. Eiben and C.A. Schippers describes the selection process as a way to exploit current solutions and improve their fitness function. It is a very good solution for our problem but we changed its implementation to suit our need of having two parents/ two children and maintaining the population's numbers. Figure 10 shows a simplified version of our binary tournament.

Step 1 : Get all the agents inside the AgentSpawner //using collision detected in UE5

Step 1.1 : If the agent's specie is equal to the specie that the Spawner uses

Step

Uniform crossover

In real-life, crossover would happen when same type chromosomes meet during meiosis. Both can switch different parts when they are lined up. In our virtual world, we will avoid this method since our chromosomes do not have the same length as the real life DNA. C.R.O is crucial to G.As per [3], it explores the set of solutions and looks for new ones that weren't available beforehand. In our paper, we shall use the uniform crossover. It separates the linear representation we evolve into n bits and randomly gives one of the parents' bits to the first child and the other parent's bit to the second child (based on a uniformly generated number( see documentation of UE5 in bibliography)). Therefore, such a method ensures we have explored more solutions. If we only had the selection process, it would mean staying in an area where the fitness might converge to a sub-optimal value. Figure 11 shows an image of the uniform crossover (credit in bibliography)

N-point mutation

Mutation in biology is described as the alteration of the DNA sequence of a bacteria, an organism, etc… It is said to be able to create changes in one's phenotype (observable characteristics). Nature of the changes are to be determined during the life of the individual whose DNA has changed. In traditional genetic algorithms, there are a few methods to reproduce the M.U.T step (in a binary linear representation) such as the random mutation, which randomly selects a bit and changes its value, the swap mutation which selects two genes randomly and switches their value. Figure 12 shows a side-by-side comparison of the random mutation and the swap mutation in a binary array. Step 1 : Get an agent's DNA

Step 2 : Loop over the length of the DNA //in our case, 8 times

Step 2.1 ; Generate a uniformly distributed float between 0 and 1

Step 2.2 : If the generated float > 0,79, change the value of the bit

Step 2.3 : Else, go to the next bit //we go to the next case of the DNA Step 3 : Repeat until no more bits to modify

Figure 13 : N-point mutation algorithm

Keeping the threshold to a high value (0.79) helps us create a balance between creating new genetic representations for the offspring (exploration of the solution search space) and preserving the DNA structure from an unnatural disruption.

To conclude this part of the thesis, we used a derivation of John Holland's genetic algorithm (1975) to be able to fine tune and observe how a DNA structure would evolve through time and a limited amount of generations. Our agents' DNA are represented by a linear and binary array of size 8, which could be changed. We evaluate them with a SurvivabilityScore that depends on their theoretical aptitude to survive in their environment, select them via binary tournament and reorganize them in an array for crossover. This ensures a protection from premature convergence to a suboptimal solution. In addition to that, we apply the n-point crossover method as well as the derived n-point mutation with the hope of discovering more solutions during our run. Finally, we could bridge the gap between reality and our world by changing the representation of the DNA to a linear array with {0, 1, 2, 3} as the possible genes instead of a binary array.

IV -Results and further possibilities

Creating the simulation was half of the process. The other half is to run some experiments with it. In order to obtain results with our G.A, we created a code which saves the agent species, survivability score, generation and DNA structure. In the following sections, we will analyze and interpret the data, discuss the results, and draw some observations. We would also like to define the next possible steps for this research, keep what is positive and understand where the simulation could be improved.

Analysis of the results

The protocol is quite simple for the analysis of the results. We would like to understand the significance of the genes for each scenario and how changing certain parameters (M.U.T threshold and number of E.A) affects the results. All the graphs and tables that are presented in this part are the work of Rhiannon Follenfant, who generously helped us and lended us her expertise in statistics.

First runs and results

These runs were made with the first formula used to calculate A.A during the simulation, the M.U.T threshold was kept at 0.79 and we did runs for two cases :

-75 trees, 150 grass and 25 water bottles (75t) -150 trees, 300 grass and 25 water bottles (150t) Figure 14 showcases graphs of the 75t case for the deer species and the fox species.

Figure 14: Fox and deer populations : Gene comparison and individuals evolution (75t)

Figure 15 showcases graphs of the 150t case for the deer species and the fox species.

Figure 15: Fox and deer populations : Gene comparison and individuals evolution (150t)

These graphs can be interpreted in a simple way. In both 75t and 150t, the genes that have the most gene count/individual (g.c/l) are the AgentSize and the AgentSpeed. The AgentSize being significant does make sense, especially knowing that the attribute is the base for all other "AgentX" attributes formulas in our code. However, AgentSpeed being significant doesn't make sense in our data.

In fact, agents do not "move", they teleport from position A to B because of how the simulation is constructed. Hence why, AgentSpeed shouldn't hold this much gene count/individual during the 50 generations. Moreover, it's important to notice that the change in E.A numbers doesn't change that much for deers.

In fact, the number of survivors stays the same (24 individuals by generation 50). It can be explained by the fact that the number of deers is important enough to survive in the test environment, reproduce and keep the population at this threshold. The number of individuals is also not high enough, ensuring that all individuals thrive in their environment and keeping deers from competing for a limited resource (75t case).

For the fox species, their population seems to be less endangered during the 150t case. The abundance of E.A in the 150t case enabled them to survive more, while competing with deers for resources and chasing them since foxes hunt deers in the simulation.

To conclude this subsection, we can say that the number of E.A does influence the ecological dynamics of agents as well as their survival, making it easier for "weaker" species to thrive and keep their population number stable as seen in the 150t case for the foxes. However, the formula used to calculate the A.A seems to be flawed as the gene count/individual was not really the expected one. Nevertheless, the mathematical models seem to hold itself vis a vis the ecological dynamics in our simulation.

In the next subsection, we will run the simulation again by changing the formulas used to calculate the A.A and changing the M.U.T threshold parameter.

Second runs and results

These runs were made by changing the formulas and making the genes more significant for each attribute, and we did runs for four cases : -75t and 150t with a M.U.T threshold of 0.79 -75t and 150t with a M.U.T threshold of 0.83 We hope to see if making mutation a rarer occurrence would make it easier for the population to stay stable or if it would make it harder to survive. Figure 16 showcases graphs of the 75t case for the deer species and the fox species with the M.U.T threshold of 0.79. In the 75t -0.79 M.U.T graph, the deer species seems to be thriving with a count of approximately 26 agents by the last generation. We also observe an augmentation of population as well as a higher count of deers individuals possessing the 3 genes for Sight just before generation 50. However, we also notice that, at the same time, the gene for Hearing gains a higher g.c/l too. A hypothesis could be that the M.U.T threshold is too low and makes gene mutation too common in this case.

The fox species doesn't seem to be thriving compared to the deer species as their final numbers decreased by 72% (25 agents to 7 agents). We can interpret this as the result of competitiveness between the species and the foxes themselves (inter and intra species competitiveness). Furthermore, we note a decrease in population multiple times as the count of sight genes drops in the foxes' generation DNA. By generation 41, the number of individuals seems to stabilize, as the number of E.A always stays the same and the number of deers also decreases, leading to a better distribution of resources between all the agents in the 75t case.

In the 150t case, deer population count got higher by approximately 18.51% and fox population count decreased by 28.6%. Sight genes also seem to hold a high importance for survival as their decrease often leads to certain deaths in the population as noticed with these examples :

-decrease of sight range g.c/l at generation 8 for deers.

-decrease of sight radius g.c/l before generation 25 for foxes. We can distinguish that fluctuation in Hearing Threshold g.c/l doesn't really influence survival of deers or foxes. A possible reason for that is that hearing sense is not implemented in the simulation. Another reason for the foxes' failure to survive in the environment, even with a higher number of E.A, could be their initial random positions on the map during spawning as well as the initial random positions of E.A each two generations. Figure 18 showcases graphs of the 75t case for the deer species and the fox species with the M.U.T threshold of 0.83. In the 75t -0.83M.U.T case, the deer species doesn't thrive as well as their 75t -0.79M.U.T counterpart. Their final population numbers decreased by approximately 25.93%. Gene count/individual pattern seems to be close to the 75t-0.79M.U.T case, except for the Hearing Threshold gene, whose g.c/l decreased drastically because of the lower mutation rate. The fox species seems to be thriving more in this case than their 0.79M.U.T counterpart. In fact, the hypothesis that sight genes count/individual and survival rate was linked tend to be validated by our data as we can see in Figure 18. In generation 10, we had an increase of ≈14.3% for the fox population as well as an increase of 23.52% of Sight Range gc/l. For the 150t -0.83M.U.T case, the g.c/l pattern stays the same as its 75t counterpart and both species tend to thrive even more (5% pop. increase for deers and ≈33.3% pop. increase for foxes). This sudden growth of both populations could be theorized as the effect of changing the number of E.A as well as their placement in the level.

To conclude this subsection, we can say that having mutation as a rarer occurrence could be beneficial for the system in its whole as it maintains balance between the two species used in the simulation. Making it a more common occurrence seems to tilt the favor to "stronger" species (example : deers) as the "weaker" species will not be able to focus on genes that should close the gap between them and the stronger species. Additionally, it could be conceivable that a higher number of resources isn't necessarily the best as their distribution in a system should be taken into consideration too. Foxes in the 75t -0.79M.U.T showcases that hypothesis.

In the next subsection, we will try to discuss more on the data, about other hypotheses and various possibilities for this research.

Discussion

The research is conducted to gain understanding about which genes are the most significant in our DNA structure, depending on the environment and other parameters. In the previous part, we analyzed the gene count/individual (g.c/l) for a few cases but we didn't focus on the significance of the genes.

Size X X x X Agent Speed X X X X Agent Damage X X X X Agent Health X x X X
Table 7: Fox and deer populations : Gene significance (second runs -0.83M.U.T)

In both runs, we can note that AgentSize, Speed, Damage and Health are the most significant. It can be explained as all those define how an agent will survive in its environment, find resources and fare against its enemies. A real life example could be the gazelle and the lion. A lion might be stronger than a gazelle but a gazelle can outrun a lion on long distances. In the first runs, sight genes who could give a bonus to the sight sense hold less 0.001 significance. It's explained as the formulas for calculating them were flawed and didn't make them stand out compared to other A.A. In the second run for 0.83M.U.T, all sight genes significance for deers are inferior to 0.001 except for SightOffset in the deer75 (is inferior to 0.05). For the fox75, sight genes all possess a significance higher than 0.05. That could be explained by the scarcity of the resource and the competition the species has to go through with deers. Fox150, doesn't seem to possess that much significance for SightRadius (inferior to 0.001) and SightRange (inferior to 0.01) as the resources are abundant and fairly close to the foxes. The lower distance between foxes and resources could also explain why its AgentHealth significance went from X to x in table 7. Less health and less energy is needed to find resources. The Hearing Threshold gene always has a significance inferior to 0.001 (except for fox75 in 0.83M.U.T). This gene, no matter this value, will never hold a significance in this version of the simulation as the hearing sense of agents wasn't replicated. In real life, foxes have a very sensitive hearing and use it to gather information about their environment. It enables them to hear sounds from behind, better than a deer, giving them a clear advantage during hunting.

It should be precised that during all the runs, no combat happened between prey and predator as each agent teleports to its next location and never managed to get closer to its enemy. If agents moved in a traditional way, we could probably observe multiple foxes teaming up to chase one or multiple deers, cornering them and defeating them in battle. On top of that, agents shouldn't be dying as combat doesn't happen. Nevertheless, population numbers decrease as a result of an U.E5 bug where the calculation of a random position inside the navigable radius leads to the void and an automatic destruction of the falling agent by the engine. In subsection 1.2 of the third part of the thesis, we talk about genetic representation. In this thesis, we use a binary representation for the agents' DNA, to simplify operations. As stated in that part, life doesn't represent genes with a {0, 1} set but with a {C, G, A, T} set. The letters that creates the real DNA are nucleobases, which can only link in a certain way : -Adenine (A) with thymine (T) -Cytosine (C) with guanine (G) Having 8 genes with {0, 1} as the values is simple but limited, as we only have 256 possible combinations for all agents. Using a numbered set like {0, 1, 2, 3} could be the solution as it would make the number of possibilities equal to 4^8 = 65536. However, DNA is a polymer in the form of a double helix where nucleobases only link a certain way as stated before. Replicating the double helix form would mean using a non linear representation for the AT, CG pairs. It is impossible to keep using a G.A for this kind of representation as a G.A is only usable for linear representation per the definition of its operators (selection, crossover, mutation). We could use a set equivalent to {AT, CG} in order to achieve that with a G.A but it would basically lead to the same result as using {0, 1} as our gene set (256 combinations) It is possible to completely change the DNA structure to get more combinations but that would mean changing how we calculate the A.A. In fact, it feels unnecessary to modify the DNA as {0, 1} can be considered like {AT, CG} and the mathematical model seems to hold itself vis a vis the ecological dynamic patterns for our species survival.

Possible parameters that have the potential to turn around the ecological dynamics in our simulation are temperature variations, diurnal cycle, season changes, diseases and the effect of human activities on the environment. Those should prove themselves extremely complicated to replicate in the U.E5 engine but could have a really interesting effect on our results and should be taken into consideration for the future of this research.

In the next subsection, we will discuss potential future directions for this study and outline the areas where our efforts should be focused, hoping to gain better understanding on the link between gene significance and survival as well as ecological dynamics.

What's next ?

In the previous subsection, we used our data to confirm the hypothesis that changes in the environment as well as in the agent (M.U.T threshold) influences g.c/l, gene significance and survival.

The first direction that the study should take is the correction of the agent teleporting, which would lead to more combat between species and hence more data concerning the predator/prey problem. The agent's behavior could also be improved as it is quite simplistic for now.

Another path the research could take is refining the agent attributes calculations. By using more accurate formulas and methodologies, we could make gene significance closer to reality and gain deeper knowledge on the specific genetic traits that lead each species and enable survival.

In addition to that, adding environmental factors and parameters could enhance the realism and complexity of our simulation. The parameters that were mentioned in the previous subsection were just examples. Implementing them in U.E5 might present some difficulties but it could lead to learning more about their impact on the simulated world, the species that inhabit it and the ecological systems that thrive there. It is also possible to expand the range of species in the simulation as only two species exist in the virtual world for now. We hope that bringing new species with unique genetic traits and behavior will help us observe more complex population dynamics and ecological relationships.

Another key component for this research is the investigation of the role of mutation and its effect on survival and evolutionary process. Exploring different thresholds (example : 0.79M.U.T, 0.83M.U.T) and adding other types of mutation could enable us to gain understanding on the M.U.T operator's influence on survival, persistence of species and genetic variations over time.

Finally, it would be key to collaborate with other researchers and scientists in related fields (ecology, computer science, genetics…) as they bring new information to light and enrich the study. Combining the expertise and resources of these different actors, we would be able to push the limits of this research even further. An example of such a case would be Rhiannon Follenfant and her statistical expertise that she generously lended us for this thesis.

In conclusion, the next steps for this study involve correcting features, improving attribute calculations, adding environmental parameters and new species and encouraging interdisciplinary collaborations. By pursuing these directions, we can ultimately advance our knowledge of genetic significance and ecological dynamics, contributing to a deeper understanding of our world and its wonders.

V -Conclusion

The problematic behind this research was studying the evolution of agents inside a virtual world. We sought to understand the mechanisms of DNA transfer between generations, identify the genes of utmost significance and pinpoint parameters influencing survival and gene significance. Those were some of the questions we asked ourselves while working on this project.

The methodology used to respond to these questions and the problematic were to create an artificial life simulation in Unreal Engine 5, which aims to mirror real life, reproduce animals' traits and behavior, run a genetic algorithm with binary tournament, uniform crossover and n-point crossover on each generation and analyze the data to find what genes hold the most significance per case.

After defining the methodology employed in our thesis, we turn to our different findings.

Analyzing the collected data during multiple runs enabled us to understand which parameters influence the most survival and a species' success in our virtual environment.

We noted that some species would thrive more than others in the virtual life environment, and thus due to recreating or not senses like sight and hearing. A perfect example of this is how deers always managed to keep between 60% and 40% of their initial population whereas foxes always kept between 40% and 20% of theirs. It is explained by how recreating the hearing sense in Unreal Engine 5 showed itself to be a challenging task compared to the sight sense's recreation.

In our simulation, deers were bigger than foxes and held a better sight sense than them, hence giving them an advantage when searching for resources or finding predators in case of danger. Subsection 1 of part IV denotes how important Sight genes were during all the runs. Moreover, resource availability and the absence of competition internally and externally of a species is a key factor for survival and a species' success. Furthermore, it was also noted that mutation, as a process, is important to create genetic diversity but a low threshold for mutation would impact agents' life and thus, survival in a negative way. As shown in Table 6 and 7, other genes held a lot of significance : AgentSize, Damage, Speed and Health. A reason behind such significance is because these genes are the ones defining how well an agent interacts with its environment (A.Size) and enemies (A.Speed and A.Damage) and how long it can search for food before needing to rest (A.Health).

While the simulation, collected data and their analysis may bring to light some new information about ecological dynamics and artificial life simulation, we should acknowledge the limitations of our research. Agents do not move in our simulation, they teleport, which causes a problem in terms of chasing/running away between predator and prey. Agents' behavior is quite simplistic and could be improved to create more cases to study. Furthermore, we only studied the effect of changing resource availability and mutation threshold. These parameters affect agents' behavior but in our world, there is a vast array of parameters whose effect on our world and agents remains undocumented. As our formulas for calculating agent attributes remain quite simple, refining or modifying them might be a route to pursue in this research's future. The paper Evaluating the Models and Behaviour of 3D intelligent Virtual Animals in a Predator-Prey Relationship created an artificial life simulation where certain parameters like predator/prey awareness, flocking or herding can be enabled or disabled. It focuses on the cycle of life and death and the predator/prey problem whereas ours focuses on DNA transmission and evolution in a virtual world. Implementing these parameters in the future of our research, as we implement newer species, could lead to a better and deeper understanding of ecological dynamics and predator/prey problem.

In conclusion, this research endeavored to explore the complex concepts of evolution and ecological dynamics using Unreal Engine 5 and artificial life simulation. Mimicking animals' behavior and the environment in a realistic manner proved itself a formidable challenge, but results were obtained and they were satisfactory. The thesis contributes to bringing new knowledge as it underlines the significance of resources availability, mutation threshold and sensory capabilities. Given how this research overlaps an array of fields, it has the potential to benefit our knowledge of evolution if resources and time were allocated to it.

Swap mutation -Figure 13 : Operators of the Two-Part Encoding Genetic Algorithm in Solving the Multiple Traveling Salesmen Problem -Shih-Hsin Chen and Mu-Chung Chen, 2011

Figure 1 :

 1 Figure 1 : Side-by-side screenshots of the complex collision and the simple sphere collision for the Deer model in Unreal Engine 5

Figure 2 ;

 2 Figure 2 ; Pseudo code of the Agent Attribute initialization algorithm

Figure 3 :

 3 Figure 3 : Pseudo-code of the birth process / spawning process

Figure 4 :

 4 Figure 4 : Graph of cycle of birth/death in our simulation

Figure 5 :

 5 Figure 5 : Screenshot of the initial level in the Unreal Engine 5 editor view

Figure 6 :

 6 Figure 6 : Side by side comparison of two runs of the EnvironmentalSpawners in the initial level

Figure 7 :

 7 Figure 7 : Forest picture taken in Unreal Engine 5

Figure 9 :

 9 Figure 9 : Pseudo-code of a regular genetic algorithm

1 . 1 . 1 :

 111 Add the agent to an array Step 1.1.2 : Go to the next agent Step 1.2 : Else go to the next agent Step 2 : Select two random agents from the Population //called FirstCandidate and SecondCandidate Step 3 : Compare their SurvivabilityScore Step 3.1 : If the FP score is higher, remove it from Population and added to a NewPop array Step 3.2 : Else, remove SP from Population and added to NewPop array Step 4 : Loop until Population array empty

Figure 10 :

 10 Figure 10 : Pseudo-code of the simulation's binary tournament

Figure 11 :

 11 Figure 11 : Graph of the uniform crossover with two parents and two offsprings

Figure 12 :

 12 Figure 12 : side-by-side comparison of the random mutation and the swap mutation in binary-coded chromosomes

Figure 16 :

 16 Figure 16: Fox and deer populations : Gene comparison and individuals evolution (75t -0.79M.U.T)

Figure 17 :

 17 Figure 17: Fox and deer populations : Gene comparison and individuals evolution (150t -0.79M.U.T)

Figure 18 :

 18 Figure 18: Fox and deer populations : Gene comparison and individuals evolution (75t -0.83M.U.T)

Figure 19 :

 19 Figure 19: Fox and deer populations : Gene comparison and individuals evolution (150t -0.83M.U.T)

Table 1

 1

	Animal Class	Average Length (base pairs)	Minimum Length (base pairs)	Maximum Length (base pairs)
	Mammals	3.5 billion	242 million	6.3 billion
	Avian	1.385 billion	1.15 billion	1.62 billion
	Serpentes	2.8 billion	1.3 billion	3.8 billion
	Fishes	1.4 billion	103 million	133 billion

shows the average, minimum and maximum length of the DNA structure in different classes of animals.

Table 1 : Average, minimum and maximum length of the DNA structure in the mammal, avian, serpentes and fish classes.

 1

  1 : Calculate a sense bonus // a bonus for each sense Step 2.2 : Calculate each statistic per agent specie // the formulas needed to calculate the stat is different for each specie Step 3 : Initialize float attributes // initializing AgentSize, Speed...

	Step 3.1 : Calculate Agent Size // the size of the agent
	Step 3.2 : Calculate Agent Speed // the speed of the agent
	Step 3.3 : Calculate Agent Damage // the damage of the agent to others
	Step 3.4 : Calculate Agent Health // the health points of the agent
	Step 3.5 : Calculate Survivability Score // how well the agent might survive
	in the environment
	Step 4 : Initialize Int Stats // initializing the AgentID by random

Table 6 : Fox and deer populations : Gene significance (first runs)

 6 Table6is the significance table for our first runs (with old formulas). It is right that those formulas were flawed. It still might be interesting to analyze gene significance there as we might be able to learn some things from it. Here is the legend for both of those tables :

	< 0.001 : xxx	< 0.01 : xx	< 0.05 : x	> 0.05 : X

Acknowledgements List of Abbreviations I. Introduction II.

Acknowledgements

, thank you for forcing me to learn how to debug my code and for reading my first draft and the rest of them. My Japanese supervisor, Dr. Okubo Masashi, you welcomed me to the AMILAB in Doshisha University, treated me like any other student in the university and gave me advice and your undeniable support during my research. あなたのサポート、アドバイス、そして私をアミラボに迎え入れてくれたことに、

VI -Annex

The source code of this project can be accessed via the link : https://github.com/hassenebenamar/Research_Project