VU Research Portal

(1)

VU Research Portal

Where is the robot?

da Silva Miras de Araujo, K.

2020

document version

Publisher's PDF, also known as Version of record

Link to publication in VU Research Portal

citation for published version (APA)

da Silva Miras de Araujo, K. (2020). Where is the robot? Life as it could be.

General rights

Copyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights. • Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain

• You may freely distribute the URL identifying the publication in the public portal ?

Take down policy

If you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediately and investigate your claim.

E-mail address:

(2)

(3)

Where is the robot?

Life as it could be

Karine Miras

MSc.

Department of Computer Science

(4)

SIKS Dissertation Series No. 2020-21

The research reported in this thesis has been carried out under the auspices of SIKS, the Dutch Research School for Information and Knowledge Systems.

Copyright˜ c 2020 by Karine Miras

(5)

VRIJE UNIVERSITEIT AMSTERDAM

Where is the robot?

Life as it could be

ACADEMISCH PROEFSCHRIFT

ter verkrijging van de graad Doctor of Philosophy

aan de Vrije Universiteit Amsterdam,

op gezag van de rector magnificus

prof.dr. V. Subramaniam,

in het openbaar te verdedigen

ten overstaan van de promotiecommissie

van de Faculteit der B`

etawetenschappen

op donderdag 17 september 2020 om 13.45 uur

in de aula van de universiteit,

De Boelelaan 1105

door

Karine Da Silva Miras De Araujo

(6)

promotor: prof.dr. A.E. Eiben

(7)

Members of the committee

prof.dr. J. Bongard University of Vermont

dr. G. Nitschke University of Cape Town

dr. G. de Croon TU Delft

dr. R. de Kleijn Leiden University

prof.dr. J. Ellers Vrije Universiteit Amsterdam

(8)

(9)

Acknowledgements

My thesis is dedicated to my loved and loving father Rogerio Miras, who inserted into my mind the subconscious desire of joining academia through Kraftwerk songs and other tricks.

Firstly, I would like to thank my mother, Eliana da Silva, for all the sacrifices she made, which allowed me to follow my path until here.

Secondly, I would like to thank my supervisor Prof. Dr. Guszti Eiben for all his support and faith, and my co-supervisor Dr. Evert Haasdijk for his fundamental advice.

Finally, I would like to thank friends who supported me during these challenging years: Phillip Kersten and Milan Jelisavcic.

(14)

“When we imagine a machine, the result is always something close to a mechanical system that works by itself. It does not bother us to think that it is nothing beyond that. But how do we feel when we imagine ourselves as a machine? Empty. We have the sensation that something is missing. And what is missing? What is it that a human being is full of, and that a machine lacks? Illusion. The emptiness of the machine is the consciousness that our subjective world is a fiction; the consciousness that our humanity is a delirium, and that there is nothing behind what we live. We are machines, and our consciousness is a dream of this machine. Nothing else. Absolutely nothing.”

(a translation from O vazio da m´aquina by Andr´e Cancian) “My thesis’ll smash a stereo to pieces

My acappella releases, classic masterpieces Through telekinesis, it eases you mentally Gently, sentimentally, instrumentally

With entity, dementedly meant to be Infinite” (Infinite by Eminem)

(15)

Contents

Summary

The field of Evolutionary Robotics (ER) “aims to apply evolutionary com-putation techniques to evolve the overall design or controllers, or both, for real and simulated autonomous robots”, and dates back to the beginning of the 90s. ER is inspired by nature, under the assumption that if natural evolution created natural intelligence, then artificial evolution could create artificial intelligence. While today ER systems are hardly autonomous, the long-term view of ER foresees (autonomously) self-engineering robot systems. Importantly, despite the well-known fact that the environment is extremely determinant towards natural living forms, investigations regarding the environment are meager, especially when regarding environmental regulation of genetic material.

The main goal of this research was shedding light on the influences that the environment has on robot artificial life systems. To make this research possible, we developed a framework that allowed us to evolve populations of robots and measure the effects that different factors have on these robots. This framework included the design of a) a robot encoding method for morphology and controller, b) morphological descriptors, c) controller descriptors, d) behavioral descriptors, e) a set of environments presenting distinct environmental conditions, f ) an Evolutionary Algorithm, g) a learning mechanism. Using this framework, we evolved populations of robots to perform the task of locomotion, so that their fitness was measured by their locomotion speed. Here we present results of multiple and extensive experiments with the evolution of morphologies and controllers of modular robots. In particular, we focus on the effects that different environmental conditions have on phenotypic and behavioral traits.

(16)

Contents

etc. Second, we demonstrated that in a flat floor environment there is a strong selection pressure for snake-shape robots (characterized mainly by having a single limb), whose principal lomocotion gait is rolling. Remarkably, these simple snake-shape robots were much faster than the other variety of shapes described before. Third, we demonstrated that by inclining the floor, a new selection pressure was created. In this case, the emergent robots had more limbs, were more proportional and smaller, and had their locomotion gait changed from rolling to rowing or dragging. Furthermore, in different experiments we tested changing the environmental conditions in two ways: across generations and during the robots lifetime. In the second case, naturally, because they disposed of one same morphology and controller, a trade-off happened. They managed to locomote in both environmental conditions (seasons), but their performance was worse than when they had evolved in each static environment separately. Finally, we designed a solution to mitigate this performance degradation caused by the trade-off described above. This solution is a novel robot encoding method called Plasticoding. This encoding gives the robots genotypes a capacity for phenotypic plasticity, meaning that one individual can develop different morphologies, controllers, and behavior according to environmental stimuli during its lifetime. Using Plasticoding we reduced the loss in performance for the flat environment. The results of Plasticoding increased the performance in 58% in comparison to our robot encoding when this plasticity capacity was not available (Baseline). Because the environment is determinant to natural life forms, we believe this subject has a lot of potential to help to improve the quality of ER systems. Nevertheless, this subject was very scarcely explored in the literature. Therefore, our work is a fundamental step towards a long-term vision: succeeding in creating robot artificial life with complexity and adaptability comparable to what we see in nature.

(17)

1

Introduction

Preamble. There are multiple tasks that, desirably, would be carried out by

(18)

Chapter 1. Introduction

The main terms we use to describe our research in this thesis are:

Term Meaning

morphology body

controller brain

genotype encoding method or representation

phenotype morphology and controller decoded from genotype

behavior interaction of morphology, controller, and environment

morphogenesis decoding genotype into phenotype for the first time

development decoding genotype into phenotype at any moment

1.1 Motivation and Contributions

The field of Evolutionary Robotics (ER) “aims to apply evolutionary compu-tation techniques to evolve the overall design or controllers, or both, for real and simulated autonomous robots” [83], and dates back to the beginning of the 90s [20; 10]. ER is inspired by nature, under the assumption that if natural evolution created natural intelligence, then artificial evolution could create artificial intelligence.

The Evolution of Things (EoT) [21; 23] is a long term ER research endeavor that aims at evolving both morphology and controller of robots in our real physical world. This means programming a physically embodied artificial system with evolvable capabilities similar to natural systems: robot organisms as a whole (morphology and controller) are subject to evolution (Fig. 1.1). This contrasts with cases of robot design in which humans hand-craft robot body structures or program its behavior deterministicaly. Notably, for safely evolving these robots in actual hardware, a generic architecture called Triangle of Life [22] was proposed, in which a robot life cycle happens in three stages: Birth (morphogenesis), Infancy and Mature Life. In spite of the current evolvable robot systems being hardly autonomous, the EoT foresees autonomous self-engineering robot systems. From an self-engineering perspective, this is interesting because these systems might allow producing functional robots even if humans do not know exactly how a task should be carried out.

(19)

1.1. Motivation and Contributions

Programmable

evolu&onary systems in silico (evolu&onary compu&ng)

Real-‐world evolu&onary systems in vivo (biosphere)

Evolu2on of Things

Figure 1.1: The Evolution of Things takes place in real space and real time based on (self-)reproducing physical -rather than digital- entities. Image from [21].

More than that, in the future these systems are expected to constantly adapt while requiring little or no human intervention, which is fundamental when adaptation is vital while intervention is not feasible. Moreover and beyond engineering, such systems could also be an aid for biological investigations about the origins of intelligence, which potentially might allow improving our living conditions. Importantly, because the technology needed to construct robots in hardware during the process of evolution is still not available, most of what currently can be done contribute to EoT through developing know-how and providing fundamental proofs of concept in simulation.

Although the long-term vision of EoT comprises futuristic scenarios in which robots have very complex behavior, in fact, current studies explore mostly relatively simple tasks like locomotion. Nevertheless, despite its simplicity, locomotion is fundamental, while it still presents great challenges to the field. Therefore, these challenges justify the focus of the field on locomotion as a task. Among these challenges is the conjoint evolution of morphology and controller, and also lifetime plasticity [83].

(20)

However, the evolution of morphology is seldomly addressed in the litera-ture [87; 26], despite evidence that intelligence depends also on the body [68]. In fact, it is estimated that around 95% of ER studies evolve only the controller [70]. Moreover, despite the well-known fact that the environment determines nat-ural life forms [18; 76; 78], the influence of the environment on ER systems has been even more scarcely explored, with very few studies investigating it [55; 5; 19; 48; 47]. Unfortunately, this negligence towards the influence of the environment impairs the long-term endeavor of creating autonomous self-engineering robot systems. This is true because these systems should have the capacity to adapt to any environmental changes, and the relevance of adapting to such changes is broad. As living beings in the real world, we face a plethora of environmental conditions. More than that, in nature or society, these conditions constantly change, and often to states that were unknown before. First, from a purely technical perspective, if we wish to succeed in the construction of highly complex ER systems, it is imperative that they become able to tackle the challenges of living in diverse and changing environments. Second, from an ethical awareness perspective, we should understand in which environmental conditions our ER systems work (and how) and prepare for the implications of having these conditions changed. For instance, if we discovered an environmen-tal condition that could change the structure and behavior of our robots in an endangering way, then this certainly should be taken into consideration in our design choices.

The bias of ER to study only the controllers indicates that the science and technology of evolving robots in the real world is in a very early stage. There is much groundwork to be done and this thesis is meant to deliver a stepping stone on the long road ahead.

When studying the behavior of natural living forms, both direct and indirect environmental influences are taken into consideration [76]. This means to account for where these living beings evolved and developed. Note that where refers to environmental conditions, which can be characterized in many ways as intra- or inter- cellular environment, the internal state of an individual,

(21)

1.1. Motivation and Contributions

or the external context where the individual is [76]. Similarly, if we wish to understand the structure and behavior of an artificial life form such as an evolved robot, one fundamental question is: “Where is the robot?”. Fundamentally, the phenotype and behavior of this robot depends on where this robot was during its development, and where its ancestors were during evolution.

This thesis presents the results of multiple and extensive experiments with the evolution of morphologies and controllers of modular robots. In particular, we focus on the effects of the environment on robot traits. Our main message is that in ER systems, robots should have both morphology and controller evolved while taking the influence of the environment into consideration from two different perspectives:

1. Indirect influence: The environment drives the course of the evolu-tionary search for fit robots, determining the genetic material needed to produce adequate traits to this environment. This means that distinct environmental conditions might create distinct selection pressures. There-fore, the robots emerging throughout evolution in each one of these distinct environments shall differentiate phenotypically and/or behaviorally. 2. Direct influence: The environment acts upon the development of evolved

genotypes regulating1 the expression of their genes, and through this giv-ing them the capacity to phenotypic plasticity. This means that accordgiv-ing to environmental conditions taking place during a robot’s life, its pheno-type (morphology and/or controller) might differentiate. Note that this phenotypic differentiation could or not result in behavioral differentiation. As an analogy, we could say that while the indirect influence produces a set of ingredients and recipes, the direct influence is a cook following a recipe of which ingredients (and how much) to use. Note that the conditions in which this recipe should be used are part of the recipe itself.

1_{This regulation happens through a process once called epigenetics [11], a term that}

(22)

Taking these perspectives into account, the main contributions to ER pre-sented in this thesis are:

1. Design of a framework and methodological approach for search space analysis of modular robots, including quantitative descriptors of their morphology, controller, and behavior.

2. Delivery of test environments for empirical studies to illuminate that phe-notypic and behavioral differentiation is possible in artificial life systems. 3. Demonstrating the existence of an ‘evolutionary memory’, i.e., the phe-nomenon that when changing environmental conditions gradually through-out generations, traits emergent in the early stages of the evolutionary period become predominant even in later stages.

4. Demonstrating that when robots have to cope with seasonal environmental changes during their life, the selection pressure for the most challenging environment is stronger.

5. Development of a novel robot encoding that gives robots a capacity for lifetime phenotypic plasticity, and demonstration of its benefits to adaptation.

Further to these contributions which have been documented through publica-tions, we present one extra contribution that was hardly documented. In one of our papers, we demonstrated an example of a pair of environments that despite having very different characteristics, did not lead to phenotypic differentiation between the emergent populations of robots. Additionally, we experimented with many other environments that resulted in this same lack of differentiation, introducing an intriguing paradox: while in nature it is well established the notion that the environment determines traits of living forms, in artificial life systems this phenomenon can be very difficult to reproduce.

(23)

1.2. Scope

This difficulty can be related to the simplicity of the current environments and tasks. While in nature creatures have to simultaneously acquire food, process food, avoid predators and dangers, support their offspring, maintain physiological aspects of their phenotype, etc, often in ER systems robots only have to locomote. Another explanation for this difficulty could be that the tasks in the tested environments were too hard, leading the search to a local optimum that did not induce the expected phenotypic differences, while robots performed poorly. This could indicate limitations in the encoding method, evolutionary algorithm or fitness function.

Because of all of the here presented reasons, we believe this subject has a lot of potentials to help to improve the quality of ER systems. Therefore, our work is a fundamental step towards a long-term vision: succeeding in creating robotic artificial life with complexity and adaptability comparable to what we see in nature.

1.2 Scope

This thesis is composed based on a collection of papers, and each of these papers is presented in a different chapter. Here, the content of the papers went through minor modifications, as for instance, rephrasing some sentences, and resizing images. From all topics approached in this thesis the foci are Evolution of Morphologies & Controllers and Environmental Effects. Through these topics we cover the three pillars that interact for the emergence of behavior: morphology, controller, and environment.

Evolution of Morphologies & Controllers All papers in this thesis, except

(24)

Environmental Effects We studied the influence of the environment on

phenotypic and behavioral traits from multiple perspectives in papers [IV, V, VII, VIII]. Firstly, we studied the effects of different types of environment on the population, showing both cases when differences were induced and cases when differences were not induced. Secondly, the effects of evolving a population in a phylogenetically changing environment. Thirdly, the effects of evolving populations of robots that spend each period of their life-time in a different environmental condition, i.e., seasonal environmental condition.

To strengthen and deepen the investigation of these two main topics, we also addressed the topics System Behavior Analysis, Learning and Evolving Controllers, and Environmental Regulation.

System Behavior Analysis While often studies analyse only robot fitness,

we dedicated a fair amount of attention to designing measurable properties for our search space, and exploring and understanding this space. This was approached in papers [I, II, III]. Because the focus of this work concerns studying the effects that the environment has on phenotypic and behavioral traits of evolvable robot, we had to design phenotypic and behavioral descriptors that could allow us to observe these possible effects. Nevertheless, before carrying out experiments aiming to observe changes in the robots induced by environmental changes, it was important to verify if the search space (a) had limitations, (b) had potential to be diverse, (c) had any tendencies/attractors, (d) could be easily explored by predefined preferences. As we are going to discuss in forthcoming sections, inducing changes on the populations of robots through environmental changes was very challenging. Therefore, without developing this machinery first, it would not have been possible to understand if differences were not being induced because the environments and tasks were not complex enough, or if it was due to restrictions of the search space.

(25)

1.2. Scope

Learning and Evolving Controllers The novelty of the robot encoding that

we developed concerns behavioral changes caused by environmental regulation of genetic material. For this reason, we decided not to incorporate lifetime learning into our system, aiming at more clearly isolating effects on behavior. Nevertheless, because we observed a strong selection pressure for snake-like rolling robots (which we will discuss later on), in one of our papers [VI] we incorporated learning into the system, targeting to investigate if this pressure was result of the inability to learn.

Environmental Regulation Finally, we proposed and designed a novel robot

(26)

List of Papers

This thesis is the result of three years of research, and is constructed using the content of six conference papers and two journal papers. These papers are listed below, along with details of my contribution to each one.

Topic 2018 2019 2020

Evolution of Morphologies & Controllers

[II, III] [IV, V] [VI, VII, VIII]

System Behavior Analysis [I, II, III]

Learning and Evolving Controllers

[VI]

Environmental Effects [IV, V] [VII, VIII]

Environmental Regulation [VIII]

[I] Miras K, Haasdijk E, Glette K, A.E. Eiben. Search Space Analysis of Evolvable Robot Morphologies. In: Applications of Evolutionary Computation - 21st International Conference, EvoApplications 2018. vol. 10784 of Lecture

Notes in Computer Science. Springer; 2018. p. 703–718.

I designed and implemented the morphological descriptors, the encoding method and the EA. Moreover, I conducted the analysis and wrote most of the text.

[II] Miras K, Haasdijk E, Glette K, A.E. Eiben. Effects of Selection Preferences on Evolved Robot Morphologies and Behaviors. In: Ikegami T, Virgo N, Witkowski O, Suzuki R, Oka M, Iizuka H, editors. Proceedings of the Artificial Life Conference 2018 (ALIFE 2018). Tokyo: MIT Press; 2018. p. 224–231.

I designed and implemented enhancements for the encoding method and the EA, and designed and carried out the experiments. Moreover, I conducted the analysis and wrote most of the text.

[III] Miras K, Gansekoele A, Glette K, A.E. Eiben. Insights in evolutionary exploration of robot morphology spaces. In: Proceedings of the 2018 IEEE

(27)

1.2. Scope

Symposium Series on Computational Intelligence. IEEE Xplore; 2018. p. 867–874.

I designed and carried out the experiments, and supervised a student who helped me to conduct the analysis. Moreover, I wrote most of the text. [IV] Miras K, A.E. Eiben. Effects of environmental conditions on evolved robot morphologies and behavior. In: Proceedings of the Genetic and Evolutionary Computation Conference. ACM; 2019. p. 125–132.

I designed the behavioral descriptors, and implemented adaptations to my system that allowed me to carry out the experiments. Moreover, I conducted the analysis and wrote most of the text.

[V] Miras K, A.E. Eiben. The impact of environmental history on evolved robot properties. In: The 2018 Conference on Artificial Life: A Hybrid of the European Conference on Artificial Life (ECAL) and the International

Conference on the Synthesis and Simulation of Living Systems (ALIFE). MIT Press; 2019. p. 396–403.

I designed and carried out the experiments. Moreover, I conducted the analysis and wrote most of the text.

[VI] Miras K, Carlo M, Akhatou S, and A.E. Eiben. Evolving-controllers versus learning-controllers for morphologically evolvable robots. Evostar 2020

I supervised a student who enhanced my system with the capabilities of learning. Moreover, I conducted most of the analysis, and wrote most of the text.

[VII] Miras K, Ferrante E, A.E. Eiben. Environmental influences on evolvable robots. PloS one 15.5 (2020): e0233848.

(28)

[VIII] Miras K, Ferrante E, A.E. Eiben. Environmental regulation for the evolution of robots using Plasticoding. [it was accepted at the Frontiers in Robotics and AI journal, and will be published soon]

I designed and implemented the phenotypic plasticity capacities of the encoding method, designed the experiments, conducted the analysis, and wrote most of the text.

1.3 Extra Publications

For this paper I have a co-authorship, and I decided not to add it to the main content of my thesis.

[9] Jelisavcic M, Miras K, and A.E. Eiben (2018). Morphological Attractors in Darwinian and Lamarckian Evolutionary Robot Systems. In 2018 IEEE Symposium Series on Computational Intelligence, SSCI 2018 (pp. 1-8). [7850166] Institute of Electrical and Electronics Engineers, Inc..

I designed and implemented the morphological descriptors analyzed. Addi-tionally, I helped to formulate the hypothesis, and interpret the results.

(29)

1.4. Methodology

Figure 1.2: At the left, the types of modules of our robot system. In the middle and at the right, example robots evolved using these modules as building blocks.

1.4 Methodology

This work originates from the field of ER, which is inspired by evolutionary Darwinian notions and envisions the automatic creation of robots [29; 83; 20; 10]. This means these systems could auto-design their structure and behavior while requiring minimal human intervention. Crucially, we investigate the evolution of both morphology and controller of populations of robots, as in the seminal work of Sims [77]. The tasks we experimented with are directed and non-directed locomotion, the former meaning locomotion in an specific direction, and the later locomotion in any direction. This way, our concept of fitness concerns the speed of a robot in the due direction, so that the faster a robot, the more chances it has to survive and reproduce.

The type of robots we work with are modular, i.e., a robot morphology (body) is built using a set of known building blocks. As an analogy, we could say that the morphologies of these robots are built as Lego toys. Figure 1.2 shows the modules and examples of evolved robots. The design of these building blocks (modules) is derived from a system called RoboGen [4]. As for their controllers (brains), we use Artificial Neural Networks. The DNA structure of the robots, i.e., encoding method or representation, is constructed using an L-System inspired by [34]. Finally, the populations of robots were evolved using a combination of classic mechanisms from Evolutionary Computing [25].

(30)

Figure 1.3: Environments where populations evolved

utilized a simulator called Gazebo, interfaced by a framework called Revolve [37]. The environments (Fig. 1.3) utilized in the simulations of evolution were a) a plane flat floor (Flat), b) a plane floor tilted in x degrees (Tilted), and c) a plane flat floor with small obstacles on its surface (Obstacles). Additionally, we experimented with a flat floor where there was a cost for robots touching the floor (Lava environment), but it did not lead to publishable results.

While in some experiments robots evolved in static environments, in other experiments these environments were combined in different ways to test diverse environmental conditions. To measure the effects that different factors had on the evolved robots, we designed a set of robot descriptors concerning morpho-logical, controller and behavioral properties. These descriptors measure diverse properties as, for instance, how many limbs a morphology has, how much the neurons of a robot are able keep memory, or how balanced is a locomotion gait.

All papers presented in this thesis are directly interconnected building upon each other to support the investigation of the influence of the environment on robotic artificial life systems. The list of papers comprised in this thesis is detailed in Section 1.2, where they are referred to using Roman numerals. The flowchart in Figure 1.4 depicts our workflow and how each paper contributed to this investigation, and is described hereafter, where we reference parts of the flowchart through letters. Despite ultimately having been able to demonstrate the benefits of environmental regulation (green box in the flowchart), many times we bumped into dead-ends (red boxes).

(31)

1.4. Methodology

Figure 1.4: Flowchart representing the main steps of our workflow.

− (a): Papers [IV, V] investigate multiple questions about the effects of the environment on robots. This includes demonstrating that different envi-ronmental conditions create different selection pressures, leading robots to phenotypic and behavioral differentiation. Trivial as is may seen, it was very difficult to find a pair of environments that create different selection pressures. Therefore, it is clear that notions observed in nature can not be taken from granted in artificial life systems, and reproducing the natural conditions that allowed complex forms of life to emerge remains a great challenge. Before this demonstration seasonal environmental changes would make no difference, and therefore developing an environmentally regulated robot encoding would have been in vain.

(32)

limbs, symmetry, walking behavior, etc. Before making sure this was true, any efforts in changing the environments with the purpose of inducing phenotypic or behavioral differentiation would also have been in vain. − (c): Paper [VII] investigates the effects of evolving robots that need to go

through seasonal environmental changes. This means robots spend each part of their life under different environmental condition and must be fit for each of these conditions. Here, because robots have to cope with multiple environmental conditions while disposing of one same morphology and controller, and thus behavior, naturally a trade-off happens. This trade-off was characterized by robots managing to locomote in both environmental conditions, but with severe performance degradation in both conditions. − (d): Finally, paper [VIII] adds a capacity of environmental regulation to the robot encoding. Most importantly, it demonstrates how it helps to mitigate the degradation mentioned above.

(33)

2

Morphological Descriptors

Chapter 2 was published as:

(34)

Chapter 2. Morphological Descriptors

We present a study on morphological traits of evolved modular robots. We note that the evolutionary search space –the set of obtainable morphologies– depends on the given representation and reproduction operators and we propose a framework to assess morphological traits in this search space regardless of a specific environment and/or task. To this end, we present eight quantifiable morphological descriptors and a generic novelty search algorithm to produce a diverse set of morphologies for any given representation. With this machinery, we perform a comparison between a direct encoding and a generative encoding. The results demonstrate that our framework permits to find a very diverse set of bodies, allowing a morphological diversity investigation. Furthermore, the analysis showed that despite the high levels of diversity, a bias to certain traits in the population was detected. Surprisingly, the two encoding methods showed no significant difference in the diversity levels of the evolved morphologies or their morphological traits.

2.1 Introduction

Evolutionary Robotics (ER) [66; 10; 83; 20] is a field that “aims to apply evolutionary computation techniques to evolve the overall design or controllers, or both, for real and simulated autonomous robots” [83]. Traditionally, the emphasis lies on evolving controllers for fixed robot bodies, but there is a growing interest in evolving the morphologies as well [77; 35; 75; 15; 84]. For instance, a generic architecture for a system of embodied on-line evolution of robots in real time and real space was proposed in [22]. However, the current technology of rapid prototyping (3D-printing) and automated assembly is a limiting factor, and studies in simulations remain important.

In this paper we address the issue of morphological diversity in an evo-lutionary robotic system. In general, there are three essential factors that determine the course of evolution in such a system, 1) the encoding, including the phenotypes (the set of possible morphologies), the genotypes (the syntacti-cal representation of these phenotypes), and the mapping from genotypes to phenotypes, 2) the reproduction operators that generate new genotypes from

(35)

2.2. Morphology Space and Morphological Descriptors

existing ones, and 3) the selection operators that depend on the environment and the task at hand. For the sake of this study we distinguish the search space and the application space of an evolutionary robotic system. The search space consists of the encoding and the reproduction operators, while the application space is formed by the environment and the given task. Besides the impact of the environment [6], clearly, the properties of the search space also have a paramount impact of what evolution can achieve. The main research question we address here is: How to investigate the effect of the search space on the set of evolvable morphologies? This question will be broken down into two subquestions:

- How to quantify and measure morphological properties?

- How to isolate the effects of the encoding and the reproduction from the effects of selection?

The measures and the methodology we propose to answer these questions will be applied to compare a direct encoding and an indirect encoding scheme for the morphological space we work with in our research programme towards physically evolvable modular robots.

2.2 Morphology Space and Morphological

De-scriptors

Our robot bodies are composed of modules1 _{as shown in Fig. 2.1, based on}

RoboGen [4]. For this study, the bodies are flat, constructed in 2D, i.e., the modules do not permit attachment on the top or bottom slots, only the lateral ones. Each module type is represented by a letter in the genotype and by a colored block in the visualized phenotype (color indicating the type of block), and any module can be attached to any other through its attachment slots. An arrow inside the block points to the parent module to which the module is attached.

(36)

(a) (b)

Figure 2.1: (a) Modules of robots: core-component C holds a controller board; brick B is a cubic module; active hinge A is a joint moved by a servo motor. C and B have attachment slots on its four lateral faces, and A has attachment slots on its two opposite lateral faces; (b) example of a simulated robot.

For quantitatively assessing a given modular body we designed and utilized eight morphological descriptors.

The maximum number of modules in a robot is limited to mmax. Given

an mmax, each descriptor can assume a discrete number of values, and the

calculation for these numbers can be found in the accompanying documentation2. Each morphological descriptor was normalized to a range between 0 and 1, as explained below.

Branching.

Captures how the attachments of the modules are grouped together in the body, and envisions to measure whether the components of the body are more spread or agglomerated. It is defined with Eq. (2.1):

B =    b bmax, if m >= 5 0 otherwise (2.1)

where m is the total number of modules in the body, b the number of modules

that are attached on all four faces, and bmax= b(m − 2)/3c – the maximum

possible number of modules that can be attached on four faces in a body of m modules. See Fig. 2.2 for a few illustrative examples.

2_{https://tinyurl.com/y9s8ssuc}

(37)

Figure 2.2: Morphology (a) has no module with its four faces attached, (b) has one module with its four faces attached, which is the maximum possible given the size of the body, and (c) has one module with its four faces attached, but could have two, if using the modules indicated by pink arrows to be attached to the one indicated by the orange arrow.

Limbs.

Describes the number of extremities of a body. It is defined with Eq. (2.2):

L =    l lmax, if lmax> 0 0 otherwise lmax=    2 ∗ b(m−6)₃ c + (m − 6) (mod 3) + 4, if m >= 6 m − 1 otherwise (2.2)

where m is the total number of modules in the body, l the number of modules which have only one face attached to another module (except for the

core-component) and lmaxis the maximum amount of modules with one face attached

that a body with m modules could have, if containing the same amount of modules arranged in a different way (Fig.2.3).

(38)

Length of Limbs.

Describes how extensive the limbs of the body are and is defined with Eq. (2.3):

E =    e emax, if m >= 3 0 otherwise (2.3)

where m is the total number of modules of the body, e is the number of modules which have two of its faces attached to other modules (except for the

core-component), and emax= m − 2 – the maximum amount of modules that a body

with m modules could have with two of its faces attached to other modules, if containing the same amount of modules arranged in a different way3 _(Fig.2.4).

Figure 2.4: While in morphology (b) the maximum possible quantity of modules was used as the extension of a limb, in (a), the module indicated by an orange arrow was used as an extra limb.

Coverage.

Describes how full is the rectangular envelope around the body. The greater this number, the less empty space there is between neighbor modules. It is defined as Eq. (2.4):

C = m

marea

(2.4)

where m is the total number of modules of the body, and marea= ml∗ mw –

the supported number of modules in the area of the body, with mlbeing the

number of modules that would fit in a column as long as the length of the body,

and mw the number of modules that would fit in a row as long as the width of

the body (Fig.2.5).

3_{The types of modules would not have to be necessarily the same, as long as the body had}

the same amount of modules.

(39)

Figure 2.5: While in morphology (a) all the area created by the body contains modules, in (b), there is space for two more modules.

Joints.

This describes how movable the body is and is defined with Eq. (2.5):

J =    j jmax, if m >= 3 0 otherwise (2.5)

where m is the total number of modules of the body, j is the number of effective joints, i.e., joints which have both of its opposite faces attached to the

core-component or a brick, and jmax = b(m − 1)/2c – the maximum amount of

modules with two opposite faces attached that a body with m modules could have, in an optimal arrangement (Fig.2.6).

Figure 2.6: Although both morphologies have two joints, in (b) the second joint is not effective, and would be only if the module indicated by the green arrow was switched with the one indicated by the orange arrow.

Proportion.

This describes the 2D ratio of the body and is defined with Eq.(2.6):

P = ps

pl

(2.6)

(40)

Figure 2.7: Morphology (a) is disproportional and (b) is proportional.

Symmetry.

Describes the reflexive symmetry of the body , and is defined with Eq.(8.9):

Figure 2.8: Morphology (a) has the modules indicated by green arrows horizontally reflected by the modules indicated by orange arrows; (b) has no modules reflected; (c) has the module indicated by the orange arrow vertically reflected by the modules indicated by the green arrow, but no reflection for the module indicated by the pink arrow.

Z = max zvzh

(2.7) where zh= oh/qh – is the horizontal symmetry, and zv= ov/qv – the vertical symmetry. For calculating each of these symmetry values, a referential center for the body is defined as the core-component. For both horizontal h and vertical v axes, a spine is determined as a line dividing the body into two parts according to the center and this axis. Each value is the number o of modules that have a mirrored module on the other side of the spine (each match of modules accounts for two), divided by the total number q of compared modules. The spine is not accounted in the comparison (Fig.2.8).

(41)

2.3. Exploring the Space of Morphologies

Size.

Describes the extent of the body in terms of number of modules and is defined with Eq.(2.8):

S = m

mmax

(2.8)

where m is the total number of modules in the body and mmaxthe maximum

number of modules permitted in any body (Fig.2.9).

Figure 2.9: Morphology (a) is bigger than (b). Example for

mmax= 20.

2.3 Exploring the Space of Morphologies

In the foregoing we have introduced eight morphological descriptors that can be used to analyze any given set of robotic morphologies. For instance, they can be measured and plotted during the evolutionary search process and/or applied to assess the final population from a morphological perspective. In this section we demonstrate how they can be used to compare two different representations. To this end, we present a generic methodology for sampling the search space (specified by the encoding and the reproduction operators) independently from

the application space (defined by the environment and the task).

The main idea is to create a set of sample morphologies through a generate-and-test search process where the generate step uses the actual reproduction operators, but the test step is based on morphological properties, not influenced by its behavior. The code of our method and the experiments can be found on GitHub4_.

For these experiments the size of the morphologies, mmax, was limited to

100 modules regardless of the genotype size. Thus, in the body construction

(42)

phase, after reaching the limit size, extra modules in the genotype were ignored and not included in the phenotype. Additionally, modules which would overlap with other modules were not included in the body. Any morphology generated via crossover or mutation was allowed to lose any part of its genome, except for the mandatory and unique core-component.

2.3.1 Encodings

2.3.1.1 Generative Encoding:

Our generative encoding represents the genotype of a robot with a Lindenmayer-System (L-Lindenmayer-System) [39; 84], which is a grammatical parallel rewriting system. The grammar of an L-System is defined as a tuple G = (V, w, P ), where

− V , the alphabet, is a set of symbols containing replaceable and non-replaceable elements

− w, the axiom, is a symbol from which the system starts − P is a set of production rules for the replaceable symbols

In our design, the symbols of the grammar represent the modules of a robotic body and the commands to assemble them together. The system starts as a simple string of elements and grows to a more complex string iteratively during the rewriting, which performs substitutions of elements through production rules according to a grammar. The alphabet is formed by three letters and two groups of commands as shown in Tab. 2.1. For every letter, there is a production rule that might contain any letter or command, and this rule takes place in the rewriting phase to replace its correspondent letter by all of its elements. This representation functions as a developmental process for the genome. Initially, the genome is turned into a single-component structure, the axiom, as the first stage of the L-System. The axiom in this L-System is C (the core-component), and the rewriting process, i.e, development of the genome, iteratively goes on substituting each letter for the items of its production rule. The rewriting results in a string of symbols that straightforwardly maps onto a morphology.

(43)

Table 2.1: Alphabet

Symbol Type Function

C module core-component

B module brick

A module joint

addr command adds the next module to the right of the current one

addl command adds the next module to the left of the current one

addf command adds the next module to the front of the current one

mover command moves the reference to the module to the right of the current

movel command moves the reference to the module to the left of the current

movef command moves the reference to the module in front of the current

moveb command moves the reference to the module behind the current

(44)

The decoding of a simple genome is illustrated in Fig. 2.10.a. The genome starts with the axiom C, and for 2 iterations the rewriting rules are performed using the production rules for the replacements. During this construction, a turtle reference is kept for the parser to be localized in the phenotype, which starts at the bottom of the core-component. The turtle reference is updated according to the direction of the new addition movement made. If the current module is a joint, any addition command attaches the new module to the front of it. If all left, front, and right faces of the core-component were occupied, any command of attachment would place a new module to its back. After the replacements, it is possible that some commands end up without a letter in front of it, and in this case the command is a violation and is ignored. Additionally, it is possible that a new module might be supposed to be added in a position where there is a module already. This also generates a violation, and the module is ignored. These violations, which result in ignoring elements of the genotype, can be thought of as non-expressed genes.

2.3.1.2 Direct Encoding:

The direct encoding (Fig.2.10.b) uses a tree-based structure as proposed in [4], and it uses the same modules as the generative encoding. The genotype is composed with one symbol in the tree directly representing each part of the phenotype, and thus, there is a direct genotype-phenotype mapping.

2.3.2 Sampling Algorithm

Our algorithm to generate the set of morphology samples is, in fact, evolutionary. However, selection is based on robot structure, not on robot behavior. We use Novelty Search [51] to maximize the morphological diversity of the sample and to cover a large part of the search space, i.e., find as many different types of morphologies as possible. The corresponding fitness measure is based on the distance of an individual from the others in a multidimensional space defined by the eight morphological descriptors proposed above. The novelty of an individual x is calculated as the average distance to its k-nearest neighbors, where

(45)

Figure 2.11: GE operators.

k = 15 and the distance is the Euclidean distance [49] using the morphological descriptors. The set of neighbors for the comparison is formed by the current population, plus an archive, to which every new individual has a 5% probability of being added. The individuals added to the archive remain in it until the end. Using this novelty objective our evolutionary sampling algorithm was run with a population size of µ = 500 for 100 generations. In each generation pairs of parents were selected by binary tournament selection, λ = 250 offspring were created, and survivors to remain in the population were selected from the set of parents and offspring by 2-tournament selection again. The experiments with each of the two encoding methods were repeated 10 times.

2.3.2.1 Reproduction Operators for the Generative Encoding

(46)

2.3.2.2 Reproduction Operators for the Direct Encoding

The population was initialized randomly, by adding between 2 and 10 modules to each genome. Crossover was implemented by swapping random subtrees between parents as is standard practice for tree-based genomes (for instance, in Genetic Programming, [45]). Mutations were performed having 10% of chance of applying one of the following the operators: removing a subtree, duplicating a subtree, swapping subtrees, inserting a node or removing a node.

2.4 Results and Discussion

To analyze the morphologies obtained by our evolutionary sampling we use the eight morphological descriptors. The full results are available on Drive5_.

2.4.1 Individual Morphological Descriptors

In Fig. 2.12 we see that the search keeps finding new values for all morphological descriptors along the generations. Regarding the distributions of the descriptors, as depicted by Figure 2.13, for all of them, the distribution of the values is not uniform, there being a concentration of phenotypes in some values, happening consistently for all runs with both encoding methods. To compare the encoding methods, the descriptors were divided into bins and the frequencies were calcu-lated for the results with both encodings. Table 2.2 shows correlations for the descriptors (p < 0.001), indicating that the concentrations (high frequencies of phenotypes) occur in the same values for both encoding methods. This seems to indicate that there are common regions of attraction, i.e., morphological traits that are more likely to occur, independent of the encoding. Nevertheless, there are other encoding methods in the literature [75; 81], for which we do not know if this result would persist.

Some of the concentrations can be explained taking the nature of our system into consideration, such as Branching, Joints, and Symmetry with concentrations in the value 0. This outcome makes sense, because they measure constrained

5_{https://tinyurl.com/ybpcvdqp}

(47)

2.4. Results and Discussion 0.00 0.25 0.50 0.75 1.00 0 25 50 75 100 Generation Measures Niche Branching Coverage Joints Length limbs Limbs Proportion Size Symmetry Generative Encoding 0.00 0.25 0.50 0.75 1.00 0 25 50 75 100 Generation Measures Niche Branching Coverage Joints Length limbs Limbs Proportion Size Symmetry Direct Encoding

Figure 2.12: The proportion (average of the runs) of the val-ues discovered for the descriptors, considering the number of all the possible values that the descriptors can assume.

Table 2.2: Pearson correlations between the

distri-butions of the descriptors using the two encoding

methods. M1=Branching, M2=Limbs, M3=Length

of Limbs, M4=Coverage, M5=Joints, M6=Proportion, M7=Symmetry, and M8=Size.