A Distributed Ad Hoc Network for Drone Swarms

(1)

Bachelor Informatica

A Distributed Ad Hoc Network

for Drone Swarms

Lars van Hijfte

June 17, 2019

Supervisor(s): A. (Toto) van Inge

Inf

orma

tica

—

Universiteit

v

an

Amsterd

am

(2)

(3)

Abstract

This thesis answers two main research questions. The first one is: ”How can a communi-cation network for a drone swarm be implemented?” To answer this question, it takes a look at the difficulties when it comes to building a communication network for a drone swarm. Then, with those difficulties in mind, an implementation of such a system is given. This implementation allows for both single-hop and multi-hop connection, and furthermore can efficiently broadcast messages over the network. After the first question is answered, the thesis answers the second research question: ”Could the implemented network be used as a distributed network to decrease computation time?”. To answer this, the thesis tests the transfer rate in this network on different network configurations like the different number of single-hop connections, or different amount of hops needed to reach the destination.

(4)

(5)

Introduction

Swarming behaviour, or swarming, is created when a behaviour emerges from a group of entities that on their own would not be able to express that behaviour.

In nature, swarming is used by animals as a vital part of accomplishing certain tasks; bees use swarming to encase larger predacious insects with their vibrating bodies, heating up and killing the hive intruders [1], ants use swarming to locate food and bring it home to the colony [2], fish use swarming to defend against predators [3], and many more examples are found in nature.

The idea of swarming is not only useful in the animal kingdom, but it can also be used with robots. A recent popular example is the so-called Kilobots [4]. This project uses hundreds of bots to complete tasks like reconstructing images using swarming behaviour. The Kilobot project is mostly useful for research on swarming, but due to their popularity, it is also used as art[5]. The problem with the bots is that they are difficult to use for practical activities, as they lack the means of communication.

Drones, on the other hand, can carry extra hardware. This ability makes processing on the drones and communicating between the drones available. Earlier research has already shown po-tential for drones to operate as a swarm [6]. However, the network used in this drones swarm has limited capabilities. An example is that the drones cannot communicate to nodes outside their range. For that reason, this paper focusses on an improved implementing of a communication network.

There are many applications for drone swarms. Three such use-cases for a drone swarm are outlined below. This paper uses these for design choices of the network. The abbreviation ”SA” is used, meaning Swarm Activity.

(SA1) Locating people or objects.

A swarm of drones can be used to search for someone or something, after which they communicate their location back to the user who requested this activity. The search is done by spreading the drones out over an area, and processing what each drone records. This activity is interesting because of the extra processing power that is needed to process the recordings. Furthermore, if more processing power is needed, and the network allows for it, some parts of the calculation could be offloaded to other boards.

(SA2) Following a people or objects.

This activity expands on SA1 where, after finding the location, the drones continue to follow the person or object. This requires the drones to be able to follow a path while avoiding obstacles in the surrounding area. It also means that all drones fly in proximity to each other, which may impact the capabilities of the network.

(SA3) Creating a deployable temporary network.

A deployable temporary network can be used to create a connection between endpoints where normally no connection would exist. For example, in disaster areas, communication could have been shut down. This drone swarm could then be used to set up a temporary network to enable everyone to communicate with each other. In this situation, the goal

(8)

would be to optimize the speed of the network while also maintaining a stable connection between all the endpoints.

Before any SA can be implemented, drones first need to be able to communicate in a network with other drones. After that, each drone needs to have some sort of location determination of the other drones, which can then be used to avoid collisions between the drones, for example.

This paper focusses on the first step, the implementation of the network. First, it states the difficulties that appear when creating a network for a drone swarm 1. Then, it outlines the current state of distributed networks 2. After that, it explores the possibilities on how to implement the network 3. In the implementation, the paper states the requirements, gives the different options to implement, and explains the design choices made for this implementation. Once the implementation is explained, the transfer rate of the implementation is tested on different network configurations 4. Lastly, the results of these experiments are discussed 5.

Problem statement

For a drone swarm to function correctly, it must not be hindered in their mobility. This creates some limitations for the communication network. First of all, this means that the drones cannot rely on a central stationary access point (AP) as that would limit them to the range of that AP. In other words, the only nodes in the communication network are the drones themselves. This again creates another limitation, the only processing power that is available to the drones, is the total processing power of the drones together. Furthermore, this processing power is limited. The most limiting factor of a drone is its battery. Every bit of weight that is added to the drone reduces the battery. This is because weight puts more strain on the rotors, which in turn require more energy.

Another characteristic of a mobile drone swarm is that they fly around. This leads to drones continuously connect to and disconnect from others as they get in or out of range. The problem, however, is that not every network protocol is designed with this in mind. In some protocols, creating a connection takes up a considerable amount of time.

With SA3, the drones are only able to connect to all other drones via indirect connections between them. This requires the network to support multi-hopping. Multi-hopping occurs when two devices send messages to each other by using other devices (nodes) as relays.

Once these difficulties in the implementation are overcome, the advantages of such a mobile network can be used. One advantage being the fact that drones can fly towards each other to improve the network speed. Also, when drones are close together, these nearby drones can be used to offload heavy computations. This creates a parallelised system where each drone can compute a part of the whole. This is also called a distributed network.

The problems and advantages above create the following research questions that are discussed in the paper:

(RQ1) How can a communication network for a drone swarm be implemented?

(RQ2) Could the implemented network be used as a distributed network to decrease computation time?

(9)

CHAPTER 2

Distributed networks with mobile drones

A solution for the limited processing power of drones is the use of a distributed network. A distributed network uses the computational power of multiple devices in a network to accomplish a single goal [7, p. 1]. This type of network can be used in a drone swarm to use all the drones together to compute heavy computations like image processing, which is needed for SA1. When a drone needs to process multiple images, it can offload parts of the image to nearby drones, whereafter the processing power of these nearby drones can then also be used. Other advantages of such a distributed network are: [7, p. 1-2]

• Resouce sharing. Using the resources that are only available on some drones. Some drones could be equipped with more processing power, while others include cameras. With distributed systems, the processing power of some drones could be used to process the detailed image of others.

• Robustness. A distributed network does not rely on any central component. Because of this, when one node from the network fails, other nodes can be used as a fallback. • Scalability. If the distributed system is correctly implemented on a small sample of drones,

it could theoretically be easily scaled up to larger swarms. This would then improve the overall processing power and may improve processing time as well. There is a limit, however. Drones that are multiple hops away also take longer to receive data.

2.1 Ad hoc network

A distributed network can be used in two prominent ways. Client/server based, where clients can send heavy loads to the server, and the server will compute and send the results back [8]. And decentralized, this does not have a central server, and thus the computation is spread over multiple devices [8]. As stated in the problem statement, drone swarms need to work without any infrastructure. Therefore the focus is on the decentralized type.

A decentralized network that can be used as a distributed network is an ad hoc network [9]. The most basic version of an ad hoc network is a network where each node is in the transmission range of every other node. This way, the network only needs to be able to support single-hops. Each node in the network can transmit messages to other nodes directly [10]. The nodes that are connected by a single-hop are also called neighbouring nodes, or simply neighbours. Such a network can be seen in Figure 2.1a. A more complex network emerges when not every node can reach all other nodes. This is the case in most drone swarm activities like SA3. In SA3, the drones cannot directly bridge the gap between the two endpoints. Therefore, multiple drones (or hops) are needed to bridge that gap. This is also called a multi-hop ad hoc network [10] and can be seen in Figure 2.1b. Each node in a multi-hop network can act both as an end point and as a relay station. Nodes are considered the endpoints when they are either the original sender, or the target node. When sending a new message to another node, the sender is considered the original sender, and the other node is called the target node. Every node that the message

(10)

travels through is considered a relay station. When a relay station receives a message, it uses its information about the network and the information inside the message to decide which node(s) it needs to forward the message. The way this information is stored is implementation specific and thus is explained in Section 3.4.

(a) Fully connected (single-hop) network

A B C D E (b) Multi-hop network A B C D E

2.2 Previous implementation

Research on distributed ad hoc networks has already been done. An example is ”Ad hoc wireless distributed computing using PYNQs” [11]. In this paper, they outline how an ad hoc network of low-cost, low-power computing nodes can be used as a distributed network. First, they set up a network of PYNQ-Z1 boards (3.2). They then test the ability of this network by naive matrix multiplication. With this, they show that this network can indeed be used to offer improvements in performance.

This implementation supported single-hop connection and assumes that the nodes used are stationary, and therefore are always connected to other nodes. This is done because the imple-mentation is not focussed on a network for a drone swarm. For a mobile drone network, however, this limits the movement of the drones too much. Therefore, this thesis expands on the ideas used and implement a multi-hop ad hoc network.

(11)

CHAPTER 3

Network design & implementation

The first research question is ”How can a communication network for a drone swarm be implemented?” (RQ1). This question is answered in this chapter. For every section, the design choices are explained, and then the implementation itself is given.

3.1 Choosing communication type

Range, bit-rate, and flight time are among the most important for the drone swarm to func-tion optimally. This secfunc-tion first explains the importance of these and then determine which communication type is best to use.

First, the importance of range is explained. The minimum range that has to be maintained between drones is dependent on how fast a drone can react to detecting an imminent collision. This range is therefore dependent of the following: the speed at which the drones fly (vmax), the

number of drones in the network (N ), and the measurement error (ϵ). A minimum range can be calculated using the following formula: [12]

dmin = 0.288· N · vmax+ ϵ

With an expected measurement error of no more than 10 centimetres, an expected max speed of 2 m/s, and 10 drones, this would mean that the minimum range needs to be 5.64 meters. Any shorter distance would result in collisions. Apart from this, it would also be useful for drones to communicate over greater distances. That way, drones can stay connected without the need for a relay station, which leads to the network needing fewer hops on average for a message to go between two drones.

The second property is bit rate. The required bit rate differs on which swarm activity the drones are used. The network for SA1 and SA2 distributes the images that need to be processed. For nodes to distribute this in time, a high bit rate is essential. The need for a high bit rate then arises, as the more bits per second can be transferred, the more data can be offloaded to other devices, which would mean that more data can be processed. The bit rate for SA3 is entirely dependent on the bit rate used by the end users.

Lastly, the maximum flight time of the drone is also important. The drones used in this project will have a maximum flight of 15 to 20 minutes. This is reduced further when adding more battery draining hardware.

Before comparing the communication type, all wired types can be excluded. Drones need to be mobile, and a wired connection between them restricts them too much. With this in mind, the most common wireless communication types are Bluetooth, Wi-Fi, and Infrared (IR). These types are compared in Table 3.1.

(12)

Table 3.1: Comparison of the communication types [13] [14] [15] [16]

Type Wi-Fi Bluetooth Infrared (IR)

Range (m) 30 - 100 5 - 30 5 - 10

Bit rate (Mbps) 600 * 24 1 - 16 **

Battery impact (KB/mWs) ∼ 1.7 *** ∼ 0.2 *** Unknown

* 600 Mbps is theoretical maximum for the 802.11n standard (from 2009). 802.11ac has an even higher bit rate of 1.3Gbps.

** 1 Mbps in broadcast mode and 16 Mbps in targeted mode. *** This is a rough estimate on full throughput

It is expected that for SA1, there is a need to process images with the use of a distributed network. Images that are used will most likely be between 1Mb and 100Mb. This means that, with the theoretical maximum transfer rate, Wi-Fi will take at most 1

6 of a second to transfer

it over the network, which translates to a transmission power of 6 images per second in perfect conditions. For Bluetooth this is around 1 image each 4 seconds and IR takes either 6.25 or 100 seconds to transfer a single image.

With a minimum needed range of 5.64 meters, both Bluetooth and IR can fall short. This means that, for those types, the swarm cannot consist of 10 drones or more, or the drones cannot fly at 2 m/s.

With the largest rage, highest maximum bit rate, and lowest battery impact, Wi-Fi shows the most potential. Because of this, the network will be implemented over Wi-Fi. However, this does not mean that there are no downsides to using Wi-Fi over Bluetooth or IR. Because Wi-Fi networks are already used a lot, it means that in areas with a lot of other Wi-Fi networks, Wi-Fi could have much interference in the network. In these cases, it could be useful to have Bluetooth as a backup.

3.2 The node composition

Several components are needed to build a communication network for drones. The drones are equipped with an autopilot and a 5000 mAh battery. This makes the flight time of the drone between 15 and 20 minutes.

The processing power will be delivered by the PYNQ-Z1 1_{. These boards hold 512MB of}

DDR3 memory, include an Arduino header, and have a 650MHz dual-core Cortex-A9 processor2_.

This processor has 256 KB on-chip memory, 512 KB of L2 cache, and 32KB L1 cache3_{. This is}

expected to be enough to process the basic information needed to operate in a drone swarm. The PYNQ-Z1 runs a Debian based operating system created by Xilinx, hereafter referred to as Xilinx OS. This OS adds a library that enables developers to program the Arduino header through a python interface.

For the implementation of this network works for both Wi-Fi and Ethernet. However, the experiments in this paper are done over an Ethernet connection.

3.3 Implementation of the network

This section explains how a multi-hop Wi-Fi ad hoc network can be implemented on a drone swarm. First, the choice between using pre-existing infrastructure and operation without is discussed. Then, it explains the implementation of neighbours discovery and the connection protocol. Finally, the implementation details for multi-hop support are described.

1_{http://www.pynq.io/board.html}

2_{https://reference.digilentinc.com/_media/reference/programmable-logic/pynq-z1/pynq-rm.pdf} 3_{https://www.xilinx.com/support/documentation/data_sheets/ds190-Zynq-7000-Overview.pdf}

(13)

3.3.1 Connecting the drones

As stated in the problem statement, the drone swarm cannot rely on any other infrastructure besides themselves. This means that they cannot communicate with an access point (AP). In the default Wi-Fi mode (basic service set, or BSS), devices cannot communicate without an AP. Therefore, the devices use the independent basic service set (IBSS) Wi-Fi mode. This allows for devices to act as an AP and be connected to other APs (which are other drones) at the same time.

3.3.2 Detecting neighbours and setting up a connection

When every device is in IBSS mode, they can start to communicate over a single network. For devices to connect, they need to set their frequency to the same frequency. There are two options, either in the 2.4Ghz range, or the 5Ghz range. The best frequency is dependent on the situation that the drones fly in. 5Gz has a higher bit rate, but has a lower range when a wall or other objects obstruct the connection 4_{. This implementation used 2.412 GHz as the 2.4 GHz range}

was the only supported one on the Wi-Fi dongles.

After the drones have chosen the same frequencies, they broadcast their IP over the network. This is to make other drones aware that there is a new node available to create a connection with. This broadcasting is done using the User Datagram Protocol, or UDP. This is a protocol that does not need any initialization as it is transaction orientated. The downside of this, however, is that UDP does not guarantee the delivery of every sent packet [17]. However, this is not a problem, as drones continuously broadcast their IP. This means that if one packet will not be received, one of the following packets most likely will be.

When a drone detects a broadcast message, it means that the other drone is in Wi-Fi range, and a connection between the two drones can be created. The disadvantage stated earlier with UDP does become a problem. Therefore, this connection is set up using TCP [18]. This protocol can detect if all the data was transmitted successfully. When it detects a failure, it tries to send the data again until it is sent over correctly. The only way packets can be lost is when they disconnect. This can happen when drones fly too far apart or when the Wi-Fi dongle malfunctions.

3.3.3 Support for multi-hop

Multi-hopping is done by using other nodes in the network to forward messages to the correct destination. To make forwarding possible in the network, each node needs to be equipped with information about how to reach every possible destination. For this a routing table is used. This table stores information to which node messages need to be forwarded in order to be received by the destination or target node.

The routing table for this network contains a list of all reachable devices in the network, either reachable via another node or directly. Per reachable device an entry that contains three values: the target node (or destination), the next-hop, and the hop count. The target node is the reachable device. The next hop is used to determine to which node a message has to be send to, in order for it to be received by the target node. Hop count is the number of hops needed to reach the destination. Using this, guesses about time of arrival can be made, and more importantly, the shortest path can be enforced with the least amount of hops. In Figure 3.1 an example displaying the routing table of node A in the network.

(14)

Figure 3.1: Routing table of node A belonging to network on the right

Routing table of node A

Destination Next-hop Hop count

B B 1 C C 1 D C 2 A B C D

The routing table of a drone is updated when one of the following events take place: (1) a new connection from or to the drone is established, (2) a connection in the network is established, (3) a connection between from or to the drone is lost, or (4) a message is received that a connection in the network is lost.

Once a connection between two drones is established (1), both devices add the other device to their routing table. The added entry will has the other node as both the destination and the next-hop, and the hop count as 1. This is also considered a single-hop connection. Next, two functions will execute in asynchronous order: (1.1) both nodes send that a new node in the network is found to all other entries in their table, and (1.2) both nodes communicate their routing table to the other node. For (1.2), when the other node receives the routing table, it will check for each entry whether it already has a connection with that node, or if it is a new node. Figure 3.2: Example of how A would react when a new connection between A and C is created.

Visualization of the network

A B

C

(a) Step by step execution of the protocol starting at node A

1.1 A notifies its network that a new connection is created.

1.1.1 A sends a message to B that the connection to C is created. 1.1.2 B adds the path through A to C with 2 hops.

1.2 A communicates its routing table to C

1.2.1 A sends its routing table to C, only containing the connection with B.

1.2.2 C adds the path through A to B with 2 hops.

This process per entry will be called the add node protocol and is described as follows. If the received node is not yet in its table, it puts that entry in its table, with next-hop set to the other node, and hop count to the received hop count + 1. If it already has another connection, it first compares its hop count to the received hop count. If its hop count is lower or the same than the received one, it does nothing. However, if its hop count is higher, it changes the entry by setting the next-hop to the other node and set the hop count to the received hop count + 1. How this protocol is executed is shown in Figure 3.3 from step 6 to 9.

After an entry is updated or inserted, the node also forwards this entry to the other entries in the routing table. This is to let the rest of the network know that there is a new node or at least a new path over which can be communicated. This leads to the other nodes receiving a message that there is a new connection in the network (2). When this is received, it executes the same add node protocol as done in (1.2).

When a connection is lost between two nodes (3), both nodes remove that entry from the routing table, and notify the other entries in its routing table that the connection is lost. When the other nodes in the routing table receive this message (4), they check if its next hop was the

(15)

the other path is not (yet) found to be disconnected. Because of this, the node sends an add node

protocol message back containing the new route. In Figure 3.3, an example of how the network

would react when the connection between nodes A and E would break. In this example, only the A side is explained, and it is assumed that all routing tables are fully satisfied with the shortest paths only.

Figure 3.3: Example of how node A would react after A and E disconnect.

Visualization of the network

A

B C

D E

Step by step execution of the protocol starting at node A

1. A removes E from routing table.

2. A sends message to B that the connection to E is lost. 3. B removes E from routing table.

4. B sends message to C that the connection to E is lost. 5. C detects that its path goes through D instead of B, and

thus does not change its routing table.

6. C sends a message to B that there is a path to E with 2 hop.

7. B adds the path through C to E with 3 hops.

8. B sends a message to A that there is a path to E with 3 hops.

9. A adds the path through B to E with 4 hops.

3.4 Sending messages

The IBSS mode enables a single-hop connection between devices without any AP. A requirement for this network, however, is that it needs to support multi-hop. This means that messages need to contain information that makes it possible to determine if, and to whom messages need to be forwarded. Furthermore, for broadcasting messages over the network, the message protocol should try to minimize the amount of data being transferred over the network, while maintaining a low packet loss ratio.

With those requirements in mind, the messages are built as follows: Table 3.2: Message header with payload attached.

message id (4 bytes) message type (1 byte)

original sender (2 bytes) number of destinations (2 bytes) payload length (4 bytes)

destinations (number of destinations∗ 2 bytes)

payload (payload length in bytes)

• message id. This is a unique message ID per node. When a message cannot be received because of a lost connection somewhere in the multi-hop route, the message ID returns to the original sender. This could be useful when expanding on this network but is not used for this implementation.

(16)

• message type. The type of message is to determine how to process the payload itself. Examples of these are ”TABLE_UPDATE” which is used in the add node protocol and ”LOST_PATH” which is used when a connection is lost.

• original sender. This is the node which sent the message. This is used to determine the next hop when updating the routing table in the add node protocol. Furthermore, this can be useful for implementations on top of this network. For example, to reply to the original sender.

• number of destinations. This is the number of destinations. With this, the size of the destinations can be calculated.

• payload length. The length of the full payload in bytes. This is to detect when the next message begins.

• destinations. Each message is equipped with a list of destinations. When sending a message to multiple destinations, the sender groups the destinations via a destinations group method. First, it splits the destinations into groups. Where each group contains only destinations that have the same next-hop in the routing table. Then, it builds a message for every group. This means that it only has to send one message per next-hop instead of one message per destination. This is to reduce the total amount of messages sent over the network. When a message is built, it is then split into parts of 4KB each, which is the same size as the buffer of the receiver. This is so the receiving end has the same amount of memory reserved as the sender is sending. After it is split, each part is put into a priority queue before it is sent. This way, more important messages can get priority over others, and thus quickly be sent, even with an already filled queue. When receiving a message, the receiver first reads the payload size from the header. Then the receiver can stitch all the parts it received through their buffer together until it received all the bytes. When the data is received, the receiver rereads the header and handles the message accordingly. First, it reads the destinations to know if and where to forward it to. Then, if the receiver was one of the destinations, it uses the message type to determine how the process the payload. For the other destinations, it applies the destinations group method as described in the build process to the destinations. After the receiver created new groups, it forwards the message to these groups via their next-hops.

(17)

CHAPTER 4

Experiments on the network

This chapter first outlines which experiments need to be done to answer RQ2. After that, it gives an overview of the equipment that is used to run the experiments. And lastly, it gives the results of each experiment with a short explanation. The further discussion of the results are found in the discussion.

4.1 Experiments

Could the implemented network be used as a distributed network to decrease computation time? (RQ2) For this to be the case, the transfer time of the data send and received needs to be lower than the time won by distributing. The problem with this, however, is that it is dependent on the swarm activity. Activities that require a lot of data transfer and not many calculations are likely to be limited on the transfer rate, whereas activities with a small amount of data and many calculations are mostly affected by the processing power. As activities differ from the amount of data that needs to be transferred, the transfer rate on different payload sizes needs to be tested. Secondly, it depends on the formation of the drones and the number of drones in the network, as this will most likely also impact the efficiency of the network. With more drones in the network, there is more processing power in total, and with a denser formation, the data can probably be offloaded more quickly.

The impact of the following types of data and network configurations were tested to test the viability of RQ2. To test RQ2, experiments on the transfer rate have been conducted. For a distributed system, the following impacts on the transfer rate are of interest:

4.1 Transfer rate against different payload sizes. This is to find the optimal payload to send to each node in the network. With this, a payload size can be advised to the implementations of the SA. It is expected that the transfer rate increases as the payload also increases. This is because the overhead of composing the message matters less, the more data is being transferred.

4.2 Transfer rate against different sizes of a single-hop network. In a distributed network, the same payload is spread over multiple nodes in the network. These nodes can be either single-hop or multi-hop. With the impact of the number of neighbours, an indication of the outgoing transfer time can be given. It is expected that the more neighbours are added to the network, the lower the transfer rate will be, as broadcasting cannot be fully done in parallel.

4.3 Transfer rate against different numbers of hops. Testing the impact with different numbers of hops can indicate how many hops is useful to use in the distributed network. It is expected that the transfer rate will be at least 1

hops of the transfer rate with just 1

hop. This is because a node waits until it receives the full payload before forwarding it to the next node.

(18)

4.4 Sending transfer rate against receiving transfer rate. When devices are finished with their computation, they send the result back. This experiment tries to find an optimal result size to be returned. It is expected that receiving transfer rate from multiple neighbours will be lower than the sending transfer rate. This is because the received data cannot be stored in parallel, where sending data only has to be stored once.

4.2 Setup

All these experiments run on two to eight PYNQ boards (from section 3.2). These boards are connected over Ethernet CAT-5 cables, using two network switches to connect them. The setup can be seen in Figure 4.1. Before each experiment is run, each node in the network synchronizes their routing table, to avoid any setup overhead in the results. Furthermore, all experiments are run with different payload sizes. The sizes start at 68.6KB and go up to 4MB in 32 linear steps. Then, for each payload size, the payload is transferred 10 times. With this, an average and standard deviation can be calculated. The transfer time is measured from the start of sending a payload to receiving an acknowledgement that it has been received. Keep in mind that this does include extra acknowledgement time.

(19)

4.3 Results

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0 Payload size (MB)

0

1

2

3

4

5

6

7

8 Trasfer rate (MB/s)

Impact on outgoing transfer rate with different payload sizes

1 neighbour

Figure 4.2: Experimentation with one neighbour on different payload sizes. Line shown is the average transfer rate of 5 iterations per payload size, with the standard deviation as area sur-rounding the line.

(20)

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0 Payload size (MB)

0

1

2

3

4

5

6

7

8 Transfer rate to all neighbours (MB/s)

Impact on outgoing total transfer rate with n neighbours and different payload sizes

1 neighbour

2 neighbours

3 neighbours

4 neighbours

5 neighbours

6 neighbours

7 neighbours

Figure 4.3: Experimentation on different numbers of neighbours. This is the transfer rate for a broadcast message to all neighbours. This means that the actual data that is sent, it this transfer rate * the number of neighbours it is sent to. This can be seen in 4.4. In this figure, the lines shown are the average transfer rate of 5 iterations per payload size, with the standard deviation as the area surrounding the lines.

(21)

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0 Payload size (MB)

0

2

4

6

8

10 Transfer rate per node (MB/s)

Impact on outgoing transfer rate per node with n neighbours and different payload sizes

7 neighbours

6 neighbours

5 neighbours

4 neighbours

3 neighbours

2 neighbours

1 neighbour

Figure 4.4: Alternate view of 4.3. This is the transfer rate per node. When the same data is send to multiple neighbours, this is the average transfer rate per node.

(22)

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0 Payload size (MB)

0

1

2

3

4

5

6

7

8 Transfer rate to furthest hop (MB/s)

Impact on outgoing transfer rate with n hops and different payload sizes

1 hop

2 hops

3 hops

4 hops

5 hops

6 hops

7 hops

Figure 4.5: Experimentation on different number of hops. This shows the transfer rate to the furthest hop. Lines shown are the average transfer rate of 5 iterations per payload size, with the standard deviation as area surrounding the lines.

(23)

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0 Payload size (MB)

0

2

4

6

8

10 Transfer rate when broadcasting to all hops (MB/s)

Impact on outgoing transfer rate with n hops and different payload sizes

7 hops

6 hops

5 hops

4 hops

3 hops

2 hops

1 hop

Figure 4.6: Experimentation on different number of hops. This shows the transfer rate when every hop is also considered a destination. Lines shown are the average transfer rate of 5 iterations per payload size.

(24)

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0 Payload size (MB)

0

1

2

3

4

5

6

7

8 Trasfer rate (MB/s)

Impact on incoming transfer rate with n neighbours and different payload sizes

1 neighbour

2 neighbours

3 neighbours

4 neighbours

5 neighbours

6 neighbours

7 neighbours

Figure 4.7: Experimentation on receiving payload sizes. Lines shown are the average transfer rate of 5 iterations per payload size, with the standard deviation as the area surrounding the lines.

(25)

CHAPTER 5

Discussion

5.1 Discussing results

5.1.1 Optimal payload size

In the beginning, Figure 4.2 shows an increase in transfer rate to around 256KB. The most likely cause of this appearance is the extra overhead needed. With smaller payload sizes, the overhead has a bigger effect on the average transfer rate, compared to larger payload sizes. This overhead consists of, but is not limited to, the acknowledgement message, building the message, and the initialization of transferring a message. After the first increase in transfer rate, the transfer rate slowly starts to decrease at around 512KB. This is also the same size as the size of the L2 cache on the CPU. The cache is used to access data that, when not in the cache, would take more time to access. Therefore, when the cache is full, the transfer rate could also slow down. Larger payload sizes cannot all be stored on the relatively small cache size and would have to be accessed from RAM. With this, it can be concluded that message sizes between 136KB and 512KB show would be best to transfer.

Using other nodes to distribute data to, is mostly only useful when the time spend transferring the data is less than the time reduced by distributing it. With this in mind, the following equation can be formed to determine the minimal reduction time needed for

∆ttotal> ∆tdist+ ∆ttrans (5.1)

∆ttrans= Q· d (5.2)

Where ∆ttotal, the total time it takes for the computation to complete, is is compared to the

sum of ∆tdist, the time the computation takes when distributed and ∆ttrans, the total transfer

time is. Transfer time is the transfer rate (Q) multiplied by the data transferred (d).

For example, if d is 512 KB, ∆tdisthas to be at least ₁₄1 seconds shorter then ∆ttotalto make

sharing it to a neighbour worth it. However, with 2 MB, the reduction has to be at least 5 4

seconds.

5.1.2 Number of neighbours

From Figure 4.3, the same trends as in 4.2 can be seen. First, an upward motion, then the transfer rate goes down again. However, the figure also shows that this trend is less visible the more neighbours are added to the network. Furthermore, this graph also clearly shows that the amount of neighbours influences the transfer time differently per payload size. Small payload sizes are more influenced than larger payload sizes. This could be since sending the same data over a network to multiple neighbours is not limited by the memory speed. This is because the data only needs to be loaded once, after which it can be transformed to all neighbours.

When sending data to all neighbours, the transfer rate can be determined in two ways. The total transfer rate, as depicted in Figure 4.3, shows how much data can be broadcasted per

(26)

second. Alternatively, the transfer rate per node as depicted in Figure 4.4, this is the transfer rate to a single neighbour. This second transfer rate is the most interesting when considering to distribute the data over the network. Figure 4.4 shows that, at least up to four nodes, the more neighbours are used, the faster the transfer rate per node will be. After four nodes, the transfer rate does not improve that much anymore. This is most likely because the slowdown is the cache is minimized, and the only overhead is from the network itself.

5.1.3 Number of hops

The results in Figure 4.5 show that the transfer rate decreases a lot when adding an extra hops. Even with larger payload sizes, it does not seem to get closer to the transfer rate. Even when all the hops are considered destinations as well (as shown in Figure 4.6) the transfer rate decreases. Only when considering all hops as destinations, and having large payload sizes, it seems to get closer to the transfer rate of one hop. However, it does not seem that it will also get faster.

It can be concluded that using nodes multiple hops away is not as viable as using single-hops. A solution would be to use the ability of other drones to physically more close and thereby get in range of the node. This would create a faster single-hop connection instead of the slower multi-hop connection.

5.1.4 Receiving transfer rate

When comparing Figure 4.3 to 4.7, the same transfer rate can be seen for payload sizes up to around 400KB. However, after that, the transfer rate is lower when receiving compared to sending data. Therefore, it can be stated that for a distributed network the result should always aim to be smaller than the initial data. Otherwise receiving the results will be the bottleneck when using the network for distribution.

The optimal receiving payload size seems to be around 512KB. This is independent of the number of neighbours, as for every neighbour the transfer rate slows down after 512KB. With this, it can be stated again that the payload should be with a maximum size of 512KB for an optimal transfer rate.

5.2 Conclusion

The first research question is ”How can a communication network for a drone swarm be imple-mented?”. Chapter 3 has presented a way to implement such network. Each node in this network is equipped with a PYNQ-Z1 for the processing power and communication. This network is build upon an Ad Hoc connection between the drones in the network over a Wi-Fi connection. The network supports sending messages via both single-hop and multi-hop connections. Further-more, the headers of each message are designed to support sending the same payload to multiple destinations efficiently.

With a working implementation of such network, the network is tested to answer the second research question: ”Could the implemented network be used as a distributed network to decrease computation time?”. Chapter 4 tested this network for its ability to be used as a distributed network. It is tested on different network sizes with both single- and multi-hop connections. It can be used as a distributed network, but the effectiveness is entirely implementation specific. This network is useful when the reduced implementation time is more than the transfer time. On average it is useful when less than 7 MBps is processed with payload sizes smaller than 512 KB, or around 2 MBps with sizes larger than 2 MB. Using devices multiple hops away are generally not useful as this does reduce the transfer rate significantly, and should only be used if multiple nodes can offer computation time reduction when processing the same data.

(27)

5.3 Future research

5.3.1 Shortcomings of the implementation

Although the network is fully functional, it is the first implementation of this communication network. There are still areas within the network that could be improved upon. Currently, a node starts forwarding a message only after it has received the full message. In theory, a relay station could start forwarding the moment it decoded the header. This could offer a large speedup, especially for broadcasts with a large payload size.

Furthermore, the network itself could be tested more. Currently, this thesis only tested network sizes up to 8 nodes. In these results, it looked like adding nodes would only improve the transfer rate. With more nodes in the network, the limit of connections could be found as well. Furthermore, with more nodes in the network, different network formats could be tested as well.

5.3.2 Expansion of the implementation

This thesis has presented an implementation of a communication network between drones. How-ever, this network is not yet actually implemented for the swarm activities stated in the intro-duction. For all of these activities to be implemented, the communication protocols still have to be designed. These are partially activity specific, but some are also needed for every drone swarm. These are protocols that decide how to avoid each other, or how to decide where to go. The mobility of the drones could be used to improve the transfer rate between endpoints. This could be done by letting the sender and destination node fly to each other, which would reduce the amount of hops between node and thereby increasing the transfer rate. This is specifically useful for a distribution network. When trying to distribute data, some drones in the network could come close to help processing.

(28)

(29)

Bibliography

[1] Dave Mosher. “”Hot Bee Balls” Cook Enemy Hornets—But How Do Bees Endure the Heat?” In: National Geographic (Mar. 2012).

[2] Tomer J Czaczkes, Christoph Grüter, and Francis LW Ratnieks. “Trail pheromones: an integrative view of their role in social insect colony organization”. In: Annual review of

entomology 60 (2015).

[3] Pitcher TJ and Parish JK. Functions of shoaling behaviour in teleosts. Behaviour of teleost

fishes. 1993.

[4] Justin Werfel, Kirstin Petersen, and Radhika Nagpal. “Designing Collective Behavior in a Termite-Inspired Robot Construction Team”. In: Science 343.6172 (2014), pp. 754–758. issn: 0036-8075. doi: 10.1126/science.1245842. eprint: https://science.sciencemag. org / content / 343 / 6172 / 754 . full . pdf. url: https : / / science . sciencemag . org / content/343/6172/754.

[5] Yu Zhou. “Art and Engineering Inspired by Swarm Robotics”. In: (Apr. 2017).

[6] Gábor Vásárhelyi et al. “Optimized flocking of autonomous drones in confined environ-ments”. In: Science Robotics 3.20 (2018), eaat3536.

[7] Wanlei Zhou Weijila Jai. Distributed Network Systems. From Concepts to Implementations. Springer, 2005.

[8] Robin Jan Maly et al. “Comparison of centralized (client-server) and decentralized (peer-to-peer) networking”. In: Semester thesis, ETH Zurich, Zurich, Switzerland (2003), pp. 1– 12.

[9] Xiang Ji and Hongyuan Zha. “Sensor positioning in wireless ad-hoc sensor networks using multidimensional scaling”. In: IEEE INFOCOM 2004. Vol. 4. Mar. 2004, 2652–2661 vol.4. doi: 10.1109/INFCOM.2004.1354684.

[10] M. Conti and S. Giordano. “Multihop Ad Hoc Networking: The Theory”. In: IEEE

Com-munications Magazine 45.4 (Apr. 2007), pp. 78–86. issn: 0163-6804. doi: 10.1109/MCOM.

2007.343616.

[11] Marco Brohet et al. Ad hoc wireless distributed computing using PYNQs. Feb. 2018. [12] Tom Kersten. Sound-based Relative Position Detection in Drone Swarms. June 2019.

[13] Bluetooth SIG. Bluetooth Core Specification. url: https://www.bluetooth.com/specifications/ bluetooth-core-specification.

[14] Grove - Infrared Emitter. url: http : / / wiki . seeedstudio . com / Grove - Infrared _ Emitter/.

[15] Wireless Communication Technologies Types and Advantages. url: https://www.efxkits. us/different-types-of-wireless-communication-technologies/.

[16] R. Friedman, A. Kogan, and Y. Krivolapov. “On Power and Throughput Tradeoffs of WiFi and Bluetooth in Smartphones”. In: IEEE Transactions on Mobile Computing 12.7 (July 2013), pp. 1363–1376. issn: 1536-1233. doi: 10.1109/TMC.2012.117.

[17] J. Postel. User Datagram Protocol. STD 6. http://www.rfc-editor.org/rfc/rfc768. txt. RFC Editor, Aug. 1980. url: http://www.rfc-editor.org/rfc/rfc768.txt.

(30)

[18] Jon Postel. Transmission Control Protocol. STD 7. http://www.rfc-editor.org/rfc/ rfc793.txt. RFC Editor, Sept. 1981. url: http://www.rfc-editor.org/rfc/rfc793. txt.

(31)

Appendix

A PYNQ Python Interface

Microphone Interface

As of the writing of this thesis, the current implementation of the microphone interface on the PYNQ-Z1 does not sample with the correct sample rate. The python interface assumes a sample rate of 192000 bits per second. While this is a close estimation, the actual sample rate differs from this. Sample rates of up to 195300 bps were found. This means that the samples would be played back at roughly 101.7 % of the actual playback speed. This led to a constant time inaccuracy when detecting for sound signals in the recordings.

An issue has been opened on the official GitHub repository1_{. However, the issue has not yet}

been resolved. In the meantime, the library was edited to continuously improve the accuracy of the sample rate based on previously recorded sample rates. This improvement can be found in the GitLab repository2_.

Grove Adapters

The PYNQ-Z1 has access to both PMOD hardware pins and an Arduino header. Most parts of the python interface that deal with these subsystems assume the usage of the both the PYNQ Grove Shield and the PYNQ Grove Adapter3_{. Over the course of the project, it was established}

that these adapters are not necessary for the usage of most systems within the PYNQ. Due to a lack of support in the documentation for working without these adapters, working with the python interface became time-consuming. As these extensions are not a necessary part of the PYNQ, it would stand to reason that the interface should support working without these extensions and provide documentation for doing so. For this project, it is essential to keep the payload of the drone to a minimum to extend flight time. Because of this, these adapters were not used.

B Wi-Fi Dongles

During the implementation of the communication network for the drone swarm, problems with the compatibility of Wi-Fi dongles were encountered. As stated in this thesis, Wi-Fi is proposed as the medium of communication. However, it was quickly established that the Wi-Fi dongle that was used at the time, the Edimax N1504 _{(the EW-7811Un wireless adapter), would not}

function as the operating system of the PYNQ did not recognise it. It was then decided to try a different Wi-Fi dongle, which also did not work. In the end, a total of four dongles were tested: the aforementioned EDIMAX N150, the EDUP EP-AC16015_{, the tp-link Archer T2U}6

1_{https://github.com/Xilinx/PYNQ/issues/862} 2_{https://gitlab-fnwi.uva.nl/11299673/droneswarm} 3_{https://store.digilentinc.com/pynq-grove-system-add-on-board/} 4_{https://www.edimax.com/edimax/merchandise/merchandise_detail/data/edimax/in/wireless_adapters_} n150/ew-7811un/ 5_{http://www.szedup.com/product-item/ac-dual-band-wifi-usb-adapter/} 6_{https://www.tp-link.com/us/home-networking/usb-adapter/archer-t2u/}

(32)

and finally the tp-link TL-WN727N7_{. Of these four, only the oldest model, tp-link TL-WN727N,}

was compatible. This dongle did not advertise native Linux support, while both the EDIMAX N150 and the tp-link Archer T2U did state it has native support. However, it seems that only certain kernels are supported and that there is only limited backwards compatibility.

C

Network Implementation Usage

The implemented network can be found in the GitLab repository of the University of Ams-terdam8_{. In this repository, the folder network contains the files that are needed to use the}

communication network. From here, the class Meshnet can be imported to use the network. An example use can be found in example/payload_test.py.

A Distributed Ad Hoc Network for Drone Swarms

Bachelor Informatica