Population structure in a 2-player, m strategy game

(1)

University of Amsterdam

Master’s Thesis Behavioral Economics and Game Theory

Population Structure in a 2-player, m

Strategy Game

Kelvin van Ham, 11399554

supervised by prof. dr. C.M. Van Veelen

(2)

Abstract

This paper departs from the standard 2-player, 2 strategy, random matching setting in the analyses of the replicator dynamics. We extend the model of Van Veelen to m strategy games. Also, we find different ways to structure populations in 2-player, m strategy games. Furthermore, it provides an example of a change in the replicator dynamics due to a different population structure. Lastly, we find an additional, intuitive restriction guaranteeing a unique solution for the model.

This document is written by Kelvin van Ham who declares to take full responsibility for the contents of this document. I declare that the text and the work presented in this document is original and that no sources other than those mentioned in the text and its references have been used in creating it. The Faculty of Economics and Business is responsible solely for the supervision of completion of the work, not for the contents.

1 Introduction and Research Question

There has been a lot of discussion about the evolution of cooperation and defection in previous literature (Wynne-Edwards, 1962; Dawkins, 1989). In models with a public goods or prisoners dilemma game setting, groups of cooperators beats groups of defectors but within a group, defectors always beat coop-erators (Wilson and Wilson, 2007). In the standard prisoners dilemma, the Nash equilibrium is to always defect. This is also the only Evolutionary Stable strategy (ESS). In a population where cooperators and defectors are randomly matched, cooperators will go extinct over time. Because they always pay the cost of cooperation, but the benefits are also reaped by the defectors. Most literature on the replicator dynamics uses a 2-player, 2-strategy setting with random matching (Weibull, 1997).

However, populations can be structured in a way that makes cooperation dynamically stable. If cooperators live closely together or if cooperators have some form of kin recognition in order to keep cooperation to their own kind and thus only cooperate with other cooperators. Van Veelen (2009) de-scribes such a model, with relatedness as a parameter for population structure. The model shows that cooperation can be stable in the replicator dynamics for a sufficiently high relatedness. Then the average payoff of cooperators becomes larger than the average payoff of defectors. This model is usually applied to 2-player games with 2 strategies, cooperate and defect. Van Veelen (2009) finds a more general ap-plication of the replicator dynamics, by adding population structure and n-player games. This paper is adding to the existing literature by investigating 2-player games with m strategies.

In 2-player games with 2 strategies, which is what most models use, there is only one way to form groups given assortment parameter r and the frequency of cooperators p. It turns out that for games with n > 2 players given r and p, there may be multiple ways to form groups, implying different results in the replicator dynamics.1 _{Something that has not been investigated is what happens in 2-player games}

with m strategies. Examples of such games are the extended battle of the sexes game and the RSP-m game. If there are different ways to add population structure given the parameter r, then this can also lead to different results in the replicator dynamics. This leads to the following research question:

Is there a unique solution in terms of group formation in 2-player games with m strategies and popu-lation structure?

First, we present a version of the model in Van Veelen (2009), adjusted to m strategy games. We then show two intuitive ways to arrange groups in 3-player games. We find that for 2-player with games more than 3 strategies, there are an infinite number of ways to formulate groups. This paper will explore some of these different ways to form groups, find an additional restriction to ensure a unique solution and give an example where different group formation leads to changes in the replicator dynamics.

1.1 The Model

Van Veelen (2011) describes a model that uses relatedness r as a parameter for population assortment and group formation. With two players per group and two strategies (Cooperate and Defect), groups can consist of 2 cooperators, 1 cooperator and 1 defector, or 2 defectors. They can be formed by random matching (r = 0), clonal interaction (r = 1) or any relatedness between 0 and 1. This paper will name the strategies A, B, .., m. This is, apart for the aesthetic reasons, also because cooperate and defect have a certain connotation. This paper is not about specific games and strategies, but rather about group formation with m strategies.

Van Veelen (2011) applies the model to n-player games with 2 strategies. The notation used in this paper is slightly different to make it suitable for analyzing 2-player games with m strategies. The popu-lation will be characterized by its shares of groups of different compositions. Groups are denoted by fij,

with i = A, B, .., m , j = A, B, ..., m and i ≤ j to avoid double counting of heterogeneous groups2_{. Hence}

fijis a 2-player group with i and j representing the type of players. For a 2-strategy game with strategies

A and B this leads to homogeneous groups faaand fbband heterogeneous groups fab. Note that we write

1_{A solution in terms of group formation f (p, r) is given by the frequencies of all groups that are formed. In other}

words, the way the population is structured. A unique solution implies that, given strategy frequencies p and population assortment parameter r, there is only one way to structure the population.

2_{We count the strategies starting from A, B, ..., m. Note that we do allow for more than 26 strategies. In the hypothetical}

(4)

faband fbaas one group named fab. As we are speaking of populations and shares, the sum of all groups

have to add up to 1. This restrictions can be written asP

i≤jfij = 1 with 0 ≤ fij ≤ 1 for all i, j. The

frequency of a strategy i is denoted by pi and can be calculated as follows: pi = fii+ 1₂Pi6=jfij. We

define p = (pa, pb, . . . , pm). Again, all frequencies pi together have to add up to 1 to form a consistent

population state. This restriction is given byPm

i=api= 1, with 0 ≤ pi≤ 1. Hence, p ∈ 4mif it satisfies

these conditions.

Relatedness is defined as r = P(T |T )-P(T |N ). This equation is based on a hypothetical chance experi-ment, where we randomly draw one individual from the population, and then another individual from the same group fij, i ≤ j. Here, P(T |T ) is the probability that, given that we have first drawn an individual

with trait T , the second drawn player also possesses this trait T . P(T |N ) represents the probability that, given that we first draw an individual that does not possess trait T , the second drawn individual does possess trait T . If there is no population structure (r = 0), this implies that P(T |T )=P(T |N ). Intuitively this makes sense, because players are randomly matched. The trait of the first drawn player does not provide extra information on the player that is drawn second. On the other hand, complete assortment (r = 1) implies P(T |T )= 1 and P(T |N ) = 0. One can see this as completely closed communities of same strategy players living together, never interacting with members of other communities. Alternatively, it can be seen as a perfect kin-recognizing ability, where each player only interact with its own kind. Lastly, r is constant and a characteristic of the whole population, meaning it does not depend on the strategy or its name tag. Thus ri= rj= ˆr ∀ i, j.

1.2 The Baseline Case: 2 players, 2 strategies

With groups of 2 players and games with 2 strategies, the composition of the population ˆf is uniquely determined by pa, the frequency of trait A and population assortment parameter r. The solution in terms

of group formation can intuitively be understood by analyzing two extreme cases of the model.

Suppose we have a population without population structure. Matching is random, implying r = 0. We now want to determine the frequency of group faa when randomly drawing individuals from the

population. For this group, we first need to draw an A-player. After this, given that we have drawn an A-player, we need to draw another A-player. The probability that we draw our first A-player from the population is simply given by the frequency of A-players in the population pa. Given this and the

random matching process, the probability that the second player (its match) is also an A-player is also pa. Hence, when r = 0, faa = p2a. By analogy, fbb = p2b. For the heterogeneous groups fab, we need to

draw an A-player first and then a B-player or vice versa. This results in fab= 2papb.

In a fully structured population (r = 1), there is clonal interaction. In other words, every individual is matched with an individual of the same type. Again, for groups faa, we need to first draw an A-player,

and given that this first drawn individual is an A-player, we need the probability that the second player is also of type A. The probability of drawing the first A-player is pa. Given that we have just drawn an

A-player, and A-players are only matched with other A-players, the probability that the second player is also of type A is 1. Hence, faa= pa, fbb= pb and there are no heterogeneous groups, implying fab= 0.

Combining these two extreme cases gives us the following solution for 2-player games with 2 strategies: faa= rpa+ (1 − r)p2a, fab= 2(1 − r)papb and fbb= rpb+ (1 − r)p2b.

This way of arranging groups f (p, r) turns out to be the unique solution for 2-player, 2 strategy games. This solution can, with some minor changes, be extended to any n-player, m-strategy game by doing a similar hypothetical chance experiment. Throughout this paper, we will refer to this (extended) solution as the standard solution.

(5)

Figure 1: The unique solution for faa, faband fbb in 2-player, 2 strategy games.

2 Different Ways to Apply Population Structure in 3-player, 2

strategy Games

Inspiration for this paper comes from an example of a 3-player game with 2 strategies that possess the same relatedness (r = 1

2) but lead to different group formation. A very appealing feature of this example

is that both solutions intuitively make sense when looking at the way they are formed.

The following example illustrates that when pa, pb and r are given, groups can be created in

multi-ple ways. We take r = 1₂in this example. The standard solution would give the following group formation. faaa= rpa+ (1 − r)p3a = 1 2pa+ 1 2p 3 a faab= 3(1 − r)p2apb = 3₂p2apb fabb= 3(1 − r)pap2b = 3 2pap 2 b fbbb= rpb+ (1 − r)p3b = 1 2pb+ 1 2p 3 b

Another way to design groups is given by a sexual haploid model. Here, parents are randomly matched to each other (r = 0), and give birth to groups of three siblings. The parent population is thus given by faa= p2a, fab= 2papb and fbb= p2b. For the offspring, this implies a relatedness of r =

1 2.

Figure 2: The haploid model where unrelated parents give birth to groups of 3 siblings. The arrows indicate the type of offspring groups created by each of the parent pairs.

(6)

We thus have the following frequencies for the offspring groups: faaa= p2a+ 1 4papb faab=3₄p2apb fabb=3₄pap2b fbbb= p2b+ 1 4papb

These ways of group formation lead to similar outcomes if and only if 1₂pa+1₂p2a= p2a+14papb. This

implies that, using pa+ pb = 1 ⇒ pb= 1 − pa, 1₄pa+1₂p3a−34p 2

a = 0 ⇒ 12pa(pa− 1)(pa− 1

2) = 0. Hence, the

group formations are only equal if pa∈ {0,1₂, 1}. In other cases both models provide a different solution

in terms of group formation when pa, pb and r are given.

3 A Unique Solution in 2-player, 3 strategy Games

We have seen an example of different population assortment in 3-player, 2 strategy games. The next step is to start looking at 2-player games with 3 strategies. The strategies are here denoted by A, B and C. Note that sometimes calculations have only been written out for strategy A. By analogy, the same conclusions can be inferred for the other strategies.

3.1 Conditions of the Model

For n=2, m=3, the following conditions must be satisfied: ra=rb=rc=ˆr

pa+pb+pc=1

pa= faa + 1₂fab + 1₂fac

pb= fbb+ 1₂fab + 1₂fbc

pc= fcc+ 1₂fac+ 1₂fbc

Relatedness r is defined as P(A|A) − P(A|¬A).

Figure 3: Groups in a 2-player, 3 strategy game. P(A|A) = 2faa+f2faaab+fac and P(A|¬A) =

fab+fac fab+fac+2fbb+2fbc+2fcc P(B|B) = 2fbb+f2fbbab+fbc and P(B|¬B) = fab+fbc fab+fbc+2faa+2fac+2fcc P(C|C) = 2fcc+f2fccac+fbc and P(C|¬C) = fac+fbc fac+fbc+2faa+2fab+2fbb

3.2 The Unique Solution

Here we use that pa= faa + 1₂fab+ 1₂fac, pb= fbb+ 1₂fab+ 1₂fbcand pc= fcc + 1₂fac+ 1₂fbc. Then the

(7)

P(A|A) = 2f2paaa = faa pa P(A|¬A) = f2pabb+f+2pacc = pa−faa pb+pc ra = P(A|A) − P(A|¬A) = f_paa a − pa−faa

pb+pc Solving this equation for faa gives us pa(pb + pc)ra =

faa(pb+ pc) − pa2+ pafaa⇒ pa(pb+ pc)ra+ p2a= faa(pa+ pb+ pc) ⇒ faa=

pa(pb+pc)ra+p2a

pa+pb+pc

By looking at this expression for faa we can already see that, when pa, pb, pc and ra are given,

there is a unique solution in terms of homogeneous groups fii. We now implement the constraints that

pa+ pb+ pc= 1 and ra= rb = rc = ˆr. Then faa = pa(1 − pa)r + p2a. Rewriting this expression leads to

faa= rpa+ (1 − r)p2a. It does not surprise us that this is the same solution as in Van Veelen (2009).

The next step is to find solutions in terms of heterogeneous groups fij,i<j. To find these solutions

some temporary variables will be introduced to make the matrix more orderly. pa= faa + 1₂fab + 1₂fac with fab+ fac= ca and ca= 2(pa− faa) pb= fbb+ 1₂fab + 1₂fbc with fab+ fbc= cb and cb = 2(pb− fbb) pc= fcc+ 1₂fac+ 1₂fbcwith fac+ fbc= cc and cc= 2(pc− fcc)   fab fac fbc  ×   1 1 0 1 0 1 0 1 1  =   ca cb cc  

This system of linear equations has a unique solution if and only if the determinant of the matrix is nonzero. 1 1 0 1 0 1 0 1 1 = 1 × 0 × 1 + 1 × 1 × 0 + 0 × 1 × 1 - 0 × 0 × 0 - 1 × 1 × 1 - 1 × 1 × 1 = -2 6= 0

This means that there is also a unique solution in terms of heterogeneous groups fij,i<j. The next

step is to solve the system of equations and find an expression or fab, facand fbc.

  fab fac fbc  ×   1 1 0 1 0 1 0 1 1  =   ca cb cc  ∼   fab fac fbc  ×   1 1 0 0 −1 1 0 0 2  =   ca cb− ca cc+ cb− ca   ∼   fab fac fbc  ×   1 0 0 0 1 0 0 0 1  =    ca+cb−cc 2 ca−cb+cc 2 −ca+cb+cc 2   

This implies that fab=ca+c₂b−cc. Now we write ca = 2(pa− faa) in terms of pa and r by substituting

the equation for faa that we derived earlier into the equation. 2(pa− faa) = 2(pa− (pa− p2a)r − p2a) =

2(pa− rpa+ rp2a− p2a) = 2pa(1 − r + rpa− pa) = 2pa(1 − r)(1 − pa) = 2(1 − r)(pa− p2a). Using similar

calculations for cband cc, filling in the equation for fabwill then give fab= pa−faa+pb−fbb−(pc−fcc) =

(1 − r)(pa− p2a+ pb− p2b− pc+ p2c). Using pc = 1 − pa− pb gives fab= (1 − r)(pa− p2a+ pb− p2b− 1 +

pa+ pb− 2pa− 2pb+ p2a+ p2b+ 2papb). Collecting terms gives fab= 2(1 − r)papb. Again, this does not

surprise us since this is exactly the result one would expect in the population structure model.

4 Unique Homogeneous Groups in 4-strategy Games

P(A|A) = fpaaa and P(A|¬A) =

pa−faa pb+pc+pc. Using pa+ pb+ pc + pd = 1, we define ra = faa pa − pa−faa 1−pa .

Rewriting gives faa= rpa+ (1 − r)p2a. Therefore this is also the unique solution for n=2, m=4.

(8)

pa= faa + 1₂fab + 1₂fac + 1₂fad with fab+ fac+ fad= ca and ca= 2(pa− faa)

pb= fbb+ 1₂fab + 1₂fbc + 1₂fbd with fab+ fbc+ fbd= cb and cb= 2(pb− fbb)

pc= fcc+ 1₂fac+ 1₂fbc+ 1₂fcdwith fac+ fbc+ fcd= cc and cc= 2(pc− fcc)

pd= fdd + 1₂fad + 1₂fbd + 1₂fcdwith fad+ fbd+ fcd= cc and cd= 2(pc− fcc)

This system has 4 equations and 6 unknowns. Note that we find no linear dependence in the equa-tions for pa, pb, pc and pd, which we prove in the next chapter. This means that we cannot eliminate

any of the equations without altering the solution set. A system of linear equations is considered under-determined if there are fewer equations than unknowns. As the system is consistent, it has an infinite number of solutions.

4.1 The Solution Set

fab= 1₂fbd+ fcd+1₂(ca+ cb− cc− cd)

fac= 1₂fbd+₂1(ca− cb+ cc− cd)

fad= −fbd− fcd+ cd

fbc= −1₂fbd− fcd+1₂(−ca+ cb+ cc+ cd)

If we choose a value for fbd and fcd, we essentially add 2 equations to the model, giving a unique

solution because then the number of equations and unknowns are both 6. Note that this solution set can also be rewritten in terms of any two of the other variables fij,i<j.

5 Generalized 2-player, m strategy games

With m strategies, we can define pi = fii+1₂Pi<jfij. Hence for a model with m strategies we also

have m equations, for pa, pb, . . . , pm. At first it might seem as if there exists an m + 1th equation in the

form ofP

i≤jfij= 1. However, this holds by definition, since we choose values for pi in such a way that

Pm

i=api = 1. Since the two restrictions are equivalent, Pi≤jfij = 1 ≡Pi≤jfij = 1, there exists no

extra equation.

5.1 Testing for Linear Dependence

We now want to test for linear dependence in the system of equations. That is, if one of the vectors is some combination of the others. This would reduce the total number of equations.

For n = 2, m = 4 we have the following set of equations:

pa= faa + 1₂fab + 1₂fac + 1₂fad with fab+ fac+ fad= ca and ca= 2(pa− faa)

pb= fbb+ 1₂fab + 1₂fbc + 1₂fbd with fab+ fbc+ fbd= cb and cb= 2(pb− fbb)

pc= fcc+ 1₂fac+ 1₂fbc+ 1₂fcdwith fac+ fbc+ fcd= cc and cc= 2(pc− fcc)

pd= fdd + 1₂fad + 1₂fbd + 1₂fcdwith fad+ fbd+ fcd= cc and cd= 2(pc− fcc)

Using vector notation, the above translates into the following:         fab fac fad fbc fbd fcd         ×     1 1 1 0 0 0 1 0 0 1 1 0 0 1 0 1 0 1 0 0 1 0 1 1     =     ca cb cc cd    

Definition: A finite set S = ~va, ~vb, ..., ~vm of vectors in Rn is said to be linearly dependent if there

exist scalars (real numbers) λa, λb, . . . , λm, not all of which are 0, such that λa~va+ λb~vb+ . . . + λm~vm= ~0

(9)

So if λa         1 1 1 0 0 0         + λb         1 0 0 1 1 0         + λc         0 1 0 1 0 1         + λd         0 0 1 0 1 1         =         0 0 0 0 0 0        

can only be satisfied by λa= λb= λc = λd= 0,

then there exists no linear dependence. It follows from the above set of vectors S that λa + λb = 0,

λa+ λc = 0, λa + λd = 0, λb+ λc = 0, λb+ λd = 0, λc+ λd = 0. Hence λa = −λb = −λc = −λd ,

λb = λc. Therefore −λb = −λc, λb = −λc⇒ λb = −λb ⇒ λb = 0 ⇒ λa = λb = λc = λd = 0. Thus, the

only solution of this system of equations is the trivial solution. We can therefore conclude that there is no linear dependence and infer that there are 4 equations when m = 4.

Now we wish to generalize this result to m strategies. We rewrite the above system withPm

l=aλl~vl= ~0.

Here λl is the scalar that is multiplied by ~vl, the vector that represents all heterogeneous groups that

strategy l is part of. We define l = (a, b, ..., m). Generalized to m strategies and splitting up fij into fij

and fjifor the purpose of recognizing a pattern, we get a transformation of ~va.

Previously, for a 4 strategy game, we defined the vector for ~va, consisting of fab, facand fadas follows:

~ va=         1 1 1 0 0 0         is now written as ~va     0 1 1 1 1 0 0 0 1 0 0 0 1 0 0 0    

for the purpose of finding a pattern.

Note that (va)ij = 1 if i = a or j = a with i 6= j. In words, all heterogeneous group that strategy A

is part of. The same vector ~va, in case of m strategies is then written as

~ va=        f a b c . . . m a 0 1 1 . . . 1 b 1 0 0 . . . 0 c 1 0 0 . . . 0 .. . ... ... ... . .. ... m 1 0 0 . . . 0       

In order for the vectors to be linearly independent, only the trivial solution where λa, ..., λm= 0 must

satisfy λa         0 1 1 . . . 1 1 0 0 . . . 0 1 0 0 . . . 0 .. . ... ... . .. ... 1 0 0 . . . 0         +λb         0 1 0 . . . 0 1 0 1 . . . 1 0 1 0 . . . 0 .. . ... ... . .. ... 0 1 0 . . . 0         +....+λm         0 . . . 0 0 1 .. . . .. ... ... ... 0 . . . 0 0 1 0 . . . 0 0 1 1 . . . 1 1 0         =           0 . . . 0 .. . . .. ... .. . . .. ... .. . . .. ... 0 . . . 0          

Here we will show that only the trivial solution, where all scalars are equal to zero, satisfies the system of equations. Suppose Pm

l=aλl~vl = ~0. P m

l=aλl(vl)ij = 0 ∀ i, j with i 6= j. We can see that

(vl)ij 6= 0 ⇐⇒ (vl)ij = 1 ⇐⇒ i = l or j = l with i 6= j. This implies that P_l=am λl(vl)ij = λi+ λj.

So ∀ i 6= j follows that λi = −λj. So λa = −λb = . . . = −λm and λb = −λc. This means that

−λb = −λc = λb ⇒ 2λb = 0 ⇒ λb = 0 ⇒ λa = . . . = λm= 0. We can thus conclude that there are m

different equations, and that we cannot eliminate any one of them without altering the solution set. The total number of variables are all groups fij,i≤j. Counting the number of groups for m strategies

gives usP fij,i≤j = Pm i=0(m − i) = Pm i=0i = m(m+1)

2 . However, there are m homogeneous groups fiithat

(10)

we have m(m+1)₂ − m = m(m−1)₂ variables left. For m ≥ 4 we find that m(m−1)₂ > m. There are fewer equations than unknowns and we can conclude that the solution for fij,i<j is not unique. The solution

set has dimension m(m−1)₂ − m =m(m−3)₂ .

6 Examples of Different Ways of Group Formation

Now that we have found that there are infinitely many solutions for 2-player games with more than 3 strategies, the next step is to find some that make sense when looking at the way these group frequencies are generated. In this chapter, we will look at 2-player, 4 strategy games in more detail.

Figure 4: The strategies and their colors used in the upcoming examples of population assortment One solution that is always part of the solution set is the extended population structure model solu-tion. The same solution that is unique in a setting with n = 2, m < 4. In a sense we can say that a type i player does not distinguish, apart from on pj, between the different j strategies in the matching process.

In the following examples we will assume p = (1₄,1₄,₄1,1₄) and ˆr = 1₃. As we shall see, there are different ways to assort populations. Furthermore, we know that fij,i≤j ⇒ p, r. In words, if we know

population state f , we can also determine the (unique) p and r that belong to this population state. But the reverse cannot be said. Given p and r, there are infinitely many ways to assort the population. As we have seen in the previous chapter, the solution set for fij,i≤j has dimension m(m−3)₂ .

We have proven that a unique solution exists in terms of homogeneous groups with n = 2 and m strategies, so the following holds for all examples: fii = rpi+ (1 − r)p2i =

1 3 · 1 4 + (1 − 1 3) 1 4 = 1 8. By

symmetry, the standard solution gives fij, i < j = 2(1 − r)pipj = ₁₂1. Checking the restriction gives

P

(11)

Figure 5: The groups and their sizes according to standard non-discriminative matching. faa, fbb, fcc, fdd= 1₈. All heterogeneous groups fij,i<j have frequency ₁₂1.

Another possibility to structure the population with r = 1₃ and pi= 1₃ ∀ i, is when the strategies are

ordered in a cycle. Groups of kin or same-strategy players live together and only interact with neighbor-ing same-strategy groups. In this case, A and C do not interact with each other, but only match locally. B and D do also not interact with each other. Alternatively, this type of heterogeneous group formation can also be seen as discriminative matching. Here, groups of same-strategy players are more likely to match with certain specific other strategy players. Thus, in this example, fab = fbc = fcd = fad = 1₈,

and fac= fbd = 0. This is a consistent population state sincePi≤jfij= 1. See figure 6.

Figure 6: The cycle of strategies, where each strategy only interact with its neighbors. Note that strategy A and C, and strategy B and D do not interact with each other.

Another, very basic solution, is one where strategy A and B players live close to each other and are separated from strategy C and D. Again, we work with r = 1₃ and pi = 1₄ ∀ i. Then fii = 1₈ ∀ i and

fab= fcd= 1₄. Intuitively, this can be seen two separated islands with (forced) local matching.

Alter-natively it can be explained as an extreme form of a discriminative matching process, where A and B discriminate C and D and vice versa. See figure 7.

(12)

Figure 7: Two islands, strategy A and B are separated from strategy C and D.

One can also think of solutions where some connections of groups of same strategy players are stronger than others. In this example, fab = fcd > fbc = fad > fac = fbd = 0. In this case, we set

fab= fcd=₂₄5, fbc= fad= ₂₄1 is an example that suffices all restrictions. See figure 8.

Figure 8: Matching is local, but there is a higher preference for matching with one neighboring strategy than with the other.

Something else that becomes important with this type of discriminative matching is the order of the strategies. A can favor being matched to B, but also to C. So even though r and p are the same, and a similar local matching process is present, the solution can still differ. See figure 9.

(13)

Figure 9: Order matters. The discriminative matching process can be a result of location and the order in which the strategies are arranged. In this example, strategy B and C switched positions. As we shall see in an application of the RSP-5 game, this can have a significant impact on the replicator dynamics.

6.1 A Function for Alternative Heterogeneous Group Formation

One might have noticed that we use specific examples of population structure, and did not present a general equation for how these alternative solutions can be created. The standard solution gives us a way to form heterogeneous groups fij,i<j as a function of r and p, for any m strategies. More specifically,

fij,i<j(r, p) = 2(1 − r)pipj. For the other, non-standard solutions it is not possible to write fij,i<j solely

as a function of r, pi and pj. One reason for this is that the solution of fij,i<j depends on whether j

is named A, B or something else. However, we cannot exclude the possibility that there is a way to write fij,i<j in terms of r, p and a third hypothetical parameter l. This parameter l, standing for local

matching, has to indicate which strategy i matches with which strategy j for all i, j. Then, a general equation can be found for fij,i<j(r, p, l). This is certainly something worth finding, but goes beyond the

scope of this paper.3

7 An Additional Restriction to Guarantee a Unique Solution

It can be useful to have an additional restriction that does cause the model to have a unique solution. An important feature of the population structure model, where all fij = 2(1 − r)pipj with i < j, is that in

the matching process all strategies i do not distinguish between the different strategies j = (A, B, ..., m). In order words, they do not have a certain preference to be matched with a particular strategy. Also, no strategy is excluded or discriminated against. This also means that the specific name tag is irrelevant for the heterogeneous group formation.

We know that in the case of n=2, all players are either matched with a same-strategy player that also plays i, or another player that plays any strategy j. In order words, there are homogeneous groups fii

and heterogeneous groups fij,i<j. For general m, we found a unique solution in terms of fii groups. We

also know that pa= faa + 1₂fab + 1₂fac + 1₂fad with fab+ fac+ fad= ca and ca= 2(pa− faa).

By looking at an important property of the standard solution, where all strategies interact in a non-discriminative way with each other, we can turn this argument around and find a restriction that does not allow for discriminative matching. By definition, pi= fii + 1₂ P_i<jfij ⇒ pi - fii = 1₂ P_i<jfij.

On the right hand side we now have all strategy i players that will be part of an heterogeneous group. If these heterogeneous players are ’neutral’ or ’unbiased’ about the strategy of their co-player, then the heterogeneous i players should be matched with strategy j players according to their frequencies pj.

We denote the sum of all heterogeneous groups that A is part of by P

a<jfaj. In an example with 4

strategies, A, B, C and D, the neutral group formation should then be as follows: 2(pa− faa) = pb pb+ pc+ pd X a<j faj+ pc pb+ pc+ pd X a<j faj+ pc pb+ pc+ pd X a<j faj = fab+ fac+ fad= X a<j faj

3_{Alternatively this parameter could be called d, standing for discriminative matching. Any other parameter specifying}

(14)

We can infer that fab = _1−ppb

a

P

a<jfaj. From the above equation, we can also infer that fab fac = pb 1−pa P a<jfaj pc 1−pa P a<jfaj = pb

pa will also hold under neutral group formation. For general m this translates into

the following condition: fij

fik =

pj

pk ∀ j, k and i 6= j, k.

We can easily see that the population structure model satisfies the additional restriction. All hetero-geneous groups are formed in the following way: fij = 2(1 − r)pipj with i < j . If we take the ratio of

two arbitrary heterogeneous groups fij and fik we find that fij fik = 2(1−r)pipj 2(1−r)pipk = pj pk ∀ j, k and i 6= j, k.

8 Application: RSP-5 with Relatedness

In this chapter, we will analyze a specific game and investigate what influence general population struc-ture can have on the replicator dynamics. Furthermore, we will sketch a situation where a different population structure can in fact change the replicator dynamics.

A well-known example of a 2-player game with more than two strategies is the Rock, Scissors, Paper game. Here Rock beats scissors, Scissors cuts paper and paper beats rock. Since we are particularly interested in different solutions for group formation and n=2, m=3 games have a unique solution, we extend the game to an RSP-5 game with 5 strategies. Maintaining the same notation as throughout this paper, the strategies are A, B, C, D and E. Here, strategy A beats strategy B and C, strategy B beats C and D etcetera. This gives us the following game in matrix form:

A =       1 2 + a 2 + a 0 0 0 1 2 + a 2 + a 0 0 0 1 2 + a 2 + a 2 + a 0 0 1 2 + a 2 + a 2 + a 0 0 1      

Figure 10: Who beats who in the RSP-5 game? An orange line pointing from i to j indicates that strategy i beats strategy j when they meet. In this example, strategy A beats strategy B and C, strategy B beats strategy C and D, etc.

8.1 The Replicator Dynamics with Random Matching

For any a the game has a unique interior Nash equilibrium strategy, p∗= (1₅,1₅,1₅,1₅,1₅), the reader is re-ferred to the appendix for an extensive analysis of the Nash Equilibria in this game. It will be shown that the product papbpcpdpeincreases (decreases, is constant) along any interior solution path of the replicator

dynamics if a is positive (negative, zero).4 We first find the equations for the replicator dynamics.

(15)

The replicator dynamics for the population share piis given by ˙pi = [¯πi,p− ¯πp]pi. Here ¯πi,prepresents

the expected payoff of pure strategy i when the population is in state p ∈ 4. The population average payoff in state p is given by ¯πp. Intuitively, it makes sense that the population share pi can only grow if

it earns a higher payoff than the average population. ˙ pa= [pa+ (2 + a)(pb+ pc) − ¯π]pa ˙ pb= [pb+ (2 + a)(pc+ pd) − ¯π]pb ˙ pc= [pc+ (2 + a)(pd+ pe) − ¯π]pc ˙ pd= [pd+ (2 + a)(pe+ pa) − ¯π]pd ˙ pe= [pe+ (2 + a)(pa+ pb) − ¯π]pe

Hence the time derivative of h(p) = log(papbpcpdpe) is ˙h(p) = _pp˙a_a +p_p˙b_b +p_p˙c_c +p_p˙d_d +p_p˙e_e = (pa+ pb+

pc+ pd+ pe) + (2 + a)2(pa+ pb+ pc+ pd+ pe) − 5¯π = 5 + 2a − 5¯π.

With ¯π = pa(pa+ (2 + a)(pb+ pc)) + pb(pb+ (2 + a)(pc+ pd)) + pc(pc+ (2 + a)(pd+ pe)) + pd(pd+

(2 + a)(pe+ pa)) + pe(pe+ (2 + a)(pa+ pb)) = p2a+ p2b+ p2c+ p2d+ p2e+ (2 + a)(papb+ papc+ papd+ pape+

pbpc+ pbpd+ pbpe+ pcpd+ pcpe+ pdpe). By the identity 1 = (pa+ pb+ pc+ pd+ pe)2= kpk 2 + 2(papb+ papc+ papd+ pape+ pbpc+ pbpd+ pbpe+ pcpd+ pcpe+ pdpe) where kpk 2 = p2 a + p2b+ p 2 c+ p2d+ p 2

e, the average payoff can be written as

¯

π = 1 + a(papb+ papc+ papd+ pape+ pbpc+ pbpd+ pbpe+ pcpd+ pcpe+ pdpe) = 1 + a₂(1 −kpk2)

Then ˙h(p) = 5 + 2a − 5(1 +a₂(1 −kpk2)) = a₂(5kpk2− 1). Note that kpk2 achieves its maximum at the vertices of the unit simplex, where it has value 1. kpk2 achieves a minimum at p∗ where it has value

1

5. We can see that the term 5kpk 2

− 1 is zero at the Nash equilibrium p∗ _{and positive anywhere else on}

the simplex. Thus when a = 0, in the original RSP-game, all solution paths are cycles on the simplex. When a > 0, the product papbpcpdpe will grow over time until the equilibrium of p∗ = (15,

1 5, 1 5, 1 5, 1 5) is

reached. In this case, p∗ is an ESS. As a gets larger, the forces towards the center become stronger and this speeds up selection. In case of a < 0, all dynamic solution paths move outward until one of the pure strategies remains in the population. Hence, p∗ is not an NSS and certainly not an ESS.

8.2 The Replicator Dynamics with Population Assortment

We will now add relatedness to the RSP-5 game. First, the payoffs of strategy A players will be calculated as an example, which can then easily be transformed into the payoffs of the other strategies B, C, D, E because of symmetry. Then also the average payoff ¯π can be calculated.

So using the standard population structure model, the group formation will be as follows: faa= rpa+ (1 − r)p2a with πa,a= 1

fab= 2(1 − r)papb with πa,b= 2 + a

fac= 2(1 − r)papc with πa,c= 2 + a

fad= 2(1 − r)papd with πa,d= 0

fae= 2(1 − r)papewith πa,e= 0

We can now derive the average payoff of strategy A players in the population:: ¯

πa =

2faaπa,a+1(·fabπa,b+·facπa,c+·fadπa,d+·faeπa,e)

2pa

For this particular game, the average payoff for A-players is then equal to ¯

πa =

2·1·((1−r)p2_a+rpa)+1·(2+a)(2(1−r)(pb+pc)

2pa . Simplifying gives us ¯πa= (1−r)pa+r+(2+a)(1−r)(pb+pc)

(16)

The population average payoff ¯π = pa¯πa+ pbπ¯b+ pcπ¯c+ pdπ¯d+ peπ¯e+ = pa((1 − r)pa+ r + (2 + a)(1 −

r)(pb+ pc)) + pb((1 − r)pb+ r + (2 + a)(1 − r)(pc+ pd)) + pc((1 − r)pc+ r + (2 + a)(1 − r)(pd+ pe)) +

pd((1 − r)pd+ r + (2 + a)(1 − r)(pa+ pe)) + pe((1 − r)pe+ r + (2 + a)(1 − r)(pa+ pb)).

Collecting terms gives ¯π = (1 − r)((p2_a+ p2_b + p_c2+ p2_d + p2_e) + 2(papb + papc+ papd+ pape+ pbpc +

pbpd+ pbpe+ pcpd+ pcpe+ pdpe)) + r(pa+ pb+ pc+ pd+ pe) + a(papb+ papc+ papd+ pape+ pbpc+

pbpd+ pbpe+ pcpd+ pcpe+ pdpe).

Using the identity 1 = (pa+ pb+ pc+ pd+ pe)2= kpk 2

+ 2(papb+ papc+ papd+ pape+ pbpc+ pbpd+ pbpe+

pcpd+ pcpe+ pdpe)) with kpk2= p2a+ p2b+ p2c+ p2d+ p2ewe can write ¯π = r + (1 − r) + a(1 − r)( 1 2(1 −kpk 2 )) = 1 + a₂(1 − r)(1 −kpk2). Then ˙h(p) = p˙a pa+ ˙ pb pb+ ˙ pc pc+ ˙ pd pd+ ˙ pe pe = 5r + (1 − r)(pa+ pb+ pc+ pd+ pe) + (2 + a)(1 − r)2(pa+ pb+

pc+ pd+ pe) − 5¯π. Rewriting gives ˙h(p) = 5 + 2a(1 − r) − 5¯π. We now fill in the expression derived for ¯π

to get ˙h(p) = a₂(1 − r)(5kpk2− 1). Based on this expression we can conclude that adding relatedness does not change the direction, but just the speed of selection in the RSP-5 game. As relatedness increases, selection is slowed down. When we consider the case where r = 0, we find that there is no change in the population composition. Intuitively, this makes sense, since when relatedness is equal to 1, all players only meet copies of themselves. Everyone thus receives a payoff of 1 and no strategy can possibly have a fitness advantage (¯π = ¯πi= 1 ∀ i).

8.3 Alternative Group Formation

Until now, we have only considered the general population structure model with non-discriminative matching. As we have seen, there are other ways to form groups given p and r with 5 strategy games. This can especially be of interest in the RSP game where the cycle of strategies beating each other keep the game in balance. As we explained in Chapter 6, we did not find a formal way to describe the alterna-tive solutions for fij,i<j(p, r). This also makes it impractical to formally analyze the replicator dynamics,

as we did not specify the exact formation process of the heterogeneous groups. At maximum, we can directly provide an alternative solution and illustrate what would happen in the replicator dynamics.

In this example we will use the only interior Nash Equilibrium as a starting point. Then we will describe what the replicator dynamics might look like. In the Nash Equilibrium, pi = 1₅ ∀ i. We take

relatedness r = 1₃. Then fii= ₇₅7 ∀ i and fab= fbc= fcd= fde= fae=₇₅8. See figure 11.

Figure 11: An example where each group of same-strategy players only interacts with its neighboring groups. Here, due to the order where each strategy is beat by one of its neighbors and beats the other,

(17)

We can change the order in which the strategies are arranged in such a way that A has a huge advantage, while E will quickly go extinct. As mentioned before, this example will only intuitively be explained as the actual replicator dynamics are very complex and difficult to formalize. As E has gone extinct, D will now be overrun by strategy B and C. Then only strategy A,B and C are left, where both strategy A and B beat C. Hence C is the next strategy to go extinct after which A beats B, and assuming no mutations during the process, A has taken over the entire population. See figure 12.

Figure 12: An example where each group of same-strategy players only interacts with its neighboring groups. Over time, it is predicted that strategy A will go to fixation.

(18)

9 Conclusion

This paper started by giving an example of different ways to structure populations in 3-player, 2 strategy games. After this, we found that 2-player, m strategy games have infinitely many ways to structure populations for m ≥ 4. We then gave the solution set for 4-strategy games and discussed some intuitive ways to assort populations with the same relatedness and strategy-frequencies. An example of this is the cycle of groups of same-strategy players only interacting with their neighbors.

We also found an additional restriction that does make the model have a unique solution for any m strategies. Here, the intuition is that strategies interact with each other in a non-discriminative way. Lastly, we show that relatedness generally slows down the speed selection in the RSP-5 game, and that different types of group formation can possibly change the replicator dynamics.

Further research may focus on finding a generalization to a model with n players and m strategies. Also, a similar model with a parameter l for local matching, or a parameter d indicating discriminative matching can be useful in understanding the role of different types of population structure in the replicator dynamics.

10 Bibliography

Dawkins, R. (1989). The selfish gene. Oxford university press.

Lieberman, E., Hauert, C., Nowak, M. A. (2005). Evolutionary dynamics on graphs. Nature, 433(7023), 312.

Nowak, M. A. (2006). Evolutionary dynamics. Harvard University Press.

Ohtsuki, H., Nowak, M. A. (2006). The replicator equation on graphs. Journal of theoretical biology, 243(1), 86-97.

Strang, G. (1993). Introduction to linear algebra (Vol. 3). Wellesley, MA: Wellesley-Cambridge Press. Van Veelen, M. (2009). Group selection, kin selection, altruism and cooperation: when inclusive fitness is right and when it can be wrong. Journal of Theoretical Biology, 259(3), 589-600.

Van Veelen, M. (2011). The replicator dynamics with n players and population structure. Journal of theoretical biology, 276(1), 78-85.

Weibull, J. W. (1997). Evolutionary game theory. MIT press.

Wilson, D. S., and Wilson, E. O. (2007). Rethinking the theoretical foundation of sociobiology. The Quarterly review of biology, 82(4), 327-348.

Wynne-Edwards, V. C. (1962). Animal dispersion: in relation to social behaviour (No. QL752 W9 1962a).

11 Appendix

11.1 Nash Equilibria in the RSP-5 game

A =       1 2 + a 2 + a 0 0 0 1 2 + a 2 + a 0 0 0 1 2 + a 2 + a 2 + a 0 0 1 2 + a 2 + a 2 + a 0 0 1      

(19)

Pure strategies: Take x = e1_{, then u(x, x) = 1.} _{Take y = e}4_{, then u(y, x) = 2 + a.} _Hence

u(x, x) ≥ u(y, x) if a ≤ −1. Only then is e1 _{a Nash Equilibrium. Because of symmetry, the same}

statement holds for e2_{, e}3_{, e}4 _{and e}5_.

Mixes of 2 strategies: It suffices to check only on a mix between e1 _{and e}2_{, again because of}

sym-metry. So we take x = [α, 1 − α, 0, 0, 0]. There is no way for this to be a Nash equilibrium other than the case where α = 1, because strategy 1 dominates strategy 2 in this case. The same reasoning holds for any mix between two strategies i and j.

Mixes of 3 strategies: The only candidates are cycles that reduce the game to an RSP-3 game. These include mixes between x1, x3, x5. The only way to make u(e1, x) = u(e3, x) = u(e5, x) is when x =

[1 3, 0, 1 3, 0, 1 3]. Then u(x, x) = 1 + 1

3a. This is only a Nash Equilibrium if u(x, x) ≥ u(y, x) ∀ y. However,

u(e4_{, x) =} 4 3+

2

3a > u(x, x) if a > − 1

3. So only for values of a ≤ − 1

3 there exists a Nash equilibrium with

3 (balanced) strategies.

Mixes of 4 strategies: Here one example is enough due to symmetry. Take x = [x1, x2, x3, x4, 0] where

x1, x2, x3, x4 are all nonzero. Then x(e2, x) = x2+ 2x3+ 2x4 > x(e3, x) = x3+ 2x4. Hence strategy 3

will never be played, and we can conclude that no 4-strategy mix can be a Nash Equilibrium.

Mixes of 5 strategies: x = [1₅,1₅,1₅,1₅,1₅] is a Nash Equilibrium because u(x, x) = 1 + 2₅a = u(y, x) ∀ y. Now suppose y = e1. Then u(y, y) = 1 > u(x, y) = 1 +2₅a for values of a < 0. Then x∗ is not an ESS, nor NSS. When a = 0, u(x, y) = u(y, y) for all y. Thus, when a = 0, x∗ is an NSS but not an ESS. Only for values of a > 0, x∗ is both an NSS and ESS.

Population structure in a 2-player, m strategy game

University of Amsterdam

Master’s Thesis Behavioral Economics and Game Theory