MSc in Economics – Specialisation Behavioural Economics and Game Theory 15 ECTS
University of Amsterdam
Breaking the Chains
in the Prisoner’s
Dilemma
Results from numerical simulations on the cycleStudent: Andreas Deligiannis Student No.: 11400188
Supervisor: Prof. Dr. C.M. (Matthijs) van Veelen
Statement of Originality
This document is written by Student Andreas Deligiannis who declares to take full responsibility for the contents of this document. I declare that the text and the work presented in this document are original and that no sources other than those mentioned in the text and its references have been used in creating it.
The Faculty of Economics and Business is responsible solely for the supervision of completion of the work, not for the contents.
Abstract
The evolution of cooperation has been extensively studied under several conditions using Evolutionary Graph theory. Here we attempt to “break the chains” of cooperators and defectors on the cycle, differentiating ourselves from the traditional setting that only allows two separate chains of strategies, one consisting exclusively of cooperators and one exclusively of defectors. This is achieved by specifying the degree of the replacement graph in such a way that a cooperator can be located inside the defectors’ chain and vice-versa. We code and run numerical simulations and obtain the fixation probabilities of cooperators in the Prisoner’s Dilemma for two update procedures, Birth-Death and Death-Birth, and two fitness functions, linear and exponential. For Birth-Death, our results suggest that, contrary to previous research, selection may favor cooperation under specific network and payoff parameters. On the other hand, for Death-Birth, whether cooperation is favored depends on the overlap of the interaction and replacement graphs.
1. Introduction
Game Theory is being extensively used in economics to interpret strategic and economic decisions. Evolutionary Game Theory, EGT, (Trivers, 1971; Maynard Smith and Price, 1973) is the extension of Game Theory to biology. Rather than considering interactions between fully rational players, which is the focal point of Game Theory, EGT studies the dynamics of a population of individuals whose fitness depends on the frequencies of the strategies used. This is also known as natural selection (Maynard Smith, 1982). In the traditional setting, each individual in a well-mixed, infinitely large population acquires an average payoff that illustrates the individual’s (biological) fitness. Therefore, the frequency of each strategy evolves according to the payoffs achieved by the individuals playing it (Taylor and Jonker, 1978; Hofbauer et al., 1979).
The importance of expanding EGT in economics is straightforward. Neoclassical economics and Game Theory assume fully rational agents. Behavioral and experimental economics have shown that such an assumption does not always sufficiently capture reality and cannot explain real-life behaviors and strategies. As Kandori (1996) pointed out, the assumption of perfectly rational agents is unable to interpret why agents choose the Nash Equilibrium (when they do). Therefore, evolutionary games can be useful in studying the findings of Game Theory. However, Friedman (1991; 1998) and Selten (1991) suggested that, rather than using the biological genetic mechanisms, economists should consider imitation and learning
processes that seem to be more appropriate in competitive markets. The application of EGT to industrial organization (entry/exit, unionization etc.), international economics and any other situation in which there are strategical interactions between agents could assist the prediction of the behaviors that may emerge, as well as their effects (Friedman 1991; Friedman 1998).
Lieberman et al. (2005) introduced Evolutionary Graph Theory, which studies evolutionary dynamics using population structures and social networks. In this setting, graph vertices represent individuals of the population and graph edges determine which players interact (interaction graph) and where individuals can place their offspring (replacement graph). An important result of the aforementioned paper is the calculation of the fixation probability of a randomly placed mutant in various graphs. In a population of individuals with only two available strategies (A, B), the fixation probability is the probability that a sole A (B) player in a world of B (A) players takes over the entire population, thus reaching the state of A (B) players. If the fixation probability of a strategy is greater (smaller) than , then selection favors (opposes) that strategy. If the fixation probability is equal to , then selection neither favors nor opposes that strategy and the mutant is neutral. Ohtsuki et al. (2006) studied the conditions under which cooperation will be favored over defection using evolutionary dynamics on regular graphs. They utilized three different update procedures (Birth-Death, Death-Birth and Imitation) and showed that, under the former updating method, cooperation is never favored, whereas under the latter two procedures cooperation is favored if and only if:
(1.1)
Here, and are the benefit and the cost of cooperation, respectively and is the average connectivity of the graph, i.e. the average number of neighbors each individual has. The first two update rules are studied in the current thesis, and will, therefore, be explained extensively in the following sections. As for the Imitation update procedure, it refers to the case in which an individual is randomly chosen to update her strategy in the following way. Proportional to the fitness of herself and her neighbors, she will either adopt the strategy of one of them or keep her own strategy. Ohtsuki and Nowak (2006a) had previously studied the condition (1.1) for specific cases of graphs, obtaining for Death-Birth updating and for Imitation.
What all the above-mentioned studies have in common is the fact that interactions and (biological) reproduction overlap for every individual in the population, meaning that the neighbors an individual interacts with are exactly those that she can replace with her offspring. The, seemingly, first study that attempts to break this overlap between interaction and replacement is by Ohtsuki et al. (2007). In this analysis, they obtain theoretical predictions for the evolution of cooperation in cases where the interaction and replacement graphs are regular and not identical. They find that cooperators can be favored over defectors only under Death-Birth and Imitation updating. For the former update procedure, they obtain that cooperation can be favored over defection if:
(1.2)
Here, and represent the average connectivity of the interaction and the replacement graphs, respectively, while is the average degree of the ovrlap of these two graphs. Ohtsuki and Nowak (2006b) also studied the replicator dynamics for the Prisoner’s Dilemma using regular graphs.
Furthermore, all these studies share the same (linear) fitness function and (low) intensity of selection, which seems to affect the obtained results. To the contrary, van Veelen and Nowak (2012) studied multi-player games using both linear and exponential fitness functions as well as different levels of intensity of selection. The conditions for the evolution of cooperation change depending on the fitness derivation method and the selection of the intensity level. However, it still remains unclear if and how the evolutionary dynamics and subsequently, the fixation probabilities change in cases where neighbors further away can affect the result of the update process on the cycle. In Prisoner’s Dilemma terminology, this is an interesting question to answer, since such a procedure could “brake the chains” of cooperators and defectors within a certain population, allowing for a more realistic analysis of the dynamics in such cases. The replacement graph will, therefore, allow each individual to take into account the payoffs of players further away. This will affect the strategy that this individual chooses and thus, the frequencies of strategies and the fixation probabilities of the whole population. The fact that cooperators and defectors need not be sequentially ordered in the population allows for more complex interactions, which, in turn, affects the dynamics of the population.
Based in the aforementioned studies, we expected that this model will not affect the dynamics under the Birth-Death update rule, since these studies indicate that selection never favors cooperation. On the other hand, under the Death-Birth update rule, the dynamics should be affected by the model in question. Defection in the population should benefit from the fact that –in the examined cases– defectors will probably meet cooperators more frequently than in the standard setting. However, the distribution of the two strategies in the population along with the exact update process used could affect the final outcome and favor either strategy.
Section 2 of the paper is a review of the theoretical prerequisites of the Prisoner’s Dilemma, and presents how these are included in our model. In section 3, we present our results. Section 4 contains our conclusions and suggests some ideas for further research. At the end of the paper, the Appendix details our theoretical predictions and results.
2. Methodology
In this section, we present the theoretical background that is required for the reader to follow our analysis. Furthermore, we present the modeling method used in order to obtain our results.
The general model examines a game with two pure strategies, and . The payoff matrix for this general case is given by:
(2.1)
This is the payoff matrix we used in order to obtain the theoretical predictions in Appendix A1.
2.1 The Prisoner’s Dilemma in Evolutionary Games
Let us now consider the Prisoner’s Dilemma. The payoff matrix of the game for the row player is given by:
(2.2)
A cooperator (playing strategy ) is someone who pays a cost for the other individual to receive a benefit . Defectors (playing strategy ) on the other hand, pay no cost and share no benefit. We assume and do not consider mixed strategies throughout this paper.
In the present analysis, the individuals of a population of size N are represented by the vertices of two graphs. The interaction graph, , which determines who plays the game with whom and the replacement graph, that controls the updating procedure in the sense of specifying who an individual’s offspring can replace. The degrees of these graphs are and , respectively. Considering the cycle, this means that and indicate the number of neighbors that an individual will interact with and might replace on e.g. her right. Therefore, refers to the case in which an individual plays the game with her two adjacent neighbors, one to the right and one to the left. The same applies for .
Each individual receives from each interaction the payoff indicated by the payoff matrix (2.2). Adding these payoffs results to the total payoff, , for every individual. For linear fitness function, the fitness of each individual in the population is given by:
(2.3)
In this case, , represents the intensity of selection. For this fitness function, the maximum value of should be lower ( ) if we allowed negative payoffs (van Veelen and Nowak, 2012). However, this is not the case throughout this paper. If , we have the case of strong selection, whereas, if , we have weak selection. For , we obtain the case of neutral drift. Throughout this paper we used for this fitness function. The reasoning behind this choice is that in real-life situations the number of different games of each player is so large that each individual game only slightly contributes to the overall performance (Ohtsuki et al., 2007).
According to the exponential fitness function, the fitness is given by:
(2.4)
In this case, measures again the intensity of selection. The limit of weak selection is again . Throughout the paper we used for the exponential fitness.
So far, we have explained how individuals interact with each other and the way their fitness is determined. As was mentioned in the Introduction section, the evolutionary dynamics of a population are highly correlated with the update procedure followed. Here, as in many other
papers (Ohtsuki et al, 2006; Ohtsuki and Nowak, 2006a; Ohtsuki et al., 2007), we consider two different update rules:
Birth-Death (BD) update procedure: An individual is chosen to reproduce proportional to fitness. The offspring randomly replaces one of the neighbors indicated by the replacement Graph .
Death-Birth (DB) update procedure: An individual is randomly chosen to die. Her
neighbors compete for the empty site proportional to fitness.
These update rules (together with the Imitation updating) are considered as “fertility selection” (Ohtsuki et al., 2008), since the reproductive success of the individuals is highly dependent on the payoffs. Here, we need to highlight the fact that we do not consider any mutations in our analysis, in the sense that the offspring always adopts the exact strategy of its parent.
2.2 Simulations
In order to make the results easy to compare, we used the following payoff matrix:
(2.5)
In this payoff matrix, represents again the cost of cooperation and the benefit is set to one. Since we do not allow negative payoffs, the so-called “sucker’s payoff” is set to one. Choosing such a payoff matrix has the benefit of reducing the bias of the results by keeping one of the parameters fixed (in this case the benefit of the cooperative act). Therefore, selecting a value for the cost, , gives us directly the cost-to-benefit ( ) ratio. For a rather intuitive explanation of how one can obtain the cost and the benefit of cooperation from the payoff matrix, see Appendix A3. The population size that we used was individuals.
In order to study the evolution of the strategies in our population, we used MATLAB R2017a. The simulation procedure starts by constructing the interaction and replacement graphs, H and G respectively. We determine their connectivity using different values of and . Figure 2.1 shows an example of H and G graphs. Players interact according to and obtain their total payoffs from the games they play. The fitness is calculated using one of the two fitness functions presented above. After that, one of the two update procedures takes place.
For each set of , we used ten different seeds (1-10) so that the position of the initial cooperator and the sequence of numbers produced by the pseudo-random number generator differed from one run to another. For each seed, we ran 10000 simulations. Thus, we obtained our results from a total of 100000 simulations for each , update process and fitness function.
Figure 2.1: Example for interaction and replacement graphs. The
individuals are represented by the vertices of these graphs. The yellow edges correspond to the interaction graph H and the black edges to the replacement graph G.(a) Individuals interact with their 2 adjacent nodes and may replace one of them (h=g=1).(b) Individuals interact with their 4 adjacent nodes, but may replace only their 2 nearest neighbors (h=2, g=1).
All of our simulations start with a single, randomly placed cooperator in a world of defectors. This gives us the opportunity to calculate the fixation probability of cooperation. When one of the two absorbing states is reached (we end up with a population that consists of either cooperators or defectors), our simulations stop. We compute the fixation probability for each set of parameters, as the ratio between the runs that ended up in a population of cooperators and the total number of simulations.
3. Results
Our results suggest an excellent agreement between theoretical predictions and simulations’ results. For the calculation of the theoretical fixation probabilities, in cases where such computations are not very complex, see Appendix A1.
(b) (a)
3.1 Birth-Death update rule
Let us first consider BD updating. As it was mentioned in the Introduction of the paper, we expected that cooperation should never be favored under this update procedure.
For exponential fitness function (table A1 in the Appendix) we find that as the cost to benefit ratio increases, the fixation probability of a single cooperator in a world of defectors decreases, as predicted (see figure 3.1(a)). This holds for any choice of and .
Figure 3.1: Fixation probabilities ( ) of cooperators under BD updating and exponential fitness function.(a) The fixation probability decreases with the cost to benefit ratio.(b) For small values of ratio, results in the lowest fixation probability given . We obtain the largest fixation probabilities when .
This is due to the fact that as cooperation becomes more expensive ( is increasing, keeping constant), the total payoff and thus the fitness of cooperators decreases relative to that of the
0 0,003 0,006 0,009 0,012 0,015 0,018 0,021 0,024 0,027 0 1/100 1/50 1/10 1/8 1/4 1/2 3/4 3/5
ρ
c/bBD,exp,n=50
h=1 g=1 h=1 g=2 h=1 g=3 h=1 g=4 h=2 g=1 h=2 g=2 h=2 g=3 h=2 g=4 h=3 g=1 h=3 g=2 h=3 g=3 h=3 g=4 h=4 g=1 h=4 g=2 h=4 g=3 h=4 g=4 (a) 0 0,005 0,01 0,015 0,02 0,025 0,03 g=1 g=2 g=3 g=4 g=1 g=2 g=3 g=4 g=1 g=2 g=3 g=4 g=1 g=2 g=3 g=4 h=1 h=2 h=3 h=4ρ
BD,exp,n=50
c/b=0 c/b=1/100 (b)defectors. For small values of ratio, when the number of potential replacements is low ( ) we obtain the smallest fixation probability for the given number of interactions (e.g. , ). In other words, when the offspring can only replace one of the two nearest neighbors, the fixation of cooperators is less likely to happen compared with the cases in which one could place her offspring in the site of neighbors even further away. What is more for the two smallest values of , and , we find the largest fixation probabilities when the number of the possible replacements is slightly bigger than the number of the interactions ( ) (figure 3.1(b)). Together with the fact that for and the fixation probability even slightly exceeds when the cost is very small compared to the benefit of cooperation, these results could suggest that there might be a condition under which cooperation will be favored under BD updating, contrary to the prevalent belief.
In the linear case (table A3), we again find that the fixation probability of cooperators is decreasing with the cost to benefit ratio (figure 3.2).
Figure 3.2: Fixation probabilities ( ) of cooperators under BD updating and linear fitness function. The fixation probability decreases with the cost to benefit ratio. Fixation probabilities form clusters according to . Larger values of interactions lead to smaller fixation probabilities, regardless of the choice of .
In general, keeping the number of potential replacements, ,constant, leads to a decrease in the fixation probability of cooperators as the number of interactions increases. This is a key
0,003 0,006 0,009 0,012 0,015 0,018 0,021 0 1/100 1/50 1/10 1/8 1/4 1/2 3/5
ρ
c/bBD,linear,n=50
h=1 g=1 h=1 g=2 h=1 g=3 h=1 g=4 h=2 g=1 h=2 g=2 h=2 g=3 h=2 g=4 h=3 g=1 h=3 g=2 h=3 g=3 h=3 g=4 h=4 g=1 h=4 g=2 h=4 g=3 h=4 g=4factor for the observed formation in clusters according to as the ratio increases (figure 3.2). The smaller the number of interactions is, the larger the fixation probability we obtain.
However, the most interesting finding is once more in this instance that there are some combinations of ( ) for which the fixation probability of cooperators is more than that of a neutral mutant in this specific population. Specifically for and we obtain fixation probabilities that are, even slightly, higher than . Therefore, the above mentioned hypothesis about certain conditions favoring cooperation under BD updating is again introduced.
In order for us to test this hypothesis, we ran more simulations (using seeds 1-60) examining the fixation probability of both cooperators and defectors. Using a total of 600000 runs any possible biases are likely, though not necessarily, to be minimized. This exploration was limited to the case where individuals interact only with their two adjacent neighbors and , since this is the only condition for interactions and cost to benefit ratio under which cooperation seems to be favored. However, we expanded the network of individuals that an offspring may replace with the maximum number of potential replacements checked being the 20 nearest neighbors. For the study of the defectors’ fixation, our code was transformed but was based on the intuition followed in the cooperators’ case. That is, each simulation starts with a single, randomly placed defector in a world of cooperators and stops when one of the two absorbing states is reached. Our results are shown in table 1.
h=1
g=1 g=2 g=3 g=4 g=5 g=6 g=7 g=8 g=9 g=10
cooperators 0,0200667 0,01967 0,0201016 0,01976 0,019758 0,019755 0,01982 0,01967816 0,0197733 0,019908 defectors 0,02031667 0,02048 0,02001667 0,02016 0,020075 0,020245 0,0201283 0,020311667 0,02047667 0,02040667
Table 1: Fixation probabilities of cooperators and defectors for the BD update process,
linear fitness and . The results are obtained for population size from a total of 600000 simulations.
We observe an upward bias in the fixation probability of cooperators in the case where compared with both our theoretical prediction and the results obtained before. Nevertheless, the fixation probability of defectors under this combination of parameters ( ) is almost identical to our expected value ( , obtained using the method described in Appendix A1). Moreover, the rest of the results are in line with previous research suggesting a favoring evolution of defection.
Even though these results seem to be even slightly biased, the hypothesis that selection may favor cooperation under specific assumptions is again confirmed. For our findings indicate that not only the fixation probability of cooperators is higher than that of a neutral mutant, but also that cooperation may be favored over defection. However, the suggestion that the evolutionary dynamics would be similar for other values of seems to be turned down.
Here we provide an intuitive explanation for this result. Keeping the number of interactions constant and changing the number of potential replacements decreases the down-up ratio ( ) of the transition probabilities in state ( see Appendix A1). This is due to the fact that the probability of an individual to be chosen to die decreases, since this “choice” is randomly done between those individuals that are connected with the chosen to give birth and therefore decreases. In combination with the unaffected probability to be chosen for reproduction (therefore is constant), this is beneficial for cooperation. Depending on the value of , I assume that this decrease in the down-up ratio may also hold for other states and a critical value ( ) under which this decrease applies probably exists. However, an increase in leads also to an increase of these ratios in states further away ( ), opposing cooperation and favoring the spreading of defection in this population. Therefore, cooperation can be favored for as long as the increase in the transition probability ratio in states is relatively larger than the corresponding decrease of this ratio in states further away.
3.2 Death-Birth update rule
Regarding the DB update process our results are in line with our intuitive expectations. For exponential fitness function (table A2) the general expected pattern concerning decreasing fixation probabilities to the cost to benefit ratio is again true. Like under BD updating, here we notice that cooperators are more likely to reach fixation whenever they interact with their nearest neighbors keeping the number of potential replacements constant and equal to . In other words, the smaller the is, given that , the greater the fixation probability of cooperators (figure 3.3(b)). Selection in this case favors cooperation under a plethora of different sets of . This result does not hold for . We also need to point out the fact that even though under small ratios and for small values of and selection is very advantageous for cooperation, the fixation probabilities decrease rapidly as interactions and replacement grow, falling even under the crucial value of , which in our case is . A comparison between our results and those obtained by Ohtsuki et al. (2007) is
not very prudent in this case, since their analysis concerned linear fitness. However, we are able to identify that even though equation (1.2) does not hold for exponential fitness, their suggestion that the bigger the intersection between and graphs is the larger the fixation probability for the cooperators, applies also in our analysis. Notice that in our setting this equation should be transformed as
. Especially whenever , we obtain the largest fixation probabilities for the given value of (figure 3.3(a)).
Figure 3.3: Fixation probabilities ( ) of cooperators under DB updating and exponential fitness function. The fixation probability decreases with the cost to benefit ratio.(a) When , we obtain the largest fixation probabilities (the result holds for all cost to benefit ratios). (b) For , the smaller the is, the greater the fixation probability we obtain. Our result could add to this finding that in these cases where the interaction and the replacement graphs are not identical ( ), the fixation probability is larger if the number of potential replacements is smaller than the number of interactions ( ) than
0 0,05 0,1 0,15 0,2 0,25 0,3 0,35 g=1 g=2 g=3 g=4 g=1 g=2 g=3 g=4 g=1 g=2 g=3 g=4 g=1 g=2 g=3 g=4 h=1 h=2 h=3 h=4
ρ
DB,exp,n=50
c/b=0 c/b=1/100 c/b=1/50 (a) 0 0,1 0,2 0,3 0,4 0 1/100 1/50 1/10 1/8 1/4 1/2 3/5 3/4ρ
c/bDB,exp,n=50,g=1
h=1 h=2 h=3 h=4 (b)in the case in which ( ), given that is constant. We also need to point out that the general finding regarding the advantageous selection of cooperation (equation 1.1) does not hold for exponential fitness function. Given that , the fixation probability using should be more than for all cases with . As indicated by table A2, this is not true in our setting.
Figure 3.4: Fixation probabilities ( ) of cooperators under DB updating and linear fitness function. The fixation probability decreases with the cost to benefit ratio.(a) When , we obtain the largest fixation probabilities (the result holds for all cost to benefit ratios, but not for ). (b) For , the smaller the is, the greater the fixation probability we obtain. For the linear case (table A4) our results are to some extent the same as those obtained for exponential fitness function and indicate the fact that selection can favor cooperation under various . However, the fixation probabilities here are significantly smaller than in the exponential case. We observe again both the downward trend of the fixation probability of
0,015 0,0175 0,02 0,0225 0,025 0,0275 g=1 g=2 g=3 g=4 g=1 g=2 g=3 g=4 g=1 g=2 g=3 g=4 g=1 g=2 g=3 g=4 h=1 h=2 h=3 h=4
ρ
DB,linear,n=50
c/b=0 c/b=1/100 c/b=1/50 c/b=1/10 (a) 0,005 0,01 0,015 0,02 0,025 0 1/100 1/50 1/10 1/8 1/4 1/2 3/5 3/4ρ
c/bDB,linear,n=50,g=1
h=1 h=2 h=3 h=4 (b)cooperators as the cost to benefit ratio increases, and the disadvantageous effect that an increase in the number of interactions has on the probability of cooperators to reach fixation, if an individual can replace only her two adjacent neighbors ( , figure 3.4(b)).
Given the fact that our results might suffer from some biases because of the generators used during the simulations, we conclude that equation 1.2 holds in our case, at least for , and . Nevertheless, for , the difference between our results and the predictions obtained by this equation is large enough and cannot be attributed solely to possible noises of our simulations. Despite this discrepancy, we can verify that the majority of our results indicate that whenever the number of interactions equals the number of replacements ( ) the probability of cooperators to reach fixation is the largest for the given number of interactions (figure 3.4(a)). Yet, for we observe a general downward trend as the number of potential replacements increases for various cost to benefit ratios, making unsafe a generalization of this finding.
4. Conclusion
Using numerical simulations, we have studied the evolutionary dynamics in a population in which cooperators and defectors are not necessarily sequentially placed on the cycle. We obtain the fixation probabilities of cooperators for different sets of payoff parameters, interaction and replacement graphs with a large enough population size of under two update procedures (Birth-Death (BD) and Death-Birth (DB)). We have also examined different fitness functions, linear and exponential, which lead to different results. The agreement between the theoretical predictions and the results from our simulations makes us confident about the validity of our simulations. For both update rules, we find that the fixation probability of cooperators is decreasing as the cost of the cooperative act is increasing relative to its benefit.
Contrary to previous research, we find that selection may favor cooperation under BD updating. For both linear and exponential fitness functions, when the cost to benefit ratio is very small, selection favors cooperation when the interactions of each individual are limited to their closest neighbors ( ), while the number of potential sites in which they can place their offspring may be larger. For the linear fitness function, cooperation is favored over defection when individuals interact with their two closest neighbors ( ) while they can replace one of their six adjacent nodes (g ). This result demonstrates the fact that, since
under BD updating birth precedes the death event, the fitness of cooperators is higher relative to that of defectors when the number of interactions is small. When the number of cooperators in the population is small, they frequently meet defectors which results in low payoffs for cooperators and relatively larger for defectors. Therefore, as the number of interactions is increasing the fitness of the few cooperators in the population is small relative to that of the defectors, decreasing the probability of a cooperator to be chosen to reproduce. For DB updating, our results are in line with our hypotheses: increasing the number of interactions leads to a lower fixation probability of cooperators. The exponential fitness function seems to be more advantageous for cooperators than the linear case. Of course, this result depends on the chosen intensity of selection. Our findings have similarities with previous results, indicating that the overlap of the interaction and replacement graphs is crucial for the evolution of cooperation.
Our interesting finding regarding the advantageous selection of cooperation under BD updating, and our results in general, suggest that further research should be conducted on this field. Obtaining theoretical results for generalized choices of interaction and replacement graphs is certainly very difficult to obtain, but not impossible. Moreover, using simulations one could examine different conditions of the population structure. A larger number of individuals and/or different levels of intensity of selection could result in different and maybe, more interesting observations. Furthermore, the examination of whether these findings are in line with economic behavior or not could be tested by experimental data and interpreted accordingly by behavioral economists. Although update procedures like BD or DB might not be suitable for real life interactions, such data would give us the ability to compare reality with theory. Last but not least, since DB updating is similar to the imitation update procedure that is proposed to be more appropriate in human economic interactions, these results could be used as an approximation of reality.
Appendix
A1. Theoretical Predictions
For the theoretical predictions of the fixation probabilities, we will use the notation followed in Nowak (2006). The difficulty mentioned in the main text does not allow for obtaining results analytically. Yet, under certain conditions some predictions are easily made. Imagine a population of size that consists of cooperators and defectors. Let us define as the frequency of cooperators when there are of them in the population (from now on referred to as state ). The probability of moving from state to state or is called transition probability, and respectively. The calculation of these probabilities becomes straightforward if we think of how the update rules affect the population. While obtaining the general values of the transition probabilities we used payoff matrix (2.1). Substituting these payoffs with those shown in payoff matrix (2.5) results to our theoretical predictions.
For BD updating, one individual is chosen for reproduction proportional to fitness and one of its neighbors indicated by the replacement graph is randomly chosen for elimination.
Let us first consider the simplest case, where . The state of the population cannot change if the selected for reproduction individual “is in the middle” of the clusters formed by the two strategies, since in such an event the reproduced offspring will replace an individual of the same strategy. Thus we should only look at the players that are on the boundaries of these clusters. The payoffs of those players are shown in figure A.1. In this case, the ratios of the transition probabilities in each state, using the exponential fitness function and substituting the parameters used in the main text for are:
(A1.1) (A1.2) (A1.3)
Therefore, the fixation probability of cooperation, , is:
Notice that here both the results obtained and the method used applies only for the specific analysis. Depending on the population size, the payoff matrix and the intensity of selection, the down-up ratios change and thus the fixation probability is different.
In the case where and , the calculation of the fixation probability is still simple. However, in this case we should also consider the change in the transition probabilities in state too. For exponential fitness and we obtain:
(A1.4) (A1.5) (A1.6) (A1.7) (A1.8)
So, in this case we have:
(b)
It is straightforward that the calculation of the fixation probabilities using linear fitness function can be done in a similar way. In this case we obtain:
Figure A.1:Payoffs of individuals under BD and DB update procedures using the payoff
matrix 2.1. (a) Payoffs for h=g=1 in state i. For BD updating , we should only consider the payoffs of the players exactly on the boundary (a+b for the cooperator and c+d for the defector). For DB , we should also consider the payoffs of the exact neighbors of those on the boundary. (b) Payoffs for h=2, g=1 in state i. Again, for BD
we should consider the players on the boundary, while for DB we should consider the four players closer to the boundary.
For DB updating, one individual is randomly chosen to die and one of its neighbors indicated by the replacement graph is selected for reproduction proportional to fitness.
The calculation of the fixation probability for this update rule might a little bit trickier than the one for BD. For the case of , we should again focus on the individuals at the boundaries of the formed strategy clusters. However, the fact that now the death event happens first allows the expansion of a strategy to happen when the chosen to die individual is exactly on the boundary. Therefore, we should also take into account the payoffs of players that will compete for the empty spot in such cases (both the players on the boundary and their nearest neighbors). Using again as a starting point the payoff matrix of figure A.1, and substituting the parameters of our model for , we find:
(a)
(A1.9) (A1.10) (A1.11) (A1.12) (A1.13)
So, the fixation probability in this case is:
For the case of and , the intuition behind the computation of the fixation probability is the same. Nonetheless, we should again take into account any changes in the transition probabilities of some states again. The down-up ratios in this case are:
(A1.14) (A1.15) (A1.16)
(A1.17) (A1.18) (A1.19) (A1.20)
And for the fixation probability we have:
For linear fitness we find:
A2. Tables
Here you can find the estimated fixation probabilities for each update procedure and fitness function from the simulations. The results were obtained using MATLAB R2017a and hold for population size individuals. For each set of , we used ten different seeds (1-10). For each seed we ran 10.000 simulations. Thus, we obtained our results from a total of 100.000 simulations for each , update process and fitness function. We compute the fixation probability for each set of parameters, as the ratio between the runs that ended up in a population of cooperators and the total number of simulations.
Fixation prob. h=1 h=2 h=3 h=4 c/b g=1 g=2 g=3 g=4 g=1 g=2 g=3 g=4 g=1 g=2 g=3 g=4 g=1 g=2 g=3 g=4 0 0,00666 0,02631 0,01166 0,01013 0,0024 0,01194 0,01349 0,0089 0,00075 0,00655 0,00681 0,00599 0,00009 0,00253 0,00355 0,00337 1/100 0,00409 0,02223 0,00939 0,00634 0,00064 0,00793 0,00992 0,00539 0,00003 0,00397 0,00494 0,00352 0 0,00106 0,00183 0,00197 1/50 0,00221 0,01715 0,00649 0,00508 0,00009 0,00494 0,00772 0,00368 0,00001 0,0023 0,00276 0,00224 0 0,00044 0,00093 0,00112 1/10 0,00001 0,00079 0,00011 0,00011 0 0 0,00006 0,00003 0 0 0 0,00001 0 0 0 0 1/8 0 0,00018 0,00006 0,00004 0 0 0 0 0 0 0 0 0 0 0 0 1/4 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1/2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3/5 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3/4 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Table A1: Fixation probabilities for the BD update process and exponential fitness. The results are obtained for population size .
Fixation prob. h=1 h=2 h=3 h=4 c/b g=1 g=2 g=3 g=4 g=1 g=2 g=3 g=4 g=1 g=2 g=3 g=4 g=1 g=2 g=3 g=4 0 0,32484 0,1151 0,06711 0,04969 0,11716 0,131 0,05138 0,03118 0,04219 0,0467 0,05739 0,02232 0,01578 0,01926 0,01969 0,02632 1/100 0,31844 0,10937 0,06211 0,04537 0,10943 0,11918 0,04614 0,02617 0,0368 0,03947 0,04782 0,01664 0,01153 0,01416 0,0145 0,01797 1/50 0,31484 0,10563 0,05735 0,04099 0,10205 0,11063 0,04045 0,02184 0,03095 0,03411 0,0402 0,01346 0,00863 0,01082 0,01041 0,01294 1/10 0,2682 0,06863 0,02334 0,01024 0,05249 0,05600 0,01316 0,0042 0,00582 0,00762 0,00719 0,00132 0,00021 0,00052 0,0005 0,00058 1/8 0,25447 0,05756 0,0165 0,00448 0,03916 0,0423 0,00829 0,00187 0,00246 0,00401 0,00387 0,00056 0 0,00026 0,0003 0,00017 1/4 0,17615 0,014 0,00024 0,00002 0,00051 0,00683 0,00003 0 0 0,00001 0,00005 0 0 0 0 0 1/2 0,00712 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3/5 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3/4 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Fixation prob. h=1 h=2 h=3 h=4 c/b g=1 g=2 g=3 g=4 g=1 g=2 g=3 g=4 g=1 g=2 g=3 g=4 g=1 g=2 g=3 g=4 0 0,0198 0,02032 0,02003 0,01963 0,01971 0,01941 0,01977 0,01919 0,01912 0,01841 0,01918 0,01978 0,01904 0,01854 0,0197 0,01993 1/100 0,01975 0,01975 0,01975 0,01944 0,01945 0,0188 0,01913 0,01951 0,01908 0,01873 0,01878 0,0193 0,01931 0,01852 0,01929 0,01894 1/50 0,01965 0,01964 0,01961 0,01924 0,01937 0,01834 0,01941 0,01881 0,01831 0,01865 0,01893 0,0192 0,01864 0,01789 0,01893 0,01896 1/10 0,01905 0,01889 0,01864 0,01894 0,01803 0,01704 0,01858 0,0177 0,01745 0,0172 0,01746 0,01658 0,01572 0,01599 0,01614 0,01727 1/8 0,01849 0,01846 0,01849 0,01975 0,01761 0,0174 0,01758 0,01718 0,01686 0,01631 0,01639 0,01615 0,01451 0,01601 0,01489 0,01517 1/4 0,01774 0,01734 0,01823 0,01789 0,01555 0,01557 0,01588 0,01542 0,0135 0,01416 0,01349 0,01335 0,01222 0,01193 0,01156 0,01242 1/2 0,01566 0,01571 0,01521 0,01484 0,01192 0,01144 0,01179 0,01236 0,00899 0,00896 0,00947 0,00936 0,00706 0,00677 0,00686 0,00703 3/5 0,01465 0,01428 0,01452 0,01453 0,01102 0,01092 0,01097 0,01117 0,00738 0,00779 0,00811 0,00776 0,00567 0,00561 0,0059 0,00543 3/4 0,01366 0,01349 0,01332 0,01331 0,00933 0,00861 0,00908 0,00913 0,0061 0,00613 0,00595 0,00592 0,00388 0,00403 0,00383 0,00399
Table A3: Fixation probabilities for the BD update process and linear fitness. The results are obtained for population size .
Fixation prob. h=1 h=2 h=3 h=4 c/b g=1 g=2 g=3 g=4 g=1 g=2 g=3 g=4 g=1 g=2 g=3 g=4 g=1 g=2 g=3 g=4 0 0,02448 0,02292 0,02079 0,02028 0,02352 0,02412 0,02169 0,02137 0,02377 0,02334 0,02263 0,02197 0,02305 0,02306 0,02195 0,0228 1/100 0,02458 0,02276 0,02067 0,02019 0,02359 0,02388 0,02171 0,0211 0,02343 0,02289 0,0224 0,02161 0,02235 0,02211 0,02187 0,02262 1/50 0,02454 0,02261 0,0206 0,02038 0,02298 0,02373 0,0216 0,02126 0,02272 0,02258 0,02236 0,02128 0,02234 0,02139 0,0218 0,02238 1/10 0,02348 0,02203 0,02 0,01973 0,02183 0,02197 0,02002 0,01978 0,02042 0,02068 0,02031 0,01971 0,01917 0,01968 0,01966 0,01996 1/8 0,02329 0,02169 0,02005 0,01921 0,02137 0,02187 0,01952 0,01906 0,01982 0,0201 0,01986 0,01945 0,01855 0,01888 0,01873 0,01925 1/4 0,02216 0,02048 0,01905 0,01776 0,01898 0,01995 0,01814 0,01688 0,01675 0,01758 0,01766 0,01649 0,01463 0,01529 0,01453 0,01502 1/2 0,01926 0,01795 0,01651 0,0164 0,01511 0,01564 0,01392 0,01327 0,01194 0,01196 0,01177 0,01065 0,00897 0,00916 0,00909 0,00864 3/5 0,01786 0,01759 0,0157 0,01589 0,01331 0,01404 0,01241 0,01195 0,01007 0,00997 0,0099 0,00947 0,00762 0,00746 0,00719 0,00742 3/4 0,01708 0,01596 0,01491 0,01467 0,01173 0,01196 0,01081 0,01022 0,00805 0,00796 0,00781 0,00747 0,00538 0,00545 0,00535 0,00524
A3. Explanation of the cost to befit ratio Here, we will use the payoff matrix (2.5).
Let us suppose that the both players are playing strategy . In this case, both players get a payoff of . If the row player changes her strategy to , while the other player remains with her strategy, she will have to pay a cost for the column player to get a benefit. The payoff of the row player in this case is and for the column player . Therefore, the cost
the row player is facing is The benefit the column player gains is . Thus, in this case the cost is and the benefit, , is equal to .
If initially the row player was a defector and the column player a cooperator, then their payoffs would be and respectively. If the row player changed her strategy, both players would get . Therefore, the benefit the column player gets from the update of the row player’s strategy is and the cost the row player is facing is . Again, the cost of the cooperative act is and its benefit is .
References
Friedman, D. (1991). Evolutionary games in economics. Econometrica: Journal of the Econometric
Society, 637-666.
Friedman, D. (1998). On economic applications of evolutionary game theory. Journal of Evolutionary
Economics, 8(1), 15-43.
Hofbauer, J., Schuster, P., & Sigmund, K. (1979). A note on evolutionary stable strategies and game dynamics. Journal of Theoretical Biology, 81(3), 609-612.
Kandori, M. (1996). Evolutionary game theory in economics. Research Institute for the Japanese Economy. Faculty of Economics. University of Tokyo.
Lieberman, E., Hauert, C., & Nowak, M. A. (2005). Evolutionary dynamics on graphs. Nature, 433(7023), 312-316.
Nowak, M. A. (2006). Evolutionary dynamics. Harvard University Press.
Ohtsuki, H., & Nowak, M. A. (2006a). Evolutionary games on cycles. Proceedings of the Royal
Society of London B: Biological Sciences, 273(1598), 2249-2256.
Ohtsuki, H., & Nowak, M. A. (2006b). The replicator equation on graphs. Journal of theoretical
biology, 243(1), 86-97.
Ohtsuki, H., Hauert, C., Lieberman, E., & Nowak, M. A. (2006). A simple rule for the evolution of cooperation on graphs and social networks. Nature, 441(7092), 502-505.
Ohtsuki, H., Nowak, M. A., & Pacheco, J. M. (2007). Breaking the symmetry between interaction and replacement in evolutionary dynamics on graphs. Physical review letters, 98(10), 108106.
Selten, R. (1991). Evolution, learning, and economic behavior. Games and Economic Behavior, 3(1), 3-24.
Smith, J. M., & Price, G. R. (1973). The logic of animal conflict. Nature, 246(5427), 15-18. Smith, M. (1982). the Theory of Games. Cambridge University.
Taylor, P. D., & Jonker, L. B. (1978). Evolutionary stable strategies and game dynamics. Mathematical biosciences, 40(1-2), 145-156.
Trivers, R. L. (1971). The evolution of reciprocal altruism. The Quarterly review of biology, 46(1), 35-57.
van Veelen, M., & Nowak, M. A. (2012). Multi-player games on the cycle. Journal of theoretical
