University of Amsterdam
Master Thesis
EconometricsEvolutionary Stability in an Experimental
Repeated Minority Game
Author: Thom Schurer (10218017) Supervisor: M.A.L. Koster Second reader: J. Tuinstra Abstract
Minority games stylize various situations such as stock trading and lane switching. Linde et al. (2014) performed a five-player minority game experiment in which students were asked to formulate strategies. In this thesis the outcome of an evolutionary competition between these strategies is investigated. We find that in general the same strategies survive the competition when parameters are varied and that the perished strategies are not able to invade when reintroduced. However, strategies can be found that are able to invade and take over the entire population. Additionally, there are strategies amongst those that perished that can form an equilibrium where the surviving strategies are not able to invade.
Statement of Originality
This document is written by Student Thom Schurer who declares to take full responsibility for the contents of this document.
I declare that the text and the work presented in this document is original and that no sources other than those mentioned in the text and its references have been used in creating it.
The Faculty of Economics and Business is responsible solely for the supervision of completion of the work, not for the contents.
Contents
1 Introduction 2
2 The model 6
3 Experimental design 9
4 Analysis of the strategies 11
5 Evolutionary competition 12 5.1 Robustness . . . 12 5.2 Invasions . . . 16 5.2.1 Reintroduction of strategies . . . 16 5.2.2 Designed invaders . . . 19 5.3 Surviving strategies . . . 21
5.4 Changes to game mechanics . . . 24
5.4.1 Different payoff structures . . . 24
5.4.2 Mimicking real world situations . . . 26
1
Introduction
A common feature in strategic games is that players can work together to ensure they all end up with a desired payoff. But there are also games in which such a thing is not possible. An example of such a game is the minority game. The essence of the minority game is simple; an odd number of players is asked to choose between two options and the players who are in the minority win. This game is played a certain number of times in the repeated version of the game. The players try to maximize their total payoff, i.e. they try to end up in the minority. Whether they actually end up in the minority depends on the option the other players choose. Neither option is inherently better than the other. It is not possible to punish other players when the players receive only information on the size of the groups formed. Since the players are not able to coordinate, it might seem as if there is no better strategy than to choose between both options with equal chances in all periods. But while this strategy is the unique symmetric Nash equilibrium, it is not evolutionary stable in the repeated game. There are strategies which are better in ensuring the minority has the maximal size. A lot of research has been done to investigate the type of strategies that are used and which strategies are the most successful. The minority game seems simple, but many strategies can be formed based on only the win history. The game stylizes various real life scenarios, such as lane changing in traffic and certain types of stock market trading where it is best to be in the minority.
The minority game is based on the famous El Farol game, introduced by Arthur (1994). In the El Farol game, 100 people are given a choice to either go to the El Farol bar or to stay home. The bar is only enjoyable when there are less than 60 people in the bar. If this is not the case, it is preferable to stay home. The bar is open once a week and the problem arises weekly. Arthur argued that everyone is basing their decision on their expectation of how many people will go to the bar. For this reason he argued that expectations were forced to differ between players. This is because if all players believe that few will go to the bar, then all will go to the bar. Similarly if all players believe that many will go the bar, then all will stay home.
The minority game itself was defined by Challet and Zhang (1997) who proposed a stylized version of the El Farol game. In this minority game an odd number of players was given the option to choose between two options A and B. The players in the minority win and each get a payoff equal to one. The game is played for a certain number of periods where in each period the players are asked to choose between the two options. Each player is only told whether they won or not and based on this, players make their decision. Only the data of the last M periods is retained, which means the choice between A and B is based on M values, there are 2M different situations and the total number of strategies is 22M. The number of possible strategies becomes quite large even for small values of M. The players have no knowledge regarding the number of periods the game will be played for.
An important difference between the El Farol game and the minority game is that the minority game is symmetric, i.e. there is no difference between the two options and it is simply the minority that wins. Since its introduction the minority game has been frequently studied by both economists and physicists. Many details vary between the experiments, such as the number of players, the payoff structure and the types of strategies, but the key concept of the game always remains the same. Studies have frequently focused on finding optimal strategies in the
game. Unlike in some other strategic games, there is no best strategy to play in the minority game. If there was all players would use this strategy and none would win. Therefore there are no dominant strategies. There have been studies that use an analytical approach to find a solution to the minority game (see e.g. Marsili et al., 2000). However the main two ways in which researchers attempt to find optimal strategies are computational and experimental research. In experimental research players are often asked to choose between the two options in every period. These decisions give some idea as to which type of strategies the players use. It can be difficult to infer exact strategies, because of the large strategy space and the limited number of information sets that are reached. The main advantage of experimental research is that the behavioral strategies are more likely to be used in practise, since someone actually formed the strategy. Whereas with computer simulations the strategies are picked more subjectively.
Physicists studying the minority game often opt to use computer simulations to find optimal strategies (see e.g. Challet and Zhang, 1997 and Challet and Zhang, 1998). In these computer simulations there is often a large number of agents (commonly between 101 and 1001). Typically, each player gets assigned S strategies which are randomly chosen from all possible strategies. The number S can vary between experiments, but is typically fixed within an experiment. In each period the players use the strategy which would have given them the largest total payoff out of all their assigned strategies in the previous periods. Challet and Zhang found that in general there were an equal number of players in both groups, but when players had a larger memory the size difference between the two groups fluctuated less. There was also a negative correlation between the switching rate and the success rate.
In laboratory experiments there are typically a smaller number of agents (often in the single or low double digits). In these experiments agents are asked to play the minority game for a certain number of periods where they can earn a monetary compensation. In Bottazzi and Devetag (2007) students were divided into groups of 5 and were asked to choose between two options for 100 periods. A student would receive a payoff equal to 1 when he was in the minority and 0 otherwise. The game was run with six different treatments. In each treatment the experiment was run with four groups of five players, resulting in a total of 120 players. Between the treatments two variables were varied. The first variable was the information the players receive. In three treatments the agents were given aggregate information and in the other three they were given full information. With aggregate information the players only know which option was the winning side. With full information the players could also see the choices of all other individuals in the group. The second variable that was varied was the number of past rounds for which the agents can see information. The agents could look back either 1, 4 or 16 periods. They find that generally the aggregate efficiency is higher than in the symmetric Nash equilibrium where all agents choose both options with equal probabilities. Few agents used strategies that were almost pure Nash equilibria, i.e. strategies where an agent almost always chooses the same option. Agents chose the same option as in the last period 65% of the time. They chose the same option more frequently when they won in the last period. Chmura and Pitz (2006) performed experiments between 9 agents who were asked to choose between two roads for 100 periods. An agent would receive a payoff equal to 1 when he was in the minority and no payoff when he was in the majority. There were two treatment groups. In the first treatment group the only information agents receive was whether they were in the minority or majority in the last period. In the second treatment group additional information was shared with regard to the distribution of the agents amongst the
two roads. The main result found in the study was a negative correlation between the number of times an agent switches and his average payoff. This result was especially prominent in the second treatment group. Another study, that found a similar result, was done by Selten et al. (2007) where 18 agents were asked to choose between a main road and a side road for 200 periods. For both roads the travel time depended on the number of people on the road and the travel times increased with the number of users. But for equal numbers of users the roads had different travel times. After each period the players receive a payoff which linearly decreases with the travel time. There were two treatment groups. In the first treatment group players were told their travel time, payoff and choice in the last period in addition to the cumulative payoff and the current period number. In the second treatment group players were also provided with the travel time of the other road. For both groups there was a negative correlation between the number of road changes and the payoff. This result was stronger for the second treatment group.
A game closely related to the minority game is the congestion game introduced by Rosenthal (1973). In a congestion game a group of players make use of resources from a common pool and their payoff depends on the number of other players that pick the same resource. An example of such a game is choosing roads in traffic, see e.g. Nagel et al. (1997). A minority game is a special type of congestion game. It has been shown that every congestion game is a potential game and therefore the minority game is too (Monderer and Shapley, 1996). This means that results found for potential games also hold for minority games. Another type of situation the minority game resembles is speculative trading (see e.g. Challet et al., 2000). The two options in the minority game represent buying and selling. When a lot of people want to buy, the price will go up and it might be better to sell and vice versa. In this situation it is always best to choose the option the fewest players choose. Real life markets are not this simple, but although the minority game might not accurately resemble reality, variants of the game can reflect it quite accurately (Challet et al., 2001).
Chmura and G¨uth (2011) use both a theoretical and experimental approach to investigate
a 3-player minority game. In order to investigate the game theoretically, the authors use both conventional equilibrium selection theory introduced by Harsanyi and Selten (1988) and alternative concepts. Using these concepts they find that it is expected that the players coordinate upon the symmetric Nash equilibrium. To test the theoretical finding an experiment was devised where agents were asked not to choose between options but to give a probability distribution over the two options. The agents were given this task for 50 periods and in each period a random draw is taken from the distributions that decides which option the agents choose. In each round the agents were given information over the last five periods regarding the choice and earnings of both themselves and the interacting players. On the aggregate level the behavior of the agents is in line with the symmetric Nash equilibrium, but only a quarter of the agents follow this strategy. There has been other research to suggest that the theoretical prediction in the minority game are not necessarily in line with the experimental results (Kets and Voorneveld, 2007). On the aggregate level the outcomes often resemble those of the symmetric Nash equilibrium, but the individual players do not actually follow the Nash equilibrium strategy (see e.g. Selten et al., 2007 and Bottazzi and Devetag, 2007).
Some studies include an evolutionary element in the minority game. When Challet and Zhang introduced the minority game they also performed an evolutionary competition between various
strategies. In every period the worst performing player would discard his own strategy and use the strategy of the best performing player from then on. This implies that eventually all players use the same strategy. In this experiment a type of learning was observed where the fluctuation of the attendance of the options drops over time. Johnson et al. (1999) introduced a modified version of the minority game, which has been subsequently referred to as the evolutionary minority game (EMG). In the EMG there are an odd number of players choosing between room ’0’ and room ’1’. Just like in the minority game the players in the minority win. Each agent is given a memory of length m which contains the outcomes of the previous m periods. An outcome is equal to the group which formed the minority, so the memory 000 indicates that in the previous three periods the minority was in room ’0’. The agents base their decisions on a common register which contains the most recent outcomes that correspond with all possible memories of length m. In every period this register is updated. To prevent all agents from choosing the same room every period, each agent is assigned a unique variable p randomly chosen from the interval [0,1]. The agents only choose the correct room according to the register with their probability p. The opposite room is chosen with probability 1-p. In every period the agents in the minority (majority) obtain a payoff equal to 1 (-1). When the cumulative payoff of a player drops below a certain number, he is assigned a new p which is drawn from a range of values centered around the old value. The authors found that the p-values tended to go the the extremes and that cautious agents performed poorly. Yuan and Chen (2004) performed several experiments based on the EMG, where instead of basing their decisions on a common register, agents were randomly assigned S strategies from the 22M possible strategies. The agents keep track of the success of their assigned strategies and use the strategies with the highest cumulative payoff in the previous M periods. When the cumulative payoff of an agent drops below a threshold, he is randomly assigned S new strategies. Several experiments are run where the memory length M, the number of assigned strategies S, the number of agents per game and the threshold at which strategies are replaced are varied. The standard deviation of the number of agents in one group σ is used to determine the efficiency of the system. The games are played both with and without evolution and it is found that when M is small evolution results in smaller σ values.
Linde et al. (2014) asked students to formulate strategies in five consecutive rounds to be used in a five-player minority game. Unlike in most laboratory experiments, where agents are asked to choose between the two options in every round, instead the agents had to formulate a strategy for all possible information sets before the game began. The students were able to update their strategies between the rounds. Another distinct feature was that strategies no longer indicated which option to choose for every information set, but instead they returned a probability to change between the two options. This introduced a component of randomness in the game. In all five rounds a competition was played, where the agents could earn a monetary payoff depending on their performance. In total 107 unique strategies were submitted in the five rounds. The authors performed an evolutionary competition between these strategies. In this evolutionary competition all strategies were assigned equal weights in the first generation and these weights were updated according to performance over time. Unlike in the previously discussed evolutionary competitions the strategies do not all compete against each other in one game, i.e. the number of strategies per game remains five. It was found that over time the average payoff rose and the number of changes between options dropped. In this thesis these 107 strategies are used as data and the outcome of the evolutionary competition is looked at more closely. First, the competition
is repeated various times with different parameters in order to find out whether the outcome was due to a specific set of circumstances or if the outcome is more robust. After that strategies that perished in the early stages of the competition are reintroduced as invaders in order to test their performance in the later stages of the game. The most successful strategies are then omitted from the game to try and find different equilibria. In the next section invaders are designed in order to find out whether strategies exist that are able to invade. Then the strategies that survived the initial evolutionary competition are looked at more closely. Finally the competition is run with various changes made to the mechanics of the game.
This paper is organized as follows. In section 2 the model is formally described. Section 3 contains a description of the experimental design. Section 4 contains a quick analysis of the strategies used as data. The results of the experiments are in section 5 and section 6 is the conclusion.
2
The model
The minority game is played between an odd number of players N who all choose between two options (say A and B). When players are in the minority, they win the game and receive a point. Let si be 0 if player i chooses option A and 1 if he chooses B. Then the payoff for player i is as
follows πi(s) = si, if PNj=1sj ≤ N −12 1 − si, if PNj=1sj ≥ N +12 .
There is a pure Nash equilibrium in the one-shot version of this game when the maximal number of players are in the minority, i.e. when N −12 players choose one side. This means there are a total of N −1N
2
pure Nash equilibria. In these equilibria the total payoff for all players is at its maximum. The equilibria are not strict since all players in the majority are indifferent between the two options.
Besides pure there are also mixed Nash equilibria, one of which is symmetric. In the symmetric mixed Nash equilibrium all players choose both options with a probability of 0.5. In this situation all players have the same expected payoff, which is lower than the average payoff in the pure Nash equilibrium, since the number of players in the minority is not guaranteed to be maximal. There are infinitely many asymmetric mixed Nash equilibria.1
Smith and Price (1973) introduced a modified form of the Nash equilibrium called the evolutionary stable strategy (ESS). This was done for a two-player game, but the ESS has also been defined for games with more than two players (Broom et al., 1997). The idea behind an evolutionary stable strategy is that when all players use this strategy, then no other strategy will be able to invade successfully. In order to introduce a formal definition of the ESS, let us first
1For instance when N −1
2 players choose option A with certainty and N −1
2 players choose option B with certainty,
the last player is indifferent between the two options. Since the last player is indifferent between the two options, he can choose option A with any probability. Since all of these choices represent a mixed Nash equilibrium, there are infinitely many.
define U(X;Y1, ..., YN −1) as the expected utility of a player using strategy X where the other N-1
players use strategies Y1, ..., YN −1.2 Furthermore when multiple players follow the same strategy
this can be denoted by a superscript to the right of the strategy which indicates the number of players using this strategy. For example U(X;Y1,Y2N −2) is the expected utility of a player using
strategy X when one of the other players uses strategy Y1 and the other N-2 players use strategy
Y2. Consider the set of players to be N = {1, ..., N }, with N a positive, odd number larger than
1. Using these conventions, a definition for an ESS in a N-player minority game is as follows Definition 2.1. A strategy X is evolutionary stable, given the set of players N , when there is an i in the interval [0,N-1] such that for all Y 6= X the following two conditions hold
• U (X; Yi, XN −i−1) > U (Y ; Yi, XN −i−1)
• U (X; Yj, XN −j−1) = U (Y ; Yj, XN −j−1) ∀ 0 ≤ j ≤ i − 1
where the second condition applies for i 6= 0.
In order to give an intuitive understanding of the definition above consider the situation where all agents use strategy X and strategy X is ESS. The first condition implies that for every invading strategy Y, there is a number of invaders i at which the agents using strategy X are better of than those using strategy Y. The second condition implies that for every number of invaders smaller than i the invaders are not better of than the agents using strategy X. In other words when the proportion of the population that uses a strategy different from X grows, there comes a point where they are worse off than those using strategy X. A concept closely related to the ESS is the neutrally stable strategy (NSS) (Smith, 1982). By definition all ESS are also NSS, but additionally a strategy X is also NSS when there is a strategy Y 6= X where
U (X; Yi, XN −i−1) = U (Y ; Yi, XN −i−1) ∀ i ∈ [0, N − 1].
It can be shown that the symmetric Nash equilibrium is also an ESS (and NSS) in the one-shot minority game.3
Proposition 2.1. The strategy to choose between both options with a probability 0.5 is an ESS in the one-shot minority game.
Proof. Let X be the strategy to choose both options with equal probabilities and let Y be an invader that chooses option A with probability p. When N − 1 players use X, then the N th player is indifferent between option A and B, i.e. U (X; XN −1) = U (Y ; XN −1) ∀ Y .
Now consider the payoff of the N th player given the situation where of the other N − 1 strategies N − 2 use X and one uses Y. Since Y 6= X, it must mean that p 6= 0.5. When p is smaller (bigger) than 0.5, the N th player is better of choosing side A (B) with a higher probability. Or equivalently
U (X; Y ; XN −2) > U (Y ; Y ; XN −2) ∀ Y 6= X.
This is because all except one of the other players choose between both options with equal chances, so the one player determines which side is most likely to be the minority. The conditions for ESS according to the definition have now been met for i = 1.
2
Strategies can be both pure and mixed.
We now consider the repeated version of the minority game with five players. In this game all five players have to choose between the two options for a certain number of consecutive periods. As before the players in the minority get a payoff in each period.4 In this game there is once again a unique symmetric Nash equilibrium that is to choose both options with a probability p = 0.5 in every period. Unlike in the one-shot version of the game, here the symmetric Nash equilibrium is neither an ESS nor an NSS.
Proposition 2.2. The strategy to choose between the two options with a probability p = 0.5 is not an ESS nor an NSS in the repeated minority game with five players.
Proof. Let X be the strategy to choose both options with equal probabilities and let Y be an invader. In order for X to be an ESS or an NSS the following condition must hold
U (X; X4) ≥ U (Y ; X4) ∀ Y 6= X. (1)
When 4 players choose options A with a 50% probability, then the 5th player will be indifferent between both options, so the above condition always holds with equality. Since the condition holds with equality the following condition must also hold in order for X to be an ESS or NSS
U (X; X3, Y ) ≥ U (Y ; X3, Y ) ∀ Y 6= X. (2)
Let Z be the strategy to choose both options with equal probabilities until any player has obtained a positive payoff in which case you never change again. We can calculate both U (X; X3, Z) and U (Z; X3, Z) and show that condition 2 does not hold.
As long as no player is able to obtain a positive payoff, both strategies are equal which implies both strategies give the same expected payoff. Now consider the situation where there has been at least one winner. Assume without loss of generality that player 1 uses strategy X and of the other players, three use strategy X and one uses Z. Player 1 gets a payoff in three scenarios: a) when he is in the minority with the invader, b) when he is alone in the minority and c) when he is in the minority together with a player that also uses strategy X. These three scenarios happen with probabilities 0.54, 0.54 and 3 ∗ 0.54 respectively, so
U (X; X3, Z) = 0.54+ 0.54+ 3 ∗ 0.54 = 0.3125.
Now consider the case where player 1 uses strategy Z and of the other players three use X and one uses Z. When player 1 is on the same side as the other player using Z, then he gets a payoff when all the other three players choose the other side, which has a probability of 0.53. When player 1 is on the opposite side as the other player using Z, then he gets a payoff with a 50% probability. Or equivalently
U (Z; X3, Z) = p ∗ 0.53+ (1 − p) ∗ 0.5,
where p is the probability that both players using Z have chosen the same option. Let A be the situation where both players using Z are on the same side and B the situation where there is at least one winner. Then
p = P (A|B) = P (A ∩ B) P (B) = 0.5 − 0.54 1 − 0.54 = 7 15,
4Let us assume that an agent can observe whether he won, the group sizes and whether he changed in the
previous periods. Using this information strategies can take any form as long as they choose both options with non-negative probabilities that add up to 1 in every period.
and then
U (Z; X3, Z) = 7 15 ∗ 0.5
3+ 8
15 ∗ 0.5 = 0.325.
Since U (X; X3, Z) < U (Z; X3, Z) it has been shown that strategy X is not ESS nor NSS.
3
Experimental design
In an experiment designed by Linde et al. (2014) students were asked to program strategies to be used in a five-player repeated minority game for 100 periods. The students were part of the beta-game bachelor program and the experiments took place in the CREED laboratory at the University of Amsterdam. The students submitted a strategy in five consecutive rounds. In the first round, they designed a strategy based solely on a few test games against other players.
Strategies can be based on information from the last five periods. For each of these five periods, the players receive information regarding the size of the group they were in and whether the strategy itself changed between options. Using this information, a strategy determines a certain probability to switch. So an agent is not asked to choose between options, but is instead asked when he wants to switch between options. This is done in order to impose symmetry.5 In the experiment the options were referred to as Red and Blue. Figure 1 shows what a participant sees on his screen when formulating a strategy. A strategy is formed by a number of IF-statements. When an IF-statement is met, a certain value in the interval [0,1] is returned which indicates the probability to switch. All IF-statements following the first are considered else if statements. When none of the IF-statements is met, the probability 0 is returned and the agent using the strategy does not change. The IF-statements can make use of various logical expressions such as AND, OR, (in)equality and negation.
Once all participants have designed and submitted a strategy, a tournament is run between the strategies. For every possible combination of the submitted strategies a game is played. Since all strategies are based on the previous five periods, random outcomes are chosen for the first five periods of the game. After that, 100 periods are played according to the 5 strategies. After all possible combinations of strategies have played a game, the average payoff is calculated for each strategy. This average payoff is based on the the last 100 periods of each game the strategy was present. So the random outcomes in the first five periods are solely to create a history for each player and are not included when the average payoffs are calculated. Based on the average payoffs the strategies are ranked. The students all receive an e-mail containing this ranking together with the average payoff for all strategies.
After the first round, the students are given the opportunity to test strategies. They can submit strategies to play the minority game against four strategies randomly chosen from the strategies submitted in the previous round. Rounds two through five all take place a week after the previous round. In each round there is a competition similar as in round 1 and after each round the students can update their strategies.
In total, there were 107 unique strategies amongst those submitted in the five rounds. Linde et al. used these 107 strategies to run an evolutionary competition as follows. All strategies
5This seems a reasonable restriction to impose, since the payoffs depend solely on the group sizes and not on
Figure 1: Computer screen a participant of the experiment is faced with when formulating a strategy.
are given the same weight in the first generation (equal to 1071 ). In every generation g 2000 simulations are run in which a 100-period minority game is played. The five strategies which play against each other in a simulation are chosen (with replacement) based on the weights. A weight of a strategy can be seen as its chance to be picked. For each strategy the average number of points earned is calculated which is denoted as P(i,g) for strategy i. After all 2000 simulations, the weights are updated as follows:
¯
w(i, g + 1) = max((1 + λ[P (i, g) − M (g)])w(g, i), 0) (3)
where λ is the selection pressure and M(g) is the average payoff earned by all strategies in generation g. Equation 3 implies that when a strategy performs better than average its (non-normalized) weight increases. When a strategy is not chosen in any of the 2000 simulations,
its weight is set at 0. The weights are normalized to make sure they add up to one:
w(i, g + 1) = Pw(i, g + 1)¯
jw(j, g + 1)¯
(4)
Linde et al. ran the evolutionary competition five times between the 107 strategies for 500 generations with an evolutionary selection pressure equal to 0.05. A graph that shows the evolution of the strategy weights averaged over all five competition can be observed in Appendix A. After 500 generations there were only four strategies left with a positive weight. A description of these strategies can be found in Table 1. The strategies are all named x-y, where x is the round in which they were submitted and y is the id of the agent who submitted the strategy.
1-47 If you won the last period: don’t change
Else if you lost the last period in a group of 3 or 4: change with probability 0.2 Else if you lost one or both of the periods -2 and -3: change with probability 0.6 2-35 Only change (with probability 1) when you lost the last period in a group of 5
1-32 Only change (with probability 1) when you lost the last period in a group of 4 and/or if you won in period -3 in a group of 1 1-34 Never change
Table 1: The surviving strategies after 100 generations.
A cluster analysis that grouped the strategies into 6 clusters showed that all four of the surviving strategies were from clusters 5 and 6. A common feature amongst these strategies is to seldom switch. In general as the competition progressed the average payoff increased and the average percentage a strategy changes decreased.
4
Analysis of the strategies
In total 107 unique strategies have been submitted in the five rounds of which 42, 26, 14, 13 and 12 in rounds 1, 2, 3, 4 and 5 respectively. As mentioned before each strategy consists of a number of IF-statements, that if met return a certain probability to change in the interval [0,1]. On average the strategies consist of 4.36 IF-statements with a maximum of 17. Due to the design of the strategies it is possible that IF-statements are never met. This is the case when the IF-statement tests for a condition that can never be met or when one of the preceding IF-statements is always met. An example of a strategy of the first type is displayed below. IF ($W2[1] AND ($W1[2] AND $W2[2] AND $C[1])){
RETURN 1; } IF ($C[2] AND ($L3[2] OR $L4[2] OR $L5[2])AND $L3[1]){ RETURN 0; } IF (1){ RETURN 0.75; }
In order for the first condition to be met, the player must have won in both a group of 1 and 2 two periods ago. Since this is obviously impossible, the condition is never met. An example of a strategy of the second type is the following
IF (($W1[1] OR $W2[1])OR ($L3[1] OR $L4[1] OR $L5[1])){ RETURN 0.5; } IF ($C[1] ){ RETURN 0.5; }.
The first condition is met when the player won the previous period in a group of 1 or 2 or when he lost the previous period in a group of 3, 4 or 5. This condition is always met, so the second IF-statement is never reached. Out of the 107 strategies 29 contain IF-statements that are never reached. More than half of these strategies (17) were submitted in the first round, which suggests that the students did learn to some degree not to include unnecessary strategies. When these
IF-statements are excluded the average number of IF-statements used becomes 3.85.
When an experiment is performed where all possible scenarios are investigated for all strategies it is found that on average the strategies change 42.8% of the time. They change slightly more after losing the last period than after winning the last period (45.1% versus 39.4%). On average the strategies look back 2.39 periods. Most strategies consider the last period (92.5%) and few strategies look back five periods (9.35%).
In order to test the behavior of the strategies in actual games 200,000 simulations are run of the 5-player minority game with 100 periods where in each simulation the strategies are chosen randomly. Here the strategies change roughly the same number of times with an average of 44.5 per simulation, earning an average payoff equal to 31.57. The Pearson correlation between the average payoffs and average number of changes is -0.620 and the Spearman correlation is -0.531 (both p-values are smaller than 0.01). This indicates that strategies that switch less are relatively more successful. In 63.4% of the periods there are two winners and 30.9% of the time there is one winner. This is slightly better than in the symmetric Nash equilibrium.
To test the performance of the strategies in homogeneous situations the minority game is played 100 times for every strategy where all five players use the same strategy. Here the strategies perform a lot worse than against other strategies and the average payoff is only 22.82. This suggests that most strategies are designed under the assumption that the other players will be using different strategies.
5
Evolutionary competition
5.1 Robustness
In the evolutionary competition, there are several variables involved for which values are chosen. The number of simulations is set at 2000, presumably to ensure that all strategies are picked a large enough number of times to reduce the variability of the outcome. But the number of periods per game and the selection pressure seem somewhat arbitrarily chosen. To investigate the impact of these choices, the same evolutionary competition is repeated with different values for the number of periods and selection pressure. First, the results from Linde et al. (2014) are reproduced by keeping the selection pressure at 0.05 and the number of periods per game at 100.6 The competition is run for 100 generations and the results of this experiment can be observed in Figure 2. After 100 generations, the fractions of the strategies are virtually the same as those found by Linde et al. after the same number of generations. This is to be expected, since Linde et al. repeated their experiment five times and found similar results each time. Once again strategy 1-47 is the most successful and is able to obtain a share of around 60% after 60 generations, which it is able to retain. The four strategies with the largest weights after 100 generations are the same strategies that survived the Linde et al. experiment. The strategy with the largest proportion after those four perished the last in the Linde et al. experiment.
To determine the effect the choice of the selection pressure has, the evolutionary competition is repeated with four different values: a) 0.01, b) 0.04, c) 0.06 and d) 0.10. These variables are
Figure 2: Evolutionary competition between all 107 unique strategies with the selection pressure equal to 0.05. On the horizontal axis is the generation and on the vertical axis the proportion of the population.
chosen to investigate a small and large change in the selection pressure both positive and negative. The results, as observed in Figure 3, show that the same strategies survive the competition with similar proportions. The main effect the change of selection pressure has is that the final proportions emerge after fewer (more) generations when the selection pressure is higher (lower). A higher selection pressure also causes the final proportions to fluctuate more around a certain constant. This is because although the a certain equilibrium is reached, there will always be some fluctuations. When a strategy has obtained a higher (lower) than average payoff the proportion of this strategy will increase (decrease). A higher selection pressure causes this change in proportion to be larger.
In order to test whether the number of periods per simulation has an effect on the outcome, the experiment is repeated with the selection pressure once again at 0.05, but now the number of periods per simulations is set at 20. If these periods are representative of all 100 periods, then it is expected that the average payoffs decrease by 80%. Equivalently, the expression [P (i, g) − M (g)] in equation 3 drops by 80%. In equation 3 it can be observed that decreasing the selection pressure with 80% would have the same effect on the way the weights are updated. This means that the results for this competition should be the same as those for the experiment with 100 periods and a selection pressure equal to 0.01 (from now on referred to as game001). The results as observed in Figure 4a show that the outcome is quite different from game001. It now takes 300 generations before strategy 1-47 is able to obtain a share of 30%, while in game001 it only took 143 generations. Although the same six strategies were the most successful in both games, there are also some changes regarding how the strategies perform. Strategy 2-32 was now able to
(a) Selection pressure = 0.01 (b) Selection pressure = 0.04
(c) Selection pressure = 0.06 (d) Selection pressure = 0.10
Figure 3: Evolutionary competitions with differing values for the selection pressure. On the horizontal axes are the generations and on the vertical axes the weights.
obtain a share of almost 25%, while in game001 it had a share of only 3.8% after 300 generations (which decreased even further). The other strategy with such a large discrepancy between the two games was strategy 2-35, which only obtained a share of 2.5% compared to the 11.6% share it obtained in game001. These results clearly show that the number of periods has an effect on the performance of the various strategies. A possible reason for this is that it takes a couple of periods before the strategies are able to distinguish themselves from each other. In order to test this, the same experiment is run where the first 15 instead of 5 periods do not count towards the average payoff. This means that after the first five periods, where all strategies choose at random, the strategies play for 10 periods to orientate and after that the game is played for 20 periods. The results of this experiment can be observed in Figure 4b. Here the result is similar as in the game without the orientation period, which suggests that lack of an ability to distinguish is not the reason behind the discrepancy observed earlier.
In order to examine the impact the number of periods has more closely, the average payoff is calculated for every period. This is done for every strategy separately. Then for every period and every strategy the difference is taken between the average payoff the strategy has in the period and the average of all strategies in that period. For the six strategies with the largest weight in the original evolutionary game, the results can be observed in Figure 5. There seems to be a clear linear trend for all strategies where they perform either better or worse as the period number increases. This linear relationship is estimated using the following linear model
(a) Without orientation (b) With 10 periods as orientation
Figure 4: Evolutionary competitions with selection pressure = 0.05. In (a) the competition is run for 20 periods. In (b) it is run for 30 periods, but the first 10 periods are not included when the average weights are calculated.
where yit is the difference between the average payoff of strategy i and the average payoff over
all strategies in period t and it is an error term. For all six strategies this model is estimated
using ordinary least squares. The results can be observed in Table 2. For strategies 1-47 and 4-1 the β1-estimate is positive and for the other strategies it is negative. This means that strategy
1-47 performs especially well in the later periods of every simulation and will have comparatively lower payoff when the number of periods decreases. For all strategies the β-values are found to be significant with p=0.001. The high values of R2 suggest that the estimated relationship is close to linear.
Figure 5: The difference between the average payoff for a strategy and the average payoff for all strategies on the y-axis and the period on the x-axis.
Strategy β0 β1 R2 1-32 0,02708 (0,000523) -0,000396 (8,99E-06) 0,951919 1-34 0,01210 (0,000472) -0,000131 (8,12E-06) 0,726721 1-47 -0,00063 (0,000171) 0,000185 (2,94E-06) 0,975893 2-32 0,01779 (0,000593) -0,000255 (1,02E-05) 0,864594 2-35 0,01144 (0,000346) -0,000134 (5,95E-06) 0,838250 4-1 -0,01272 (0,000820) 0,000271 (1,41E-05) 0,790723
Table 2: Results of OLS regression with standard deviations between brackets.
5.2 Invasions
5.2.1 Reintroduction of strategies
Most strategies die out quite quickly in the original competition. After only 17 generations, less than a quarter of the initial strategies have survived. When the selection pressure is lowered, it takes longer before strategies die out. While this could allow for strategies that died out in the first generations to regain some weight, instead the same strategies die out only after more generations. In order to test the performance of these strategies in the later stages of the game, an experiment is performed where they are reintroduced in the game. In these experiments the strategies act as invaders and are given a 1% proportion. The original four surviving strategies are given proportions equal to 99% of their weights after 500 generations in the original game. The competition is run for five generations after which the weights of the four non-invading strategies are restored and another invader is introduced. The number of simulations per generation is set at 200 to allow for a faster computation time. This seems reasonable, since there are only five strategies. The proportion development of the invading strategies can be observed in Figure 6a. After two generations only 55% of the invaders have a positive proportion and after five generations only 27%. There is only one strategy that is actually able to increase in proportion after 5 generations. This strategy (2-32) was the last strategy to die out in the original competition, so presumably even this strategy will eventually die out. A description of this strategy can be found in Table 5 in Appendix B. There are 28 strategies which are able to survive the first 5 generations. When these strategies are reintroduced once more, only five have a positive weight after 20 generations. These five strategies are reintroduced a final time, where the number of simulations is increased to 1000 to decrease the variance. These competitions are played for 50 generations and the result can be observed in Figure 6b. The only strategy to survive after 50 generations is 2-32, which was also the most successful invader before. Despite dying out in the original game, it seems this strategy is able to invade in some capacity.
There seems to be a fairly steady equilibrium composed of four strategies, where only one of the other strategies is able to invade, but the question arises whether there are other equally steady equilibria which can be composed of the 107 submitted strategies. To find other equilibria the evolutionary competition is repeated without the four best performing strategies. This ensures any equilibrium that forms is composed of different strategies than the previously found equilibrium. The results from this game can be found in Figure 7a. There are once again four strategies that survive the game (see Table 5 in Appendix B for a description of these strategies). As in the original game these strategies are all from clusters 5 and 6. The average payoff per
(a) Invasions of all the strategies that per-ished.
(b) Invasions of the most successful in-vaders.
Figure 6: Strategies that perished in the original game are reintroduced one at a time and given a starting weight of 1%. The graph on the left displays the proportion development of these invading strategies over 5 generations. The graph on the right shows the development of the weight for the 5 most successful invaders with 1000 simulations per generation and 100 generations.
simulation in the last 10 generations is 35.0 compared to 38.4 before. Meanwhile the average number of times a strategy switches has increased from 8.5 to 18.0. So in this new equilibrium strategies switch more often and have a lower average payoff.
(a) Without the four surviving strategies from the original game.
(b) Without the strategies from clusters 5 and 6.
Figure 7: Evolutionary competitions where certain strategies are omitted from the game. Now the four strategies which were omitted are introduced one by one into the game and given a starting weight of 1%. The other strategies are given a starting proportion equal to 99% of their average over the last 10 generations (between generations 190 and 200). The number of simulations is set at 1000 to allow for a faster computation time. In Figure 8a it can be observed that all of the originally omitted strategies are able to invade, i.e. all strategies are able to obtain a proportion larger than 1% after 50 generations. Although all strategies are able to invade on some level, there is an apparent difference between the strategies with respect to the speed at which they invade and their success. Three of the invading strategies invade quickly while the other one largely keeps the same proportion over time. Because the invaders gain in weight, there must be strategies that lose in weight, but not all strategies suffer in the same way from the
invasions. When strategy 1-47 invades there is one strategy (4-1) which loses 45% of its starting weight after 20 generations, while another only loses 12% (2-4). But while strategy 2-4 was the most effective in retaining its proportion in the first 20 generations, it was the first strategy to die out after 48 generations. On the other hand when strategy 1-34 invades, this same strategy has actually gained weight after 100 generations.
The introduction of the strategies also has an effect on the height of the average payoff. Especially for the two most successful invaders their introduction results in an increase in average payoff up to 6.5% compared to the last 10 generations in the game where they were not present. For the other two invaders there is only a slight increase. Despite the increase in average payoff it was still lower than in the original equilibrium.
Figure 8b shows that when the four strategies are introduced together the original equilibrium is quickly restored. All strategies have weights after 100 generations that are roughly the same as the weights they had at the same point in the original game.
(a) Invaders are introduced separately. (b) Invaders are introduced together.
Figure 8: Evolutionary competitions where the four strategies that survived are introduced as invaders and given a starting weight of 1%.
Omitting the best performing strategies does not have a large effect on the outcome of the game. Similar strategies survive the game and the omitted strategies are able to quickly invade when reintroduced. In order to try and find an equilibrium with a different type of strategies the evolutionary competition is played again without all strategies from clusters 5 and 6. The result from this competition between the remaining 76 strategies can be found in Figure 7b. Whereas in all previous competitions the strategies which seldom switched were successful, this time the most successful strategies switch more than average. In addition to that, the switch percentage and average payoff are now positive correlated, see Figure 9b. This shows that strategies that seldom switch do not necessarily win the evolutionary competition. The average payoff in the last 10 generations is now 37.58, compared to 35.15 at the same point in the game without the four surviving strategies. This shows that by leaving out strategies the average payoff can actually end up being higher at the end of the evolutionary competition. When the four originally surviving strategies are introduced as invaders into this game, they are unable to invade. As before, they are introduced separately, given a starting weight of 1% and the number of simulations is set at 1000. All strategies perish after only one generation since their average payoff is far below the average.
(a) Without the four surviving strategies from the original game.
(b) Without all strategies from clusters 5 and 6.
Figure 9: On the horizontal axis the generations, on the left vertical axis the average percentage strategies change and on the right vertical axis the average number of points earned over all strategies.
the original game is able to invade. When the four originally surviving strategies are omitted from the game, the equilibrium that forms consists once again of strategies from clusters 5 and 6. When the four strategies are reintroduced, they are able to invade. An evolutionary competition without all strategies from clusters 5 and 6 results in an equilibrium with strategies that often change. In this equilibrium the originally surviving strategies are not able to invade.
5.2.2 Designed invaders
In the previous section it was found that the equilibrium that is formed in the evolutionary competition is quite stable. Only one of the other 103 strategies is able to invade and this strategy also eventually perishes in the original game. In order to test whether alternative strategies exist that are able to invade, the competition is continued with the goal of designing strategies that are able to invade. The way this is done is by keeping track of the expected payoff of switching and not switching depending on the history. Different history lengths of 1, 2 and 3 periods are considered. The history length indicates how many periods the agent takes into account when making his choice whether to switch. When the history length is 1, the choice whether or not to switch is based solely on the previous period, i.e. whether the agent won, the group size and whether the agent switched in the previous period. History lengths of 1, 2 and 3 result in respectively 10, 100 and 1000 different situations to consider. To find out what is the best choice for every situation, payoffs are initialized at 0 for every situation. Then, without updating the weights, 100,000 simulations are run and every time an agent has to choose between switching and not switching the expected payoff for both choices is added to the total for that history situation. After these simulations, there is an expected payoff for both switching and not switching for every situation (all situations are reached at least 10 times even when the history length is 3). Three strategies are formed by looking back either 1, 2 or 3 periods and choosing to either switch or not switch depending on what gives the highest expected payoff in every situation. For instance, the strategy that looks back only 1 period is to simply switch with a 100% chance when the last period was lost in a group of 4 or when you lost the last period in a group of 5 and also switched in the previous period and to never switch otherwise. After the strategies are formed, they are introduced one by one as invader similar to the invaders in the previous section. The number of
simulations per generation is reduced to 1000. It can be observed in Figure 10a that all three of the designed strategies are able to successfully invade. It seems that a longer memory has a positive impact on the speed at which a strategy is able to invade, but in the long run all three strategies perform similar.
The introduction of the strategies results in higher average payoffs. The means of the average payoffs in the last 10 generations are 0.42%, 0.73% and 0.89% higher compared to the mean between generations 90 and 100 in the original game for the strategies with memory 1, 2 and 3 respectively. The invaders also switch between the two options less than average.
When all three strategies are introduced together, they are also all able to increase in proportion as can be seen in Figure 10b.7 The number of simulations is set at 2000 when all three strategies invade together to ensure all of them are picked a sufficient number of times. Although the strategies are able to invade as they were designed to do, their growth quickly stagnates. As a result, they are not able to take over the entire population. Once again, the introduction of the invaders results in an increase in the average payoff.
(a) Invaders are introduced separately. (b) Invaders are introduced together.
Figure 10: Evolutionary competitions where strategies with different memory lengths are introduced as invaders.
The previous results show that there are strategies that are able to invade, although it is not clear yet whether there are any that are able to take over the entire population. The invasions also result in higher average payoff, but it still remains lower than the maximum of 40 per simulation. Now we design a strategy that tries to ensure there are two winners as quickly as possible when all players use this strategy. In the first 5 periods all strategies choose randomly between the two options. This information can be used to quickly coordinate and ensure that there are two winners. The chance that there are two winners in at least one of the first five periods is 99.3%. Since all players keep track of whether they switched in the last 5 periods, they are able to all to choose the same option as they did in any of the last five periods. This means that if all players choose the same option as the last time there were two winners in 99.3% of the simulations they can switch such that in period 6 and onwards there are always two winners. When in none of the first five periods there are two winners it is possible that in none of the periods there was a winner. In this case the quickest way to ensure there are two winners is to all switch with 50% chance until there are two winners. A last possibility is that in at least one
7When there are multiple invaders, they are all given starting weight of 1%. Since there are four invaders, the
of the first five periods there is at least one winner. This information can be used to obtain a situation with two winners more quickly that switching randomly would if the player who won alone chooses the same option he had then and the other players switch with a chance of 36.6% relative to the option they had then. This way almost 70% of the time there are two winners in period 6 after which there are always two winners.
When this strategy is introduced as an invader it is able to invade successfully as can be observed in Figure 11 (the invading strategy is referred to as Invader 110). Not only does this strategy invade more quickly than the strategies that were designed to invade, it is also able to take over the entire population within 120 generations. The average payoff rises from 38.5 in the beginning to 40.0 at the end of the competition. The average number of times a strategy switches drops from 8.2 to 0.19 times per simulation.
Figure 11: A strategy designed to maximize the average payoff is introduced and given a starting weight of 1%.
5.3 Surviving strategies
In this section we take a closer look at the four strategies which were able to survive the original evolutionary competition. In the original evolutionary competition there seemed to be a certain ratio between the four surviving strategies. To test whether the strategies always converge to the same proportions, an evolutionary competition is run between the four strategies where they all have the same starting weight of 25%. The number of simulations per generation is set at 1000.
The results, as can be observed in Figure 12a, show that there is a quick convergence to the final proportions.
(a) With 4 strategies. (b) With 6 strategies.
Figure 12: Evolutionary competitions where only the most successful strategies are included. The strategies are all given the same starting weight.
There is one strategy (1-47) that is a lot more successful in the early stages of the competition, but at a certain point its proportion no longer increases. This suggests that this strategy performs better in groups with few players using the same strategy, because otherwise you would expect the strategy to take over the entire population. To investigate this, an experiment is run that keeps track of the average payoff of all strategies in all possible groups. A group consists of 5 players and there are 4 strategies, which means there are 56 different possible groups, since the order of players does not matter. For every possible group the minority game is run 1000 times while keeping track of the average payoff for all strategy types in the group. When an agent with strategy 1-47 competes against four agents with the same strategy his average payoff is 36.85. Agents with strategies 1-32, 1-34 and 2-35 have an average payoff equal to 43.20, 41.26 and 41.45 respectively when the other four players use strategy 1-47. This confirms that the strategy 1-47 performs relatively poorly when all other agents also play with this strategy. On the other hand if strategies compete only against different strategies then 1-47 performs best with an average payoff equal to 50.75. For strategies 1-32, 1-34 and 2-35 these averages are 48.84, 38.72 and 42.17 respectively. All strategies perform better than average when none of the other group members use the same strategy, which helps explain why none of the strategies die out.
In section 5.2.1 we found that only one strategy was able to invade. Since this strategy was the last to perish in the original game, it is expected that it will also fail to invade. To investigate this, the evolutionary competition is repeated with the six best performing strategies in the original game. This experiment shows that even though strategy 2-32 was able to invade somewhat successfully it still dies out, see Figure 12b. During this experiment data was gathered regarding the groups that were formed and whether the strategies were right when they switched or stayed. In 92.4% of the periods there were two winners, while 7.14% of the time there was one. Almost none of the strategies changed when they won the previous period. Strategy 2-32 changed 50% of the time it won in a group of one and 1-32 changed with a 1.5% chance when it won the previous period in a group of 1 and with a 6.7% chance when it won the previous period in a group of two. All other strategies never changed when they won the previous period. The majority of the changes that were made after winning the previous period were also incorrect
changes, i.e. the agent would have been in the minority had he not changed. On the other hand not changing after losing in the last period resulted in a loss the majority of times.
Throughout the experiments run so far strategy 1-47 has been the most successful. The strategy is actually quite simple, as can be seen in Table 1. The first part of the strategy is to never change after winning in the previous period. This makes sense intuitively and previously we found that the strategies that changed after winning would have been better off had they not changed. Since some of the other strategies also never change after winning, it appears that the success of strategy 1-47 is due to its decisions after losing. After losing in the last period, there are two situations which 1-47 considers: a) losing the last period in a group of 3 or 4, and b) losing the last period in a group of 5. In the evolutionary competition between the six most successful strategies, a player with strategy 1-47 was only in a group of 5 0.41% of the time compared to 61% of the time in a group of 3 of 4. For this reason it is likely that the part of the strategy that states what to do after losing in a group of 3 or 4 in the last period is what makes it so successful. To confirm this an experiment is run where the probabilities to change in both situations are varied from 0 to 1 with steps of 0.1. For each combination of changing probabilities an evolutionary competition is run between strategy 1-47 that uses these probabilities and the other three strategies that survived the original competition. For strategy 1-47 the probability of changing after winning the last period is fixed at 0. The number of simulations per competition is set at 200 and each competition is run for 25 generations. Table 3 shows the average weight of strategy 1-47 in the last 5 generations of the competition for all parameter combinations. Here it can be seen that indeed the probability of changing in situation a has the largest influence on the weight the strategy is able to obtain. It can also be observed that the strategy would have performed slightly better if this probability was 0.1 instead of 0.2. For most combinations of the parameters 1-47 has a larger weight at the end of the competition than at the beginning. The strategy is especially successful when it switches with a small probability in situation a.
Player 47, who submitted strategy 1-47, actually considered losing in the last period in a group of 3 separately from losing in a group of 4, i.e. in the code there are two different IF-statements. Now the previous experiment is repeated, but the probability of changing in situation b is fixed at 0.6 and instead the probabilities of changing after losing the last period in a group of 3 and after losing in a group of 4 are varied separately. The results, as seen in Table 4, show that strategy 1-47 is able to take over almost the entire population after 25 generations when it never changes after losing in a group of 3 and only changing with a small probability after losing in a group of 4 in the last period. The strategy with these parameters can be seen as a simple version of Invader 110 from section 5.2.2 that uses only the information from the last period.8 The reason the strategy performs better when it does not change after losing in a group of 3 is likely due to the improvement in playing in a homogeneous setting. In this setting the strategy now has an average payoff equal to 39.83 compared to 36.73 before. This causes the strategy to be able to outperform the other strategies even when it has already obtained a large weight.
To summarize, it has been shown that in a competition between the best performing strategies, the proportions always converge to the same proportions. None of the strategies die out or take over the entire population, since they all perform better in groups with other strategies. All of the
8The strategy that tries to get two winners as quickly as possible using only information from the last period
would be to never change after winning or losing in a group of 3, to change with probability 0.366 after losing in a group of 4 and to randomly change after losing in a group of 5.
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 0.0 0.1912 0.2885 0.2211 0.3352 0.0299 0.5516 0.4021 0.5350 0.2841 0.3360 0.3138 (0.0133) (0.0204) (0.0150) (0.0125) (0.0050) (0.0104) (0.0153) (0.0107) (0.0088) (0.0150) (0.0090) 0.1 0.5461 0.6449 0.6486 0.6065 0.6480 0.6369 0.6525 0.6914 0.6887 0.6881 0.6302 (0.0134) (0.0076) (0.0115) (0.0070) (0.0081) (0.0133) (0.0105) (0.0059) (0.0130) (0.0139) (0.0159) 0.2 0.4982 0.5697 0.6263 0.5686 0.6047 0.6055 0.5732 0.6404 0.5988 0.6025 0.5412 (0.0066) (0.0124) (0.0117) (0.0081) (0.0068) (0.0095) (0.0073) (0.0096) (0.0202) (0.0068) (0.0067) 0.3 0.4735 0.4887 0.4984 0.5462 0.5381 0.5845 0.5293 0.4821 0.5186 0.5566 0.4971 (0.0138) (0.0081) (0.0120) (0.0172) (0.0064) (0.0114) (0.0044) (0.0177) (0.0087) (0.0100) (0.0175) 0.4 0.4255 0.4534 0.4411 0.4519 0.4609 0.4788 0.4755 0.4888 0.4542 0.4407 0.3992 (0.0237) (0.0171) (0.0129) (0.0153) (0.0118) (0.0071) (0.0154) (0.0147) (0.0075) (0.0057) (0.0110) 0.5 0.3547 0.3980 0.3886 0.4426 0.4268 0.3830 0.4091 0.3749 0.4380 0.4209 0.4033 (0.0101) (0.0211) (0.0098) (0.0034) (0.0121) (0.0159) (0.0129) (0.0107) (0.0081) (0.0146) (0.0130) 0.6 0.3074 0.3820 0.3448 0.3398 0.3671 0.3626 0.3996 0.3440 0.3721 0.3686 0.3085 (0.0211) (0.0162) (0.0126) (0.0051) (0.0206) (0.0125) (0.0138) (0.0128) (0.0035) (0.0081) (0.0163) 0.7 0.3017 0.2841 0.3320 0.3117 0.2733 0.3383 0.2732 0.2855 0.3648 0.3100 0.3021 (0.0174) (0.0038) (0.0129) (0.0149) (0.0200) (0.0094) (0.0145) (0.0196) (0.0180) (0.0187) (0.0260) 0.8 0.2581 0.2501 0.2585 0.3073 0.2345 0.2293 0.2524 0.3023 0.2698 0.2301 0.2236 (0.0115) (0.0109) (0.0070) (0.0135) (0.0172) (0.0255) (0.0234) (0.0025) (0.0110) (0.0274) (0.0120) 0.9 0.1623 0.1104 0.2112 0.2159 0.2632 0.2036 0.2457 0.2143 0.1770 0.1808 0.1926 (0.0137) (0.0114) (0.0142) (0.0060) (0.0077) (0.0146) (0.0161) (0.0099) (0.0120) (0.0041) (0.0040) 1.0 0.0000 0.0923 0.0003 0.0987 0.0772 0.0988 0.0901 0.0993 0.0000 0.0606 0.1053 - (0.0046) (0.0003) (0.0117) (0.0058) (0.0149) (0.0159) (0.0098) - (0.0072) (0.0084) Table 3: Average of the weight of strategy 1-47 between generations 21 and 25, with the standard deviation of the average between brackets. On the horizontal axis is the probability of changing after losing the last period in a group of 5 and after losing in period -2, -3 or both. On the vertical axis is the probability of changing after losing the last period in a group of 3 or 4.
best performing strategies do not change after winning in the last period. The best performing strategy performs better than the others due to it changing with a low probability after losing in a group of 3 or 4. It would have performed even better if it did not change at all after losing in a group of 3.
5.4 Changes to game mechanics
In this section the effect of several small changes to the mechanics of the game are explored. First, several experiments are run where the payoff structure is changed. Then, changes are made that mimic the real world situations the minority game stylizes.
5.4.1 Different payoff structures
In all previous experiments there was no difference between winning in a group of 1 and winning in a group of 2 when it comes to the height of the payoff. For this reason two winners are preferred when it comes to the total payoff of the group. But when the payoff for winning in a group of 1 becomes equal to 2, then this is no longer the case. The graph in Figure13a shows the result of the evolutionary competition where the payoff is doubled when you win alone. The most successful strategy is once again 1-47, which reaches an end weight approximately equal to before. But the second most successful strategy (5-32) died out in the original game within the
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 0.0 0.2732 0.2841 0.2089 0.3544 0.2318 0.1907 0.1949 0.2359 0.1544 0.0593 0.1179 (0.0211) (0.0100) (0.0186) (0.0179) (0.0240) (0.0086) (0.0325) (0.0517) (0.0225) (0.0168) (0.0153) 0.1 0.8764 0.6105 0.6301 0.4362 0.4142 0.5105 0.2918 0.3626 0.2974 0.2945 0.2791 (0.0083) (0.0029) (0.0140) (0.0096) (0.0100) (0.0130) (0.0235) (0.0305) (0.0106) (0.0194) (0.0096) 0.2 0.9192 0.7307 0.6083 0.5308 0.3888 0.3783 0.3301 0.3642 0.3346 0.2572 0.3125 (0.0142) (0.0048) (0.0049) (0.0110) (0.0148) (0.0142) (0.0168) (0.0142) (0.0186) (0.0213) (0.0105) 0.3 0.9605 0.7283 0.6367 0.5319 0.4057 0.4054 0.3467 0.3459 0.3449 0.3063 0.3060 (0.0050) (0.0049) (0.0080) (0.0097) (0.0156) (0.0126) (0.0074) (0.0176) (0.0199) (0.0208) (0.0217) 0.4 0.8978 0.7472 0.6396 0.6006 0.4945 0.4414 0.4394 0.3367 0.2939 0.2889 0.2269 (0.0084) (0.0065) (0.0052) (0.0149) (0.0074) (0.0130) (0.0204) (0.0208) (0.0189) (0.0040) (0.0139) 0.5 0.6591 0.7721 0.6147 0.5504 0.4937 0.4432 0.4035 0.3530 0.3626 0.3012 0.2273 (0.0286) (0.0026) (0.0186) (0.0063) (0.0150) (0.0187) (0.0166) (0.0231) (0.0070) (0.0121) (0.0165) 0.6 0.6352 0.7171 0.5582 0.4807 0.4395 0.3860 0.3765 0.3247 0.2720 0.3012 0.2689 (0.0188) (0.0060) (0.0091) (0.0156) (0.0102) (0.0272) (0.0107) (0.0126) (0.0063) (0.0073) (0.0156) 0.7 0.6272 0.6868 0.6009 0.5313 0.3839 0.3974 0.3586 0.2924 0.3066 0.2639 0.2545 (0.0125) (0.0125) (0.0132) (0.0227) (0.0124) (0.0097) (0.0153) (0.0182) (0.0108) (0.0175) (0.0179) 0.8 0.5047 0.6619 0.5799 0.4925 0.4018 0.3677 0.3206 0.3116 0.2925 0.2616 0.2010 (0.0151) (0.0101) (0.0133) (0.0050) (0.0177) (0.0164) (0.0219) (0.0319) (0.0241) (0.0227) (0.0154) 0.9 0.4815 0.6242 0.5416 0.4257 0.3284 0.3341 0.3285 0.2667 0.1923 0.1642 0.1698 (0.0082) (0.0123) (0.0102) (0.0096) (0.0127) (0.0139) (0.0109) (0.0356) (0.0165) (0.0131) (0.0214) 1.0 0.4161 0.4939 0.3161 0.2851 0.2777 0.2277 0.2288 0.2282 0.2273 0.1686 0.1047 (0.0270) (0.0117) (0.0041) (0.0128) (0.0224) (0.0088) (0.0068) (0.0201) (0.0092) (0.0165) (0.0068) Table 4: Average of the weight of strategy 1-47 between generations 21 and 25, with the standard deviation of the average between brackets. On the horizontal axis is the probability of changing after losing the last period in a group of 3. On the vertical axis is the probability of changing after losing the last period in a group of 4.
first 20 generations. One of the strategies (1-32) which survived the original game now perishes and three strategies which perished in the original game are now able to survive. There is not a large difference between the outcome of this game and the original game, which is to be expected since the situation where there is one winner is still not stable. This is because in the situation with one winner all players in the majority can unilaterally switch and be better off. Even when the experiment is repeated with a payoff equal to 3 for those who win in a group of 1 the results are roughly the same (see Figure 13b). In this game it would be best for the total payoff of the group to have one winner in every period. This would result in an average payoff equal to 60 in every simulation, but instead the average in the last 10 generations is only slightly higher than before (41.73). The average payoff even drops as the competition goes on.
The previous experiments showed that there can be a conflict of interest between the interest of the group and that of the individual. This conflict disappears when there is only a payoff when you win in a group of 1. Players in the majority group can now no longer switch and obtain a higher payoff. Figure 14 shows the evolutionary competition with only a payoff (equal to 1) when you win alone. Although there are some changes when it comes to performance of the strategies, the three most successful strategies in this game (2-35, 1-34 and 1-47 respectively) also survived before. Unsurprisingly, the average payoff is lower, but what may come as a surprise is that the average payoff drops over time. This average payoff would be 20 if there were one winner in every period, but it is not even an eight of this at the later stages of the game. Both the decrease in average payoff and the survival of similar strategies as before suggest that the
(a) Payoff equal to 2 in a group of 1. (b) Payoff equal to 3 in a group of 1.
Figure 13: Evolutionary competitions with all submitted strategies where the height of the payoff depends on the size of the minority.
strategies coordinate in such a way that minority groups of two players are formed instead of one.
Figure 14: Evolutionary competition between all 107 submitted strategies where an agent gets 1 point when no other players are in his group and 0 otherwise.
5.4.2 Mimicking real world situations
The minority game stylizes events such as lane switching and certain forms of stock trading. A common feature in these types of events is that there is a certain barrier that makes it less likely for people to switch. For instance, when someone switches lanes in traffic he cannot immediately switch back and when someone wants to sell his stocks, he cannot buy them back for the same price. To mimic these situations the evolutionary game is repeated where agents cannot switch for one period after they have switched. The result of this experiment can be observed in Figure 15.
