! ! ! ! !
Saving!the!Best!for!Last!
! ! ! ! ! Summary:! ! This%paper%seeks%to%extend%the%work%of%Andreoni%and%Samuelson%(2006)%and%examines%how% changing%the%cooperative%payoff%in%a%repeated%prisoner’s%dilemma%affects%the%overall% frequency%of%cooperation.%%First,%previous%literature%concerning%cooperation%in%prisoner’s% dilemmas%is%reviewed%and%a%useful%Bayesian%model%is%introduced.%%Then,%the%design%and% results%of%an%original%laboratory%experiment%are%analyzed.%%In%the%experiment,%participants% play%a%10Jstage%repeated%prisoner’s%dilemma%with%two%treatments:%a%“Start%Big”%treatment,% where%the%cooperative%payoff%was%increased%in%the%first%three%stages,%and%an%“End%Big”% treatment,%where%the%cooperative%payoff%was%increased%in%the%last%three%stages.%%The%End%Big% treatment%produced%more%cooperative%outcomes%than%the%Start%Big%treatment,%which%was% largely%driven%by%an%absence%of%endJgame%defection%in%the%former%treatment.%%However,%this% difference%was%only%significant%at%the%.10,%and%not%.05,%level.% ! ! ! Daniel!Neri! Student!ID:!10492399! University!of!Amsterdam!–!Faculty!of!Economics!and!Business! Master!Thesis,!MSc!Economics:!Spec.!in!Behavioral!Economics!and!Game!Theory! Advisor:!Prof.!Dr.!Theo!Offerman!1. Introduction!and!Research!Question! !
Cooperation% is% a% prerequisite% for% nearly% every% economic% interaction% (Alexrod% and% Hamilton,%1981).%%In%economic%research,%the%most%ubiquitous%game%used%to%model%situations% where%cooperation%is%socially%desirable,%yet%often%elusive,%is%the%prisoner’s%dilemma%(Figure% 1).% % But% even% given% the% importance% and% desirability% of% achieving% cooperation% in% these% settings,% most% prisoner’s% dilemma% (PD)% research% does% not% focus% on% whether% relatively% innocuous%changes%to%the%stage%payoffs%can%increase%levels%of%cooperation.%%Rather,%they%take% the% game% as% it% is% instead% of% seeking% to% make% small,% but% potentially% effective,% changes.%% Consequently,%this%leaves%a%large%stone%unturned.%
This% paper% seeks% to% discover% if% modifying% selected% stages’% payoffs% in% a% repeated% prisoner’s% dilemma% (RPD)% is% indeed% a% “lowJhanging% fruit”% that% can% be% exploited.% % Could% increasing%cooperative%payoffs%(R)%in%the%early%stages%of%the%game%lead%to%more%cooperation% across%all%stages%because%of%a%statusJquo%bias%and/or%because%increased%cooperative%payoffs% strengthen%the%belief%that%an%opponent%will%choose%to%cooperate,%which%may%help%an%agent% rationalize%his%or%her%decision%to%cooperate?%%Or%might%a%larger%R%in%the%later%stages%of%an% RPD% lead% to% greater% overall% cooperation% because% agents% will% desire% to% coordinate% at% the% cooperative% outcome% early% in% the% game% to% ensure% that% they% will% be% coordinated% at% the% cooperative% outcome% later% on,% when% it% is% most% lucrative?% % Generally,% which% of% these% two% modifications%to%the%standard%payoffs%of%an%RPD%–%“Starting%Big”%(SB)%or%“Ending%Big”%(EB)%–% is%more%effective%in%achieving%a%greater%overall%frequency%of%cooperation%in%an%RPD?%%This% the%central%question%posed%in%this%paper.%
Increasing% R% in% an% RPD% may% seem% both% an% obvious% and% costly% way% to% increase% cooperation.%%After%all,%in%the%real%world,%a%government%would%likely%pay%out%the%increased%R,% and%increased%government%spending%is%not%always%popular%or%cost%effective.%%However,%it%is% not%unreasonable%to%believe%that%the%benefits%of%increased%cooperation%could%outweigh%the% costs.%%Governments%routinely%spend%money%to%increase%cooperation%in%many%areas%of%their% % % Agent%2%
!
% % Cooperate% Defect%!
Agent%1% Cooperate% R,R% S,T%!
Defect% T,S% P,P%!
!
!
Condition:%T>R>P>S%!
!
!
Figure%1.%%Payoff%Matrix%of%the%Classic%Prisoner’s%Dilemma% %domain,%such%as%public%goods,%by%using%laws%and%regulations.%%Thus%it%could%be%viewed%as%a% worthwhile% investment% to% institute% a% subsidy% for% cooperative% outcomes% between,% for% example,%corporations,%if%these%outcomes%are%socially%optimal.%%A%simple%example%modeled% by% a% prisoner’s% dilemma% is% two% competing% firms% deciding% over% whether% to% pay% a% cost% to% reduce%emissions.%%Both%firms%would%earn%most%if%the%other%firm%reduced%their%emissions%and% they%did%not,%but%they%also%would%not%mind%reducing%their%own%emissions%on%the%condition% that%the%other%firm%did%so%as%well.%%In%this%situation,%a%government%could%intervene%and%say% that%they%will%either%subsidize%the%cost%of%reducing%emissions%in%the%first%three%months%or%the% last%three%months%of%a%year.%%This%is%obviously%less%expensive%than%subsidizing%every%month’s% costs%of%reducing%emissions,%but%it%also%has%the%potential%to%be%quite%efficient%and%effective.%% This%paper’s%experiment%addresses%which%variation%would%result%in%more%cooperation%over% the%course%of%that%year.%%It%is%also%worth%noting%that%it%may%be%possible,%especially%in%longer% repeated% interactions,% to% finance% this% subsidy% by% marginally% decreasing% payoffs% to% other% outcomes.% % For% instance,% the% increased% R% payoffs% could% be% at% least% partially% financed% by% marginal% reductions% to% the% R% payoffs% in% other% stages,% or% reductions% in% unilateral% defection% (T)%payoffs.%%While%this%is%a%potentially%important%question%affecting%the%feasibility%of%such%a% policy,%it%is%not%a%concern%of%this%paper.%
% Andreoni%and%Samuelson%(2006)%is%one%of%the%few%papers%that%discusses%the%intrigue% and%potential%impact%of%payoff%modifications%in%an%RPD.%%They%attempt%to%show%how%shifting%
cooperative% payoffs% between% stages% of% an% RPD% affects% the% actions% of% the% agents% and% outcomes%of%the%stages.%%They%use%a%twiceJplayed%prisoner’s%dilemma%where%the%cooperative% payoffs%are%shifted%between%periods%in%various%proportions%(Figure%2).%%In%the%game%!!+ !!% is%fixed%but%! = !! !!!!!%fluctuates.%%A%!%of%0%means%that%the%combined%R%from%stages%1%and%2%is% Stage%1% % Stage%2% % Cooperate% Defect% % % Cooperate% Defect% Cooperate% 3x1,%3x1% 0,%4x1% % Cooperate% 3x2,%3x2% 0,%4x2% Defect% 4x1,%0% x1,%x1% % Defect% 4x2,%0% x2,%x2% % % % % % % % % % ! = !! !!!!!%%and%x1,%x2%≥%0% % %
!
!
!
!
!
!
!
!
!
Figure'2.'Payoff'matrices'from'Andreoni'and'Sameulson'(2006)''!
!
!
!
!
%
entirely%concentrated%in%stage%1.%%A%!%of%1%means%R%is%entirely%concentrated%in%stage%2.%%A%!%of% ½%means%R%in%stages%1%and%2%is%equivalent.%
% %Andreoni% and% Samuelson’s% primary% goal% was% to% determine% which% value(s)% of%!% maximized%payoffs.%%To%address%this%question,%they%conducted%an%experiment%consisting%of% five%sessions%with%22%participants%each.%%Each%participant%was%randomly%and%anonymously% paired%with%a%new%participant%for%each%of%the%20%twiceJplayed%RPDs%that%took%place%in%each% session.%%When%participants%were%matched,%a%value%of%!%was%randomly%selected%from%the%set% {0,% 0.1,% 0.2,% …,% 1},% and%!!+ !!= 10.% % Choices% of% actions% were% represented% as% “push”% and%
“pull”%rather%than%the%traditional%choices%of%“cooperate”%and%“defect,”%in%order%to%keep%the% choices%in%a%more%neutral%frame.%%%
The% authors% created% a% model% to% make% predictions% using% both% the% parameters% in% Figure% 2% and% the% following% assumptions,% drawn% from% a% review% of% commonalities% in% the% literature:%% % (1)%Agents%sometimes%like%to%cooperate% (2)%Agents%like%it%when%their%partner%cooperates%% (3)%If%an%agent’s%counterpart%is%more%likely%to%cooperate,%then%that%agent%is%also%more% likely%to%cooperate% (4)%Agents%are%heterogeneous%in%the%magnitude%of%their%desire%for%cooperation% %
Most% relevantly,% their% model% predicts% that% cooperation% in% stage% 1% is% increasing% in%!%for% ! ∈ (0,1).%%As%they%eloquently%say,%“the%shadow%of%the%future%enhances%current%cooperation”% (129).%%Their%model%also%makes%another%interesting%prediction%that%is%then%validated%by%their% results.%%When%!%is%nearer%to%1,%agents%will%cooperate%more%in%stage%1%with%the%rationale%of% coordinating%now%so%they%are%“linedJup”%to%be%coordinated%to%capture%the%larger%cooperative% payoffs% in% stage% 2;% however,% when% stage% 2% arrives,% they% will% defect.% % It% appears% part% of% an% agent’s%motivation%for%cooperation%is%trying%to%trick%their%opponent%into%cooperating%when% mutual% cooperation% is% more% enticing,% and% then% defecting% to% capture% additional% payoffs,% especially%in%the%final%stage%of%a%game.%
They% also% found% that% when%! ∈ (0,1),% a% cooperative% outcome% occurred% 44%% of% the% time%in%stage%1%and%only%20%%of%the%time%in%stage%2%over%1100%total%trials.%%Average%earnings% reach% an% interior% maximum% when%! = 0.6,% which% is% consistent% with% their% hypothesis.%% Additionally,% average% earnings% are% greater% at% ! = 0.7 %than% ! = 0.5 %and% are% generally%
increasing%in%!.%%This%result%supports%their%hypothesis%that%lower%stakes%in%stage%1%facilitate% more% cooperation.% % However,% I% do% not% completely% agree% with% the% story% they% created% to% explain%this%behavior,%namely%that%“starting%relationships%with%small%stakes%allow[s]%people% a%lowJcost%opportunity%to%exhibit%a%willingness%to%cooperate%and%to%assess%the%propensity%of% others% to% cooperate,% making% investments% in% cooperation% that% can% pay% off% in% the% form% of% mutual%cooperation%for%larger%subsequent%stakes”%(118).%%While%there%is%an%opportunity%for% agents%to%access%their%counterpart’s%willingness%to%cooperate,%the%error%in%this%statement%is% that% the% cost% of% cooperation% is% not% low% when% cooperative% payoffs% are% lower.% % In% fact,% cooperation% is% more% costly% when% R% is% low% because% the% opportunity% cost% of% cooperation% is% greater% when% the% difference% between% R% and% T% is% larger.% % % However,% I% do% agree% with% their% reasoning%that%early%cooperation%manifests%itself%as%the%result%of%a%desire%to%be%coordinated% now% in% order% to% capture% cooperative% payoffs% later% on% (whether% these% payoffs% are% actually% captured%is%a%different%matter).%%
In%their%concluding%remarks,%Andreoni%and%Samuelson%note%that%they%chose%a%twiceJ played%RPD%for%its%simplicity%and%ease%of%modeling.%%However,%an%RPD%experiment%with%more% stages%will%likely%have%more%external%validity%and%may%yield%different%results.%%For%instance,% in% a% longer% RPD,% increases% in% R% in% the% later% stages% may% be% too% distant% to% affect% agents’% behavior.% % Agents% may% get% impatient% waiting% for% these% late% rewards% and% decide% to% defect% early%in%the%RPD,%which%would%decrease%the%likelihood%that%they%would%be%able%to%return%to%a% cooperative%outcome%later%on%in%this%extended,%though%still%relatively%short,%RPD.%%Indeed,%it% may% be% that% in% an% RPD% that% is% longer% than% two% stages,% increasing% R% in% early% stages% coordinates% agents% at% a% cooperative% outcome,% which% then% carries% over% into% later% stages% because%of%inertia,%the%statusJquo%bias,%or%other%factors.%%There%are%compelling%reasons%for% both%arguments,%which%are%explored%in%greater%detail%in%the%Literature%Review%(Section%2)% and%Theory%(Section%3)%sections%of%this%paper.% % Research'Question' ' This%paper%picks%up%from%where%Andreoni%and%Sameulson%left%off,%using%a%10Jstage% RPD1%experiment%to%address%the%following%research%question:%does%increasing%R%in%the%first% three% stages% of% a% 10Jstage% RPD% or% the% last% three% stages% of% a% 10Jstage% RPD% result% in% more% overall%instances%of%cooperation?''The%experiment%featured%two%treatments:%(1)%SB,%where%R%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
is%equal%to%140,%126,%and%114%over%the%first%three%stages,%respectively,%and%98%thereafter,%and% (2)%EB,%where%R%is%equal%to%114,%126,%and%140,%respectively,%over%the%final%three%stages,%and% 98%in%all%stages%beforehand%(Figure%3).%%The%reasoning%for%these%choices%is%explained%in%the% experimental%design%section%(Section%4).'A%high%!%(lower%first%stage%R%payoffs)%in%Andreoni% and%Samuelson’s%experiment%is%analogous%to%the%EB%treatment%in%this%paper’s%experiment,%as% the% final% three%stages% have% higher% R% than% the% first% seven% stages.% % Similarly,% a% low%!%(higher% first% stage% R% payoffs)% is% analogous% to% the% SB% treatment% in% this% paper’s% experiment,% as% R% payoffs%are%higher%in%first%three%stages%than%the%last%seven%stages.'
%
%
The%first%stage%of%the%SB%treatment%and%last%the%stage%of%the%EB%treatment%both%have% the%unique%feature%that%the%cooperative%payoff%(R)%is%equal%to%the%single%defection%payoff%(T).%% Like% much% of% this% experiment’s% design,% the% reason% for% this% decision% stemmed% from% the% results% of% Andreoni% and% Samuelson.% % As% mentioned% above,% they% found% that% a% greater% R% in% stage%2%increased%stage%1%cooperation,%but%then%led%to%many%instances%of%mutual%defection%in% End!Big!(EB)!Treatment!
!
Start!Big!(SB)!Treatment! Stages%1J7%!
Stage%1% % A% B%!
% A% B% A% 98,98% 0,140%!
A% 140,140% 0,140% B% 140,0% 28,28%!
B% 140,0% 28,28% % %Stage%8% %!
% % %!
Stage%2% % A% B%!
% A% B% A% 112,112% 0,140%!
A% 126,126% 0,140% B% 140,0% 28,28%!
B% 140,0% 28,28% % Stages%9%% %!
% % %!
Stage%3% % A% B%!
% A% B% A% 126,126% 0,140%!
A% 112,112% 0,140% B% 140,0% 28,28%!
B% 140,0% 28,28% % Stage%10%% %!
% % %!
Stages%4J10% % A% B%!
% A% B% A% 140,140% 0,140%!
A% 98,98% 0,140% B% 140,0% 28,28%!
B% 140,0% 28,28%! !
Figure%3.%Stage%payoffs%in%the%EB%and%SB%Treatments%!
!
!
!
!
%
stage% 2% itself.% % Thus,% by% making% R% =% T% in% the% final% stage% of% the% EB% treatment% (and% for% equivalence,% the% first% stage% of% the% SB% treatment),% this% problem% may% be% avoided.% % Though% cooperation%in%the%final%stage%of%the%EB%treatment%is%still%weakly%dominated%by%defection,%it% may%still%serve%to%deter%final%stage%defection%that%was%rampant%in%Andreoni%and%Sameulson’s% experiment%even%when%R%was%increased.%
The%present%experiment%was%run%in%the%CREED%lab%at%the%University%of%Amsterdam% across%two%sessions,%one%for%each%treatment.%%There%were%24%participants%in%each%treatment.%% In% each% session,% participants% were% randomly% paired% with% another% participant% with% whom% they%played%a%10Jstage%RPD.%%The%outcomes%of%each%stage%of%each%RPD%were%categorized%as% cooperative%(C,C)%or%not.%%ChiJsquared%significance%tests%show%no%difference%in%cooperative% outcomes%between%treatments%at%the%.05%level.%%However,%the%difference%is%significant%at%the% .10%level.%%Most%interestingly,%the%largest%difference%occurs%in%the%final%stage,%where%the%EB% treatment’s% increased% R% quells% much% of% the% typical% endJgame% defection% behavior% that% is% widespread%in%the%SB%treatment.%
The%remainder%of%this%paper%is%as%follows.%% Section%2%summarizes%a%broad%array%of% relevant%economic%literature,%from%experiments%to%theory,%each%of%which%contributed%to%the% design%and%execution%of%this%paper’s%experiment%and%analysis.%
Section%3%introduces%a%Bayesian%framework%for%understanding%how%cooperation%can% be% a% Nash% equilibrium% of% a% stage% in% a% finitely% RPD% under% certain% conditions.% % This% model% informs%my%hypotheses,%which,%along%with%the%experimental%design%choices,%are%presented%in% Section%4.%
Section%5%presents%and%analyzes%the%full%results%of%the%experiment%both%in%relation%to% the% hypotheses% and% to% previous% findings.% % Section% 6% offers% concluding% remarks,% a% brief% discussion%of%limitations,%and%areas%for%future%research.%
%
2. Literature!Review% %
Since% the% advent% of% the% PD% by% Merrill% Flood% and% Melvin% Dresher% while% working% at% RAND% nearly% 70% years% ago,% it% has% become% perhaps% the% most% studied% game% in% economics% (Sally,%1995).%%This%is%because%a%PD%simply%models%a%strategic%situation%where%agents%have%to% take%into%account%their%beliefs%about%their%opponent’s%actions%when%deciding%their%own%best% actions.%%The%PD%becomes%more%complicated,%but%more%relevant%and%interesting,%when%it%is% examined%as%a%repeated,%rather%that%oneJshot,%game.%%In%infinitely%repeated%games,%the%Folk%
Theorem% states% that% a% cooperative% equilibrium% can% exist,% given% a% sufficient% discount% rate.%% However,% a% finitely% RPD,% like% that% used% in% this% paper’s% experiment,% has% only% a% subgame% perfect%Nash%equilibrium%(SPNE)%where%both%players%chose%to%defect%in%each%stage.%%This%is%a% troubling% result,% as% cooperation% in% every% stage% by% both% players% would% be% a% large% Pareto% improvement.% % % Thus% it% is% a% nonJtrivial% question% to% ask% how% this% improvement% can% be% attained.%%Andreoni%and%Samuelson%(2006),%as%discussed%in%the%Introduction,%provided%one%of% the%better%answers%to%date%to%this%question.%%However,%there%have%been%many%other%relevant% studies%that%have%attempted%to%answer%this,%or%a%closely%related,%question.%
Andreoni% and% Miller% (1993)% reported% that% behavior% in% laboratory% PD% experiments% cannot% be% explained% well% by% standard% theory.% % For% example,% many% players% chose% to% cooperate% in% a% oneJshot% PD% where% cooperation% is% a% strictly% dominated% action.% % % They% also% reported%that%in%finitely%RPDs,%although%rates%of%cooperation%decrease%over%all%stages%of%the% game,% they% never% fall% to% zero.% % Finally,% they% found% that% laboratory% PD% experiments% often% discover%that%agents%are%heterogeneous,%and%that%one%of%the%most%common%agent%types%is%the% “conditional% cooperator.”% % These% agents% “seem% to% undertake% significant% efforts% to% build% reputations% for% altruism”% (571).% % Andreoni% and% Miller% also% relayed% the% finding% of% psychologists%Kelley%and%Stahelski%(1970),%who%argued%that%there%exist%two%types%of%stable% individuals% in% these% games,% “described% approximately% as% cooperative% or% competitive% personalities”% (66).% % The% importance% of% the% existence% of% conditional% cooperators% is% that% wherever%they%exist,%there%is%an%opportunity%to%alter%the%conditions%on%which%they%base%their% decision%to%cooperate.%
Kreps% et% al.% (1982)% showed% that% in% an% RPD% where% there% is% uncertainty% about% the% “type”% of% an% agent’s% counterpart,% rational% cooperation% can% still% be% supported% as% an% equilibrium% of% a% stage.% % Simply,% if% δ% represents% the% probability% that% your% opponent% is% “irrational,”% or% in% Andreoni% and% Miller’s% terms,% a% “cooperative”% rather% than% “competitive”% personality,%then%a%nonJzero%δ%means%an%agent%will%believe%his%counterpart%will%not%always% defect.% % Instead,% the% counterpart% might% play% TFT% or% a% similar% conditionally% cooperative% strategy.%%When%δ%is%sufficiently%high,%cooperative%outcomes%can%therefore%rationally%occur,% because%an%agent%will%be%able%to%cooperate%while%minimizing%his%chances%of%earning%S%and% maximizing% his% chances% of% cooperating% in% subsequent% stages.% % In% their% analysis,% they% used% Kreps%and%Wilson’s%(1982)%concept%of%a%sequential%equilibrium.%%In%a%sequential%equilibrium,% each%agent%has%an%assessment,%which%consists%of%a%set%of%beliefs%for%every%information%set,% even%those%not%on%any%equilibrium%path,%and%an%action%at%each%information%set%conditioned%
on% those% beliefs.% % This% concept% allows% for% a% sequential% equilibrium% to% emerge% even% when% players%are%at%a%node%in%a%game%tree%that%had%a%prior%probability%of%zero.%%That%is,%if%players% were% arbitrarily% assigned% to% a% node% in% a% game% tree,% their% assessments% could% form% a% sequential% equilibrium% even% if,% prior% to% being% arbitrarily% assigned% this% node,% it% lay% off% the% equilibrium% path.% % As% Kreps% and% Wilson% said,% “a% sequential% equilibrium% provides% at% each% juncture%an%equilibrium%in%the%subgame%(of%incomplete%information)%induced%by%restarting% the%game%at%that%point”%(864).%%This%is%a%useful%concept%for%extensive%games%in%experiments,% as%participants%do%not%always%behave%according%to%the%predictions%of%SPNE,%and%thus%fall%off% an% equilibrium% path.% % Using% the% concept% of% a% sequential% equilibrium% can% allow% for% an% equilibrium% analysis% from% a% “this% point% forward”% perspective% in% an% extensive% game% with% incomplete%information.%%%
The% authors% also% showed% that% when% they% considered% all% nonJPareto% dominated% sequential% equilibria,% cooperation% will% occur% in% all% but% the% last% several% stages% of% the% game.%% This%is%an%important%result%that%has%an%impact%on%the%hypothesized%effectiveness%of%the%EB% and% SB% treatments% in% the% present% experiment.% % If% cooperation% is% likely% to% occur% in% early% stages%of%an%RPD,%but%unlikely%to%occur%in%the%last%several%stages,%the%marginal%effect%of%the% payoff%modification%to%R%is%likely%to%be%greater%in%the%EB%treatment%than%the%SB%treatment.%% One%reason%for%this%result%could%be%that%the%EB%treatment%alters%δ%–%Andreoni%and%Miller’s% parameter%for%the%probability%an%agent%assigns%to%his%counterpart’s%chance%of%cooperation–% in% the% final% few% stages% where% cooperation% is% usually% sparse.% % Given% altruistic% preferences% (discussed% below),% an% increase% in% δ% could% be% the% difference% in% a% cooperative% or% nonJ cooperative%outcome.%%%It%is%worth%noting%(as%did%Andreoni%and%Miller)%that%as%it%is%becoming% apparent% that% the% role% of% an% agent’s% beliefs% about% his% counterpart’s% type% may% be% a% driving% force%behind%achieving%cooperation%in%a%finitely%RPD,%these%beliefs%do%not%need%to%be%correct,% only%rationalizable.%%Cooperative%and%competitive%types%need%not%actually%exist;%agents%just% must%believe%they%do.%%%However,%evidence%–%like%that%discussed%above%–%suggests%they%do.%%% In%tandem%with%an%agent’s%beliefs%about%his%counterpart’s%type,%if%an%agent’s%utility% function%is%not%purely%selfish,%there%is%also%room%for%a%cooperative%outcome%in%a%finitely%RPD.%% Andreoni%and%Miller%summarize%the%three%main%ways%to%model%altruism%in%this%setting,%using% three%variations%of%the%“warm%glow”%model:% % (1) Pure%Altruism% !! = !!+ !!!!%
0! ≤ ! ≤ 1% % (2) Duty% !!= !!+ !!% ! ≥ 0%if%i%cooperates;%0%otherwise% % (3)%Reciprocal%Altruism%% !!= !!+ !!% ! ≥ 0%if%i%and%j'cooperate;%0%otherwise% % Pure%Altruism%(1)%and%Duty%(2)%can%support%cooperation%in%equilibrium%of%a%oneJshot%game%if% !%is%sufficiently%large.%%Reciprocal%Altruism%(3)%can%support%either%cooperation%or%defection% as% an% equilibrium% in% a% oneJshot% game% if%!! > ! − !.% % Equilibria% of% finitely% RPDs% are% more% complex.%%If%an%agent%plays%a%finitely%RPD%against%different%opponents,%it%is%likely%defection% will%occur%progressively%earlier%in%the%game%(Andreoni%and%Miller,%1993).%%But%if%players%play% against%the%same%opponent%in%a%repeated%game%(as%they%do%in%this%paper’s%experiment),%there% is%an%opportunity%for%altruistic%reputation%building.%%In%repeated%games,%if%these%reputations% (game% histories)% are% common% knowledge,% it% is% more% likely% that% a% cooperative% equilibrium% could%occur.%%However,%in%this%paper’s%experiment,%while%histories%are%common%knowledge,% agents% play% only% one% 10Jstage% RPD,% and% thus% only% have% access% to% the% history% of% the% game% they%are%actively%playing.%%There%is%therefore%no%way%for%cooperation%to%pervade%across%all% participants% (though% it% can% emerge% in% pairings),% as% it% would% have% the% chance% to% do% in% a% supergame.% % Thus,% for% this% experiment,% Reciprocal% Altruism% (3)% is% the% most% accurate% approach,% and% can% help% to% provide% an% explanation% for% incidences% of% cooperation,% but% does% not% help% in% making% predictions,% as%!%is% unobserved.% % However,% it% is% likely% that%!! > ! − !% early%in%the%SB%treatment%and%late%in%the%EB%treatment.%%A%point%in%favor%of%the%SB%treatment% leading%to%greater%cooperation%(though%I%do%not%make%this%prediction)%is%that%if%!! > ! − !%in% stages%1%–%3%of%the%SB%treatment,%agents%will%likely%cooperate%(if%their%utility%can%be%modeled% as%reciprocal%altruism).%%This%developing%history%of%cooperation%may%alter%an%agent’s%belief% about% his% counterpart’s% type% such% that% cooperation% may% continue% to% be% supported% as% an% equilibrium%even%as%payoffs%decrease.%%This%is%because%–%in%the%terminology%of%Kreps%et%al.%–%δ% is% sufficiently% large% (and% is% increasing% with% every% incidence% of% cooperation).% % However,% if%
cooperation% is% high% in% the% early% stages% of% both% treatments% anyway,% this% effect% is% likely% to% minimal%or%even%negligible.%
Harrington% (1987)% extends% the% work% of% Kreps% et% al.% (1982).% % While% Kreps% et% al.% argued%that%if%both%agents%are%rational%but%one%agent%is%not%completely%sure%the%other%agent% is%rational,%cooperation%can%emerge%in%a%finitely%RPD,%Harrington%focuses%on%the%concept%of% bounded% rationality.% % While% previous% work% had% focused% on% limiting% agents’% strategies% in% finitely%repeated%nonJcooperative%games%(e.g.,%Neyman,%1985),%here%Harrington%focuses%on% limiting% beliefs% over% strategies.% % He% shows% that% cooperation% can% emerge% in% games% of% these% types% if% agents% do% not% have% complicated% beliefs% about% their% counterpart’s% strategies% and% beliefs.% % Indeed,% in% a% 10Jstage% RPD% it% is% reasonable% to% assume% that% an% agent% believes% his% counterpart’s%beliefs%and%strategies%are%not%complicated.%%In%fact,%it%is%likely%they%are%binary%–% either%an%agent’s%counterpart%is%a%conditional%cooperator%or%a%competitive%player%who%likely% defects.%
To%this%point,%all%PDs%have%been%treated%equally%as%long%as%they%met%the%condition%of% Table%1%(T>R>P>S).%%However,%Peysakovich%and%Rand%(2013)%show%that%all%PDs%that%meet%this% condition% do% not% necessarily% lead% to% the% same% outcomes.% % In% a% twoJpart% experiment,% participants%played%an%RPD%of%stochastically%determined%length%in%part%one.%%In%the%first%stage% participants% were% divided% into% two% groups:% CJculture% and% DJculture,% where% the% difference% between%treatments%was%a%more%tempting%payoff%for%deviation%in%the%DJculture.%Specifically,% in%the%DJculture%group,%the%T%payoff%was%6,%while%in%the%CJculture%group,%the%T%payoff%was% only%5%(Figure%3).%%In%both%groups%the%R%payoff%was%4.%%Agents%played%one%stage%of%the%game% and%then%were%randomly%paired%with%a%new%partner%7/8%of%the%time%in%the%DJculture%group% and%1/8%of%the%time%in%the%CJCulture%group.%%These%conditions%were%created%with%the%goal%of% creating% social% heuristics% for% cooperation% and% defection% in% the% CJ% and% DJculture% groups,% respectively,%which%is%exactly%what%did%happen.%
% In% the% second% part% of% this% experiment,% players% from% both% cultures% played% various% oneJshot% games.% % Participants% from% the% CJculture% group% were% more% cooperative% and% more% willing%to%punish%those%who%were%uncooperative%in%oneJshot%games.%%But%the%most%pertinent% result%of%Peysakovich%and%Rand%is%that%a%small%change%in%the%relationship%between%T%and%R% had% a% large% impact% on% cooperation;% a% slight% increase% to% T% (along% with% the% differing% probabilities% of% counterpart% reassignment)% clearly% had% an% effect% on% the% amount% of% cooperative%outcomes.%%The%significant%difference%in%rates%of%cooperation%between%the%CJ%and% DJculture%groups%was%recognized%and%accounted%for%in%the%design%of%the%present%experiment;%
precisely%how%this%was%done%is%explained%further%in%the%experimental%design%section%(Section% 4).%
Rand% et% al.% (2014)% extends% and% formalizes% the% idea% of% social% heuristics.% The% social% heuristics%hypothesis%states%that%individuals%form%social%heuristics%after%repeated%everyday% interactions,% and% that% these% heuristics% are% often% optimally% cooperative.% % When% they% enter% into%a%new%situation%and%have%a%short%time%to%make%a%decision,%they%will%often%employ%these% heuristics,% matching% them% as% best% they% can% to% the% current% situation.% % However,% when% they% have%more%time%to%deliberate,%this%deliberation%may%lead%them%away%from%cooperation.%%In% Peysakovich% and% Rand% (2013),% the% authors% showed% how% social% heuristics% are% dynamically% updating,%such%that%strategies%used%in%repeated%play%in%CJ%and%DJculture%groups%carried%over% into% subsequent% oneJshot% games.% % This% suggests% two% things% for% the% present% study.% % One,% if% agents% are% bringing% cooperative% heuristics% into% the% laboratory% from% their% everyday% interactions,%the%early%stage%cooperation%rates%are%likely%to%be%high.%%Two,%there%may%be%a%link% between% the% SB% treatment,% where% cooperation% is% likely% in% the% first% three% stages,% and% the% formation% of% a% CJculture.% % These% stages% –% if% the% frequency% of% cooperation% is% indeed% high% –% could% possibly% reinforce% the% cooperative% social% heuristics% agents% are% bringing% into% the% laboratory%and%thus%result%in%continued%cooperation%beyond%the%initial%three%stages,%just%as% the% CJculture% in% Peysakovich% and% Rand% was% carried% over% into% the% subsequent% oneJshot% games.%%However,%if%the%EB%treatment%also%has%high%cooperation%in%the%first%few%stages,%the% increased%R%in%the%first%three%stages%of%the%SB%treatment%may%not%make%much%of%a%difference.%% This%is%only%a%conjecture,%however,%as%Peysakovich%and%Rand%only%examined%the% effects%of% part%one’s%CJ%and%DJ%culture%on%part%two’s%one@shot%games,%so%it%is%unclear%whether%the%three% early% stages% of% the% SB% treatment’s% possible% “CJculture”% could% be% enough% to% form% social% heuristics%that%agents%would%carry%with%them%into%the%latter%stages%of%the%RPD.%
However,% whether% or% not% participants% will% cooperate% equally% as% much% in% the% early% stages%of%both%treatments,%Fréchette%and%Yuksel%(2013)%show%that%cooperation%in%stage%1%of% an% infinitely% RPD% is% extremely% important% and% significantly% affects% whether% a% cooperative%
CJCulture%
!
DJCulture% % Cooperate% Defect% % ! Cooperate% Defect% Cooperate% 4,4% 0,5% % Cooperate% 4,4% 0,6% Defect% 5,0% 1,1% % Defect% 6,0% 1,1% ! Figure'4.'Peysakovich'and'Rand'(2013)'C@'and'D@culture'Payoff'Matrices'! !!
!
!
!
%
outcome%is%reached%as%far%off%as%stage%five.%%Thus,%history%of%the%game,%and%especially%of%the% first% stage,% has% a% significant% and% persistent% effect% on% future% play.% % Compounding% the% importance% of% this% result% is% that% in% their% randomJtermination% infinitely% RPD,% 71%% of% the% strategies% employed% by% agents% playing% the% game% are% Grim% or% TFT.% % Given% that% agents% play% Grim% and% TFT,% the% outcome% of% the% first% stage% is% of% extreme% importance.% % If% a% cooperative% outcome%is%not%reached%then,%it%may%never%be%reached.%%If%a%cooperative%outcome%is%reached,% it% may% pave% the% way% for% continued% uninterrupted% cooperation% (at% least% until% endJgame% defection%starts).%%
While% payoffs% changed% in% certain% stages% of% the% present% experiment% and% did% not% in% Fréchette%and%Yuksel,%this%does%not%disqualify%the%importance%of%their%results,%as%the%games% remain%very%similar.%%Also,%there%is%reason%to%believe%that%gradual%changes%to%stage%payoffs% may%not%cause%agents%to%change%their%behavior,%so%that%a%high%R%in%early%stages%could%initiate% a% series% of% cooperative% outcomes% that% persist% in% later% stages% even% when% R% payoffs% have% gradually%decreased%back%to%a%more%standard%level.%%Evidence%of%agents’%unresponsiveness%to% gradual%changes%in%game%conditions%came%in%part%from%Offerman%and%van%der%Veen%(2013),% who% ran% a% public% goods% game% experiment% with% a% subsidy% for% contributing% to% the% public% account.% % They% discovered% that% when% the% subsidy% was% large,% there% was% a% significant% difference% in% its% effect% on% contributions% when% it% was% introduced% at% once% as% opposed% to% gradually.%%When%the%large%subsidy%was%introduced%at%once,%more%agents%responded%to%the% change% by% taking% advantage% of% the% subsidy% and% contributing% more% to% the% public% account.%% When%the%large%subsidy%was%introduced%gradually,%significantly%fewer%agents%took%advantage% of%the%subsidy,%so%contributions%to%the%public%account%were%less.%%However,%if%an%agent%did% choose%to%respond%to%the%subsidy,%their%contribution%was%not%significantly%different%between% the% treatments% of% immediate% or% gradual% introduction.% % Their% results% did% not% hold% for% a% smaller% overall% subsidy.% % Yet,% their% study% does% lend% credence% to% the% theory% that% agents’% actions%may%be%insensitive%to%large%changes%in%R%payoffs%that%are%implemented%gradually.%%In% other% words,% a% statusJquo% bias% can% pervade% the% game.2%% If% this% is% the% case% in% the% present% experiment,% the% SB% treatment% would% make% sense.% It% would% coordinate% players% at% the% cooperative% outcome% and% then% the% gradual% decrease% in% payoffs% would% not% provoke% many% agents%to%deviate%from%cooperation.%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
2%Additional% evidence% of% the% idea% that% agents% may% not% respond% at% a% high% rate% to% small% changes% in% payoffs% comes% from% Schram% and% Sonnemans% (2011).% % In% a% laboratory% experiment,% they% had% participants% choose% health% policies% and% found% they% were% slower% than% optimal% in% adjusting% their% choices%when%their%health%deteriorated%gradually.%%
%
3. Theory! %
It%is%well%known%that%by%means%of%backward%induction%the%SPNE%outcome%in%every% stage% of% a% finitely% RPD% is% that% both% players% defect.% % However,% as% the% above% studies% have% shown,% thinking% only% in% terms% of% SPNE% is% overly% simplistic.% % The% SPNE’s% predictive% power% and%ability%to%model%experimental%behavior%in%finitely%RPDs%has%been%proven%questionable% from% the% very% beginning.%%Even%in% the%first%recorded% PD%experiment,%participants% chose% to% cooperate% 68% and% 78% times% over% 100% stages% (Sally,% 1995).% % And% in% a% finitely% RPD% with% uncertainty,% backwards% induction% can% lead% to% false% conclusions.3%% It% is% very% unlikely% that% every%player%will%choose%to%defect%in%every%period%of%the%present%experiment.%%As%such,%it%is% important%to%have%another%model%which%has%both%greater%predictive%and%descriptive%power% for%a%finitely%RPD.%%This%is%exactly%what%Li%and%Kendall%(2010)%offer.% % A'More'Realistic'Model' %
Li% and% Kendall% (2010)% introduce% a% new% Bayesian% model% that% better% models% and% predicts%behavior%in%a%finitely%RPD.%%To%do%this,%they%first%make%the%simplification%that%agents% either% play% strategies% Always% Defect% (AD)% or% TFT.% % This% seems% to% be% a% reasonable,% fairly% innocuous% assumption,% given% that% Fréchette% and% Yuksel% (2013)% found% that% in% random% termination%infinitely%RPDs,%84%%of%strategies%are%either%AD,%TFT,%or%Grim%Trigger.4%%Next,% they% assign% probability%1! ≥%!!! ≥ 0%that% agent% 1% (given% he% is% a% TFT% player)% chooses% to%
cooperate%in%stage%i.%%They%then%assume%that%(1)%!!%is%decreasing%in%stages%in%order%to%reflect% endJgame%behavior%and%(2)%that%if%agent%1’s%counterpart%chooses%to%defect%in%stage%i,%!!!!%=%0%
(which%is%true%by%definition%for%a%TFT%player).%%The%same%assumptions%apply%to%!!%for%agent%2% of% a% pairing.% % Both%!!!and%!!%are% not% fixed,% and% can% change% during% the% RPD.% % These% assumptions%allow%them%to%model%a%finitely%RPD%as%a%centipede%game.%%This%is%useful%because% it% clearly% shows% the% driving% force% for% cooperation% is% an% agent’s% belief% that% his% counterpart%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
3%Li%and%Kendall%described%the%Hangman’s%paradox%to%illustrate%the%shortfall%of%backwards%induction% in%a%situation%with%uncertainty.%%The%paradox%goes%like%this:%a%prisoner%is%told%he%will%be%executed% during%the%next%week%(Mon.%to%Fri.),%but%the%exact%day%will%be%a%surprise.%%By%backwards%induction,%he% knows%it%won’t%be%Friday,%because%if%he%were%not%hung%by%Thursday%night,%he%would%know%it%would%be% Friday%and%thus%it%wouldn’t%be%a%surprise.%%However%it%can’t%be%Thursday%for%the%same%reason.%%By%this% reasoning,%he%will%never%be%executed.%(But%alas,%he%is%hung,%and%it%was%a%surprise,%due%to%his%faulty% reasoning.)% 4%Bó%and%Fréchette%(2013)%found%80%%for%the%same%three%strategies.%will%cooperate%(as%in%a%centipede%game%where%an%agent%must%essentially%choose%his%action% based% on% the% level% of% trust% he% has,% at% that% moment,% in% his% opponent).% % At% each% node,% the% decision% of% an% agent% to% cooperate% is% based% on% whether% he% believes% his% counterpart% will% cooperate% and% the% expected% future% profit% of% cooperation.% Using% backwards% induction% to% determine% the% SPE% of% this% game% is% counterproductive.% % It% can% lead% to% false% conclusions% because%strategies%are%dependent%on%beliefs%that%are%formed%in%the%previous%periods.%%The% real% reason% the% amount% of% cooperation% decreases% in% stages,% according% to% Li% and% Kendall’s% model,%is%the%expected%payoff%to%each%agent%is%decreasing.%%%With%this%model,%any%finitely%RPD% can%be%reduced%to%two%stages%–%the%current%stage%and%the%stage%immediately%following%the% current%stage.%% Some%basic%algebra%(see%Li%and%Kendell%for%details)%leads%to%the%following%conditions% for%cooperation%to%be%a%Nash%equilibrium%(NE)%in%stage%i%of%a%finitely%RPD:% % 1 !!! !!!!! + 1 − ! !!!!+ 1 ! + ! − ! − ! > !! − !% (2)%!! !!!!! + 1 − ! !!!!+ 1 ! + ! − ! − ! > !! − !% % These%equations%will%be%shown%to%be%very%useful%in%making%theoretical%predictions%for%the% behavior% of% agents% in% an% RPD.% % For% example,% values% for% P,% R,% T,% and% S% from% the% present% experiment%can%be%inserted%in%equation%(1)%for%both%the%EB%and%SB%treatments%(see%Table%1% and%explanation%below).%%%
If,% as% Li% and% Kendell% maintain,% cooperation% is% driven% by% expected% payoffs% to% cooperation,% then% as% the% number% of% stages% decreases,% the% amount% of% cooperation% will% decrease.%%This%is%because%the%expected%payoff%decreases%with%time%(as%the%number%of%stages% is% finite).% % But% this% reliance% of% cooperation% on% the% expected% payoff% to% cooperation% gives% an% opportunity% to% increase% cooperation% by% increasing% later% stages’% R,% thereby% increasing% the% expected%payoff%to%cooperation.%%This%is%exactly%what%the%EB%treatment%does.%Meanwhile,%in% the% SB% treatment,% R% is% increased% when% the% expected% value% to% cooperation% is% already% high% enough%to%support%a%cooperative%NE%in%a%stage,%so%the%marginal%effect%would%likely%be%less,% and%would%likely%entirely%disappear%after%the%first%three%stages.%
Table% 1% shows% how% this% model% can% be% used% to% show% that% the% EB% treatment% should% have%more%cooperative%outcomes%than%the%SB%treatment.%%It%takes%inequality%(1)%from%above% and%demonstrates%how%this%inequality%can%support%a%NE%for%a%stage%of%the%game%(with%the% assumption% that% inequality% (2)% for% agent% 1’s% counterpart% also% meets% these% criteria).%% Arbitrary%decreasing%values%of%δ%(0.9,%0.8,%0.7,%…,%0.25)%are%plugged%into%the%leftJhand%side%of% inequality%(1)%and%the%result%is%outputted%in%the%EQ%Value%column.%%These%δ%values%are%the% subjective%probabilities,%implicitly%formed%by%an%agent,%given%he%or%she%is%a%TFT%player,%that% he%or%she%will%cooperate%in%the%stage%the%subscript%indicates.%%The%importance%of%this%table%is% that% it% shows% how% the% EB% treatment% can% be% superior% to% the% SB% treatment% with% only% a% few% simple% assumptions.% First,% assume% that,% as% suggested% in% the% literature,% cooperation% decreases%in%stages%of%a%finitely%RPD.%%Second,%assume%that%δ%starts%at%a%high%value%in%both% treatments,% as% previous% literature% shows% that% initial% cooperation% in% RPDs% is% typically% high.%% For%the%equilibrium%condition%(inequality%(1))%to%support%a%NE%in%a%stage,%the%leftJhand%side% (the%“EQ%Value”)%must%be%greater%than%the%rightJhand%side,%PJS%=%28.%%It%can%be%seen%that%as%δ% decreases%in%stages,%the%EQ%Value%approaches%the%threshold%of%28.%%When%stage%7%is%reached,% using%these%arbitrary%values%of%δ,%the%EQ%Value%barely%exceeds%the%threshold%(28.08%>%28).%% However,%this%is%the%last%stage%before%the%increased%R%payoffs%of%the%EB%treatment%kick%in.%%In% stage% 8,% R% payoffs% increase% from% 98% to% 112,% and,% as% inequality% (1)% shows,% the% EQ% Value% is%
δ% If%player%is%a%TFT%player,%probability%that%he/she%cooperates%at%given% stage% PJS% EQ%Value% EQ%Value%>%PJS% δ1% 0.9% 28% 68.04% EQ% δ2% 0.8% 28% 51.52% EQ% δ3% 0.7% 28% 41.16% EQ% δ4% 0.65% 28% 36.76% EQ% δ5% 0.63% 28% 34.22% EQ% δ6% 0.61% 28% 31.77% EQ% δ7% 0.59% 28% 28.08% EQ% δ8% 0.55% 28% 30.80% EQ% δ9% 0.5% 28% 21.00% Not%EQ% δ10% 0.25% 28% 7.00% Not%EQ%
%
Table'1.'Hypothetical'Values'for'Equation'(1)'in'the'EB'Treatment'' A'decreasing'sequence'of'values'for'δi'were'arbitrarily'chosen'to'demonstrate'how'cooperation'can' be'a'NE'of'a'stage'in'a'finitely'RPD.''This'table'reflects'the'EB'treatment.''Notice'that'the'EQ'value' increases'from'stage'7'to'stage'8'to'remain'above'P@S'='28'because'R'increases'from'98'to'112.'dependent%on%R.%%Thus,%even%though%δ%decreases%from%stage%7%to%stage%8%(from%0.59%to%0.55),% and%!!!! = !!= 0.5,% the% EQ% Value% increases% (from% 28.08% to% 30.80)% and% stays% above% the% threshold% (30.80% >% 28).% % This,% while% merely% a% demonstration% with% arbitrary% values% of% δ,% clearly%shows%how,%even%assuming%a%decreasing%δ,%the%EB%treatment%can%support%a%NE%in%a% later%stage%of%the%game.%%This%equilibrium%would%very%likely%not%exist%in%the%SB%treatment,% because%R%does%not%increase%in%stages%8J10%of%the%EB%treatment.% % Conversely,%it%could%be%argued%that%raising%R%in%early%stages%might%cause%a%positive% shock%in%δ,%increasing%its%value%before%its%subsequent%decline%through%the%remaining%stages.%% However,%if%δ%is%already%assumed%to%be%high%in%both%treatments%in%stage%1,%there%is%not%much% room% for% the% SB% treatment% to% increase% δ,% as% it% is% a% probability% and% thus% bounded% by% 1.%% However,%as%δ%decreases%in%stages,%there%is%much%more%room%for%an%increase%in%later%stages% when%the%value%of%δ%is%lower.%%%Therefore,%this%objection%does%not%seem%very%powerful.%%
The%main%insight%of%this%model,%as%demonstrated%in%Table%1,%is%that%it%is%theoretically% possible% to% have% TFT% players% reach% a% cooperative% NE% more% easily% in% a% later% stage% of% a% 10J stage%RPD%in%the%EB%treatment%than%in%the%SB%treatment.%%Additionally,%in%the%framework%of% this%model,%the%increased%early%stage%R%payoffs%of%the%SB%treatment%would%have%likely%had%a% less% significant% effect% on% whether% a% cooperative% NE% is% reached% because% the% EQ% Value% is% already%well%above%the%threshold%value%of%28.%%It%is%this%model%and%predictions%that%inform% my% hypotheses,% which% are% elaborated% in% the% following% section,% after% descriptions% of% the% experimental%procedure%and%design.% % 4. Experimental!Design!and!Hypotheses! % Experimental'Procedure' %
The% experiment% consisted% of% two% treatments,% one% of% which% was% conducted% in% a% morning% session% (11% am),% and% a% second% which% was% conducted% in% an% afternoon% session% (2% pm).% % Each% treatment% lasted% less% than% 20% minutes% and% consisted% of% a% 10Jstage% RPD% where% participants% were% randomly% paired% with% another% participant% with% whom% they% played% the% entire% game.% % Different% subjects% participated% in% each% session.% % Both% sessions% had% 24% participants,%meaning%there%was%a%total%of%12%pairings%in%each%treatment.%%Summary%statistics% for%the%participants%are%provided%in%Table%2.%
'
This% experiment% was% part% of% an% amalgamation% of% University% of% Amsterdam% masters% students’% thesis% experiments.% % In% each% session% there% were% seven% experiments% of% varying% lengths% such% that% each%session%lasted%approximately%two%hours.%%This%experiment%was%the%second%experiment% to%be%run%in%each%session,%following%a%public%goods%game%(PGG)%experiment.%%It%is%possible% that% the% PGG% experiment% may% have% had% a% spillover% effect% on% this% experiment,% as% it% was% designed% to% publicly% rank% players% based% on% their% contributions% to% a% public% goods% account.%% This% gave% information% to% the% participants% about% the% general% cooperative% or% altruistic% preferences%of%the%other%players,%which%could%be%useful%information%to%employ%when%playing% an% RPD.%%However,%this% potential% spillover% was%unavoidable% and% impossible% to%measure% in% this%context,%and%it%is%not%obvious%ex%ante%that%the%PGG%should%have%an%effect%on%the%present% experiment.% % Additionally,% because% the% PGG% experiment% was% performed% prior% to% both% treatments,%if%it%did%have%an%effect,%it%should%have%affected%both%treatments%relatively%equally.% During%the%experiments,%each%participant%sat%at%his%or%her%own%computer%positioned% inside% a% cubicle.% Their% only% interaction% with% other% participants% was% through% the% computer% network% as% part% of% each% experiment.% % The% instructions% were% presented% to% the% participants% only% after% all% participants% had% completed% the% previous% experiment.% % In% order% to% ensure% all% experiments% were% run% in% a% uniform% and% expedient% manner,% master’s% students% were% instructed% by% the% experimental% coordinator% not% to% read% their% instructions% out% loud.% % Thus,% participants% read% the% instructions% at% their% own% pace% and% had% to% correctly% answer% two% comprehension% questions% before% advancing% to% the% 10Jstage% RPD.% % The% instructions% (see% appendix)%clearly%explained%the%changes%in%R.%%Participants%in%Session%1%were%informed%only% of% the% EB% treatment% and% those% in% Session% 2% were% informed% only% of% the% SB% treatment.%% Additionally,%a%printed%copy%of%the%instructions%was%given%to%each%participant%for%reference% during% the% game,% if% desired,% in% order% to% recall% how% the% R% payoffs% changed.% % In% each% stage,% agents%had%a%choice%between%two%actions:%A%and%B.%%A%corresponded%to%cooperation%and%B%to% % Avg.% Age% Age% Min% Age% Max% %% Male% %% Dutch% GT% %% Session%1%(EB)% 23.79% 20% 39% 75%% 96%% 54%% Session%2%(SB)% 26.13% 19% 58% 58%% 83%% 46%%
%
Table'2.'Summary'statistics'for'experimental'sessions,'obtained'via' questionnaire'(Dutch'='%'of'participants'who'identified'as'Dutch;'GT'='%'of' participants'who'had'taken'a'course'in'game'theory)'defection,% but% there% was% no% mention% of% cooperation% or% defection% in% order% to% keep% the% experiment%in%a%neutral%frame.%
%
Experimental'Design'
%
The% general% question% this% experiment% seeks% to% answer% is% whether% it% is% better% to% frontJload% or% backJload% the% R% payoff% in% a% 10Jstage% RPD% in% order% to% achieve% the% greatest% frequency% of% cooperation.% % While% this% experiment% is,% roughly% speaking,% an% extension% of% Andreoni%and%Samuelson%(2006),%there%is%no%simple%or%straightforward%way%to%extend%their% design%to%a%10Jstage%game%without%changing%the%strategic%nature%of%the%game.%%Andreoni%and% Samuelson% chose% a% twiceJplayed% PD% was% because% it% allowed% for% the% game% to% be% modeled% mathematically.%%More%than%twoJstages%results%in%dynamics%that%are%extremely%complicated% as%well%as%more%difficult%choices%in%terms%of%game%structure.%%Thus,%to%answer% the%present% research%question%while%generally%following%in%the%wake%of%Andreoni%and%Samuelson,%I%had% to%make%the%following%decisions:%(1)%the%length%of%the%game;%(2)%the%number%of%periods%in% which% R% is% modified;% (3)% if% R% is% changed% in% more% than% one% stage,% whether% it% should% be% increased%the%same%amount%in%each%stage%or%gradually%increased%or%decreased;%(4)%whether% the% game% should% be% finitely% or% infinitely% repeated;% and% (5)% the% value% of% the% R,% T,% S,% and% P% payoffs.%%The%rationale%for%these%decisions%is%explained%below.%
% First,% I% chose% a% 10Jstage% game% because% of% time% constraints% (20% minutes% per% experiment)%set%by%the%experimental%coordinator.%%Secondly,%I%chose%to%modify%R%payoffs%in% three%stages%of%the%RPD%for%several%reasons.%%One,%changing%only%the%first%or%the%last%period% may% have% been% too% insignificant% to% see% any% meaningful% differences% in% behavior% from% a% standard% game% with% static% payoffs.% % The% effect% of% a% change% in% only% one% of% 10% stages% would% likely% be% isolated% –% players% may% have% cooperated% in% stage% 1% of% the% SB% treatment,% and% immediately%have%gone%back%to%playing%as%if%that%stage%did%not%exist.%%Conversely,%to%extend% Andreoni%and%Samuelson’s%design%more%literally%to%a%10Jstage%RPD%by%borrowing%from%the% last%five%stages%to%increase%R%in%the%first%five,%or%vice%versa,%would%not%have%made%the%game% significantly% different% strategically% from% Andreoni% and% Sameulson’s% twiceJplayed% game.%% Instead%it%would%have%been%more%of%a%test%of%social%heuristic%formation%in%the%five%repeated% stages% where% R% was% higher% or% lower,% which% is% not% what% this% experiment% seeks% to% test.%% Additionally,% there% would% have% been% a% sharp% dropJoff% between% stages% 5% and% 6% unless% the% payoffs% were% gradually% decreased.% % Thus,% modifying% payoffs% in% three% stages% is% a% fruitful%
middle%ground%that%is%related%but%not%equivalent%to%Andreoni%and%Samuelson’s%experiment% and% that% modifies% R% in% enough% stages% to% potentially% allow% for% the% potential% formation% of% a% social%heuristic%of%cooperation%in%those%modified%stages%in%the%SB%treatment%that%could,%via% the%statusJquo%bias%or%for%other%reasons,%carry%over%to%the%seven%unmodified%stages.%
% Similarly,% I% chose% to% have% a% gradual% decrease% and% increase% in% the% R% payoffs% of% the% three%modified%stages%of%the%SB%and%EB%treatments,%respectively.%%As%discussed%in%Section%2,% gradual% changes% in% the% R% payoff% may% not% elicit% a% change% in% strategy% (likely% a% change% from% cooperation% to% defection)% by% some% participants.% % Therefore,% this% design% choice% was% in% harmony%with%a%normative%goal%of%this%experiment%–%to%create%conditions%where%cooperation% has%a%greater%chance%of%occurring%naturally.%%
% I% chose% a% finitely% repeated% game% instead% of% an% infinitely% repeated% game% (with% a% continuation% probability)% for% purposes% of% both% simplicity% and% to% ensure% that% all% agents% in% both%treatments%would%play%the%same%number%of%stages.%%This%was%important%given%the%small% sample%size.%%If%the%number%of%participants%was%larger,%it%would%have%been%more%feasible%(and% preferable,%because%of%the%increased%external%validity%of%infinitely%RPDs)%to%use%an%infinitely% repeated%game,%because%the%expected%number%of%stages%in%each%treatment%would%approach% the%same%value.% Finally,%the%specific%values%of%T,%R,%P,%and%S%were%chosen%because%the%expected%payoff% per% participant% needed% to% average% around% 10% euros% and% so% that% the% difference% between% T% and%R%was%as%neutral%as%possible.%%Peysakovich%and%Rand%(2013)%showed%that%a%“CJculture”% and%“DJculture”%can%emerge%in%part%depending%on%the%temptation%of%the%T%payoff.%%I%measured% the%temptation%of%the%T%payoff%as%the%ratio%of%R%to%T.%%In%Peysakovich%and%Rand,%the%CJculture% group%had%R=4%and%T=6%and%the%DJculture%group%had%R=4%and%D=5.%%Thus,%!
! = !0.8%for%the%CJ
culture% group% and%!
! = 0.667%for% the% DJculture% group.% % I% reasoned% that% keeping% the% ! !%ratio%
between%these%two%values%(that%is,%between%0.667%and%0.8)%would%help%to%ensure,%as%much%as% possible,% a% more% neutral% setting.% % If%!!%is% too% close% to% either% 0.667% or% 0.8,% the% effect% of% the% formation% of% a% CJ% or% DJculture% could% overpower% the% effect% of% the% treatments% that% this% experiment%is%trying%to%determine.% Thus,%for%all%stages,%I%chose%an%overall%! !%ratio%of% !! !!!" !!! !! !!!" !!! = !0.76,%while%for% the%baseline%stages%(stages%with%an%unmodified%R%payoff),%I%chose%! != ! 98 140%=%0.7.%%While%
stages%are%played%in%the%baseline%stage,%where%the%R/T%ratio%(0.7)%is%closer%to%the%DJculture% (0.667).% % It% is% very% difficult% to% create% a% completely% neutral% setting% between% the% CJ% and% DJ cultures,%but%it%can%be%said%with%reasonable%certainty%that%the%!!%ratio%used%in%this%experiment% does%not%create%a%CJ%or%DJculture.%
%
Hypotheses' '
J H0,1:! There! will! be! no! significant! difference! in! the! overall! frequency! of! cooperation!between!the!EB!and!SB!treatments.!
o I%believe%that%Li%and%Kendall’s%model%has%the%most%predictive%power%for%this% experiment.% % There% is% reason% to% believe,% like% in% many% previous% RPD% experiments,%that%cooperation%in%the%early%stages%will%be%high%even%without% any% increases% in% R% payoffs,% but% that% endJgame% defection% will% be% common.%% With%these%beliefs,%the%model%predicts%that%increasing%R%in%the%later%stages,%by% increasing%the%EQ%Value%in%Table%1,%is%more%likely%to%create%additional%NE%that% support% cooperation% in% the% later% stages% that% wouldn’t% exist% if% R% was% only% increased%in%the%early%stages.%%Thus,%I%believe%that%enough%participants%in%the% EB% treatment% will% coordinate% at% these% cooperative% NE% in% order% to% make% a% significant% difference% between% the% treatments% in% the% frequency% of% cooperative%outcomes.%%Therefore,%I%predict%that%I%will%be%able%to%reject%H0,1%at% the%.05%level.% J H0,2:!There!will!be!no!significant!difference!in!the!frequency!of!cooperation!in! the!first!three!stages!of!the!SB!treatment!compared!to!the!first!three!stages!of! the!EB!treatment.! o While%it%has%been%discussed%above%that%there%is%good%reason%to%believe%there% should%be%a%high%level%of%early%stage%cooperation%in%both%treatments,%I%believe% that%it%will%still%be%significantly%greater%in%the%SB%treatment.%%Given%that%stage% 1%of%the%SB%treatment%is%not%a%PD%as%R=T=140,%it%is%very%likely%this%will%lead%to% very% high% rates% of% cooperation% in% this% stage.% % While% agents% may% eventually% move% away% from% cooperation% as% payoffs% decrease,% I% believe% that% this% defection%behavior%will%likely%begin%after%the%increased%R%payoffs%have%ended% in%stage%4.%%Therefore,%I%predict%that%I%will%be%able%to%reject%H0,2%at%the%.05%level.!
J H0,3:!There!will!no!significant!difference!in!the!frequency!of!cooperation!in!the! final!three!stages!of!the!EB!treatment!compared!to!the!final!three!stages!of!the! SB!treatment.!!
o I%believe%there%will%be%endJgame%defection%behavior%in%both%treatments,%but% the% increased% payoffs% in% the% final% three% stages% of% the% EB% treatment% will% mitigate%the%effects%of%this%behavior%significantly.%%This%belief%can%be%justified% by%Li%and%Kendall’s%model%and%explanation%surrounding%Table%1.%%Therefore,%I% predict%that%I%will%be%able%to%reject%H0,3%at%the%.05%level.! % 5. Results!and!Analysis! ! This%section%provides%an%overview%of%the%results%of%the%experiment,%first%in%regard%to% the%specific%hypotheses%stated%above%and%subsequently%in%regard%to%secondary%results%and% interesting%findings%that%are%related%to%the%general%principles%of%this%paper.%
%
Figure'5.'Frequency'of'cooperative'outcomes'by'stage.'In% order% to% test% for% a% statistically% significant% difference% between% instances% of% cooperation%between%treatments,%a%chiJsquared%test%was%used.%%The%outcomes%of%each%stage% were% made% into% categorical% variables:% that% is,% 1% if% the% cooperative% (C,C)% outcome% was%
%
%
%
Figure'6.'Outcomes'of'all'stages'for'all'24'pairings'across'both'treatments.''Green'=' (C,C)'outcome,'blue'='(C,D)'or'(D,C)'outcome,'red'='(D,D)'outcome.''A'line'connecting'reached% and% 0% otherwise,% as% other% categories% of% outcomes% were% irrelevant% for% the% main% analysis%of%this%paper.% % ' Main'Results'and'Analysis' ' H0,1:!There!will!be!no!difference!in!the!overall!frequency!of!cooperation!between!the! EB!and!SB!treatments! !%
There% was% no% statistically% significant% difference% in% the% number% of% cooperative% outcomes%between%the%EB%and%SB%treatment%at%the%.05%level%(p=0.056).%%However,%there%was% significantly% more% cooperation% in% the% EB% treatment% at% the% .10% level.% If% it% were% not% for% the% strange%behavior%of%one%of%the%players%in%pairing%9%of%the%EB%treatment%who%defected%in%the% first%9%of%the%10%stages,%there%would%certainly%be%a%significant%difference%at%the%.05%level%as% well.%%This%player%who%defected%in%stages%1J9%and%cooperated%in%stage%10%was%a%20%yearJold% % % % Treatment' % Hypothesis% Tested% Stages% % EB% SB% Conclusion% H0,1% St ag es '1 @10 ' Cooperative%Outcomes% 101% 89% Fail%to%reject% H0,1%at%.05% level% Observations% 120% 120% Pearson%chi2% 3.6379% PJvalue% 0.056% H0,2% St ag es '1 @3 ' Cooperative%Outcomes% 30% 35% Reject%H0,2% at%.05%level% Observations% 36% 36% Pearson%chi2% 3.9560% PJvalue% 0.047% H0,3% St ag es '8 @10 ' Cooperative%Outcomes% 27% 18% Reject%H0,3% at%.05%level% Observations% 36% 36% Pearson%chi2% 4.8000% PJvalue% 0.028% None% EB '1 @7' vs. 'S B ' 4@ 10 ' Cooperative%Outcomes% 74% 54% Difference%is% significant% at%.05%level% Observations% 84% 84% Pearson%chi2% 13.1250% PJvalue% <0.001%
!
Table'3.'Chi@squared'significance'tests'for'a'difference'in'cooperative'outcomes'between'treatments.'!
!
!
!
!
%
Dutch%female%economics%student%who%had%taken%game%theory.%%She%may%have%been%trying%to% use% her% knowledge% that,% in% a% typical% RPD,% defection% is% a% dominant% strategy% in% every% stage.%% But%by%her%apparent%logic,%defection%would%still%be%weakly%dominant%in%the%final%stage,%so%it%is% unclear% why% she% chose% to% switch% to% play% C% in% the% final% stage.% % Either% way,% her% behavior% is% somewhat% of% an% anomaly,% and% if% she% were% replaced% with% an% average% player% from% either% treatment,% there% would% be% significantly% more% cooperation% in% the% EB% treatment% at% the% .05% level.%
% Thus,%I%fail%to%reject%H0,1%at%the%.05%level,%but%it%should%be%noted%that%it%can%be%accepted% at%the%.10%level.%
%
H0,2:! There! will! be! no! difference! in! the! frequency! of! cooperation! in! the! first! three! stages!of!the!SB!treatment!compared!to!the!first!three!stages!of!the!EB!treatment.! %
% H0,2%basically%states%that%the%increased%payoffs%in%the%first%three%stages%of%the%game% matter% in% terms% of% inducing% greater% cooperation.% %H0,2% can% be% accepted% at% the% .05% level% (p=0.046).% % 35% of% 36% outcomes% over% stages% 1J3% of% the% SB% treatment% were% (C,C).% % In% the% EB% treatment,%30%of%36%outcomes%over%stages%1J3%were%(C,C).% % H0,3:!There!will!no!difference!in!the!frequency!of!cooperation!in!the!final!three!stages! of!the!EB!treatment!compared!to!the!final!three!stages!of!the!SB!treatment.!! % H0,3,%like%H0,2,%hypothesizes%that%the%effect%of%the%three%stages%of%increased%R%payoffs% matters.% % In% stages% 8J10,% there% was% significantly% greater% cooperation% in% the% EB% treatment% compared%with%the%same%stages%of%the%SB%treatment%(p=0.028),%thus%H0,3%can%be%accepted%at% the% .05% level.% % In% fact,% the% difference% was% quite% large:% 28% out% of% the% 36% outcomes% in%the% EB% treatment%were%(C,C),%while%only%17%of%the%36%outcomes%in%the%SB%treatment%were%(C,C)%over% the%same%final%three%stages.%%%
%
Other'Results%
%
% It% is% also% worthwhile% to% compare% a% few% other% sequences% of% stages% between% treatments.%%Comparing%stages%4J10%of%the%SB%treatment%with%stages%1J7%of%the%EB%treatment% (the%stages%in%each%treatment%where%R=98)%shows%there%is%a%significant%difference%between%
these% groupings% in% level% of% cooperation% (p<0.001).% % For% these% intervals,% (C,C)% was% the% outcome% 74% of% 84% times% in% the% EB% treatment% while% in% the% SB% treatment,% (C,C)% was% the% outcome% only% 54% of% 74% times.% % These% results% suggest% that% the% effect% of% a% future% higher% R% payoff%in%the%EB%treatment%is%greater%than%any%carryJover%or%statusJquo%bias%that%may%exist% after%the%higher%rates%of%cooperation%in%the%first%three%stages%of%the%SB%treatment.%%This%result% agrees%with%Li%and%Kendall’s%model,%which%says%that%cooperation%is%highly%dependent%on%the% expected%profit%from%cooperation.%%Due%to%the%increased%R%payoffs%in%the%final%three%stages%of% the%EB%treatment,%the%expected%value%of%future%cooperation%is%higher%in%this%treatment%after% stage%1%and%especially%after%stage%3.%%This%result%provides%further%evidence%that%saving%the% higher% payoffs% for% later% in% the% game% contributes% to% higher% levels% of% the% cooperation% in% the% early%to%intermediate%stages.%
% Another%interesting%finding%was%the%irreversibility%of%even%one%choice%of%defection.%% Of% the% 24% pairings% across% both% treatments,% only% once% was% a% cooperative% outcome% reached% once%either%agent%chose%action%D.%%This%indicates%that%once%an%agent%chooses%to%defect,%it%is% exceedingly%difficult%to%ever%return%to%a%cooperative%outcome.%%In%the%pairings%where%(D,D)% was%reached,%it%was%often%the%case%that%one%player%chose%to%defect%and%his%or%her%counterpart% played%some%variation%of%TFT%or%Grim%Trigger,%and%only%rarely%with%any%forgiveness.%%(For% the%full%results%with%both%players’%actions%in%each%stage,%see%Table%A1%in%appendix).%%Often,% once%one%agent%chose%to%defect,%his%or%her%counterpart%executed%the%Grim%Trigger%strategy% immediately,% and% even% when% the% agent% who% initially% defected% tried% to% signal% a% desire% to% cooperate%by%playing%C%again,%his%or%her%counterpart%did%not%stop%defecting.%%This%indicates% that% it% is% very% difficult% to% overcome% a% reputation% for% defection% and% reJestablish% trust,% especially%in%a%relatively%short%game.%%
For% instance,% 3% of% 12% pairings% in% the% SB% treatment% did% not% have% a% cooperative% outcome%in%the%remaining%6J7%stages%after%one%agent%chose%to%defect%in%the%3rd%or%4th%stage% (see%Figure%6).%%%This%highlights%one%of%the%major%flaws%of%the%SB%treatment:%once%defection% occurs,%it%nearly%always%leads%to%nonJcooperative%outcomes%for%the%remainder%of%the%game.%% It%is%likely%this%occurs%because,%without%the%increased%R%payoffs%of%the%EB%treatment,%players% lack% the% belief% their% opponent% will% cooperate% at% all% in% later% stages.% % Meanwhile,% in% the% EB% treatment,%a%(C,C)%outcome%was%reached%after%a%(D,D)%outcome%on%one%occasion%–%the%only% occasion%in%either%treatment.%%It%is%likely%this%occurred%because%both%agents%held%the%rational% belief%the%other%would%eventually%want%to%cooperate%in%order%to%capture%the%larger%R%payoff% in%the%final%stages.%%Meanwhile,%in%the%SB%treatment,%the%decreasing%payoffs%between%stages%3%
and%4%seemed%to%be%interpreted%by%some%players%as%a%nudge%to%change%their%behavior%away% from%cooperation.% Another%important%overall%observation%is%the%high%overall%frequency%of%cooperation.%% I%expected%initial%levels%of%cooperation%would%be%high,%but%not%as%high%as%they%were;%it%seems% as%if%cooperation%was%the%default%action%in%stage%1.%%In%fact,%12%of%12%pairings%in%SB%and%10%of% 12%pairings%in%EB%reached%a%(C,C)%outcome%in%this%initial%stage.%%Thus,%the%SB%treatment%may% have% been% detrimental% to% overall% cooperation% because% the% marginal% increase% from% EB% treatment%(standard%payoffs%in%stages%1J7)%in%stage%1%was%small%(two%additional%instances%of% cooperation),% but% in% stages% 3% and% 4,% three% pairings% from% the% SB% treatment% reached% nonJ cooperative%outcomes%for%the%first%time,%while%no%new%pairings%in%the%EB%treatment%reached% a% nonJcooperative% outcome% for% the% first% time% in% this% same% interval.% % Perhaps% the% simplest% reason% for% the% observed% frequency% of% cooperation% was% the% choice% of%!
!%ratio.% % It% may% have%
been%that%the%ratio%was%not%small%enough%–%the%temptation%of%T%not%great%enough%–%to%make% defection%an%enticing%enough%option.%%Also,%it%might%have%been%that%the%participants%had%an% intrinsic%desire%to%cooperate,%more%so%than%in%past%experiments.%%This%is%especially%possible% given%the%same%sample%of%only%48%participants.% As%mentioned%previously,%the%high%levels%of%cooperation%could%have%been%a%spillover% effect%from%the%PGG%experiment%immediately%preceding%the%present%experiment.%%This%could% especially% be% true% because% contributions% to% the% public% account% were% sometimes% made% common%knowledge,%and%other%times%top%contributors%to%the%public%account%were%given%the% designation% of% “star% players.”% % In% the% PPG% experiment’s% questionnaire,% most% players% answered%that%they%had%a%positive%view%of%the%star%contributors.%%Admiration%for%the%most% selfless% and% cooperative% players% in% this% game% is% likely% a% signal% that% these% participants,% on% average,% placed% a% relatively% high% value% on% cooperation.% % While% it% is% unlikely% that% this% experiment% had% a% large% effect,% or% affected% each% treatment% of% the% present% experiment% unevenly,% it% is% possible% that% it% primed% participants% in% this% experiment% to% behave% more% cooperatively%than%they%otherwise%would%have.%%Unfortunately,%there%is%no%way%to%measure% this%effect.%
% On%the%other%hand,%there%could%have%been%some%behavioral%reasons%for%defection%as% well.% % Following% the% PGG% experiment,% the% present% experiment% required% less% choice% and% reconsideration% of% strategy% during% each% stage.% % A% knowledgeable% player% could% decide% in% stage%1%to%play%TFT,%AD,%or%some%simple%variation,%and%not%update%their%strategy.%%This%leaves% less%for%the%participants%to%do.%%Thus,%some%of%the%defections%that%seem%more%irrational%could%
simply%be%the%result%of%agents%feeling%the%need%to%do%something%different%than%cooperate%in% each%stage.%%Agents%may%have%felt%there%was%some%optimal%strategy%that%was%not%the%strategy% of%cooperating%in%every%stage.%%This%is%not%an%uncommon%occurrence%in%experiments%where% participants% –% sometimes% altruistically,% trying% to% “help”% the% experimenter% –% modify% their% behavior% when% they% feel% they% should.% % Here,% that% could% be% true% if% participants% felt% the% experiment% was% not% designed% with% the% intention% of% them% simply% choosing% to% cooperate% in% each% stage.% % For% instance,% in% stage% 4% of% the% SB% treatment,% some% participants% may% have% felt% they%“should”%have%defected,%even%if%they%were%not%sure%why,%just%because%the%R%payoff%was% returning%to%its%baseline%level%of%98.% ' Analysis'of'Strategies' ' % In%some%cases,%by%viewing%the%game%histories%of%each%pairing%in%each%treatment,%it%is% possible%to%determine%the%strategies%used%by%the%agents.%%Determining%the%strategies%used%by% the% agents% allows% a% comparison% between% treatments% that% can% also% shed% light% on% how% the% treatments% affected% the% way% the% participants% played% the% RPD.% % Unfortunately,% more% often% than%not%in%this%experiment,%it%could%not%be%determined%which%strategy%an%agent%used.%%For% example,%if%both%agents%cooperated%in%all%10%stages,%they%could%both%be%playing%either%Always% Cooperate,%TitJforJTat,%any%Trigger%strategy,%or%something%else%entirely.%%Without%a%defection% and%with%only%10Jstages%of%play,%it%is%challenging%to%determine%which%strategies%were%driving% the%actions%of%the%agents.%%Defection%typically%reveals%strategies%by%eliciting%a%response%from% the% defector’s% counterpart,% but% in% some% instances% even% a% defection% doesn’t% reveal% the% underlying%strategies.%%For%example,%if%agent%2%cooperates%in%stages%1J3%and%defects%in%stages% 4J10,%and%agent%1%responds%by%defecting%in%stages%5J10%(after%cooperating%in%stages%1J4%and% observing%agent%2%defect%in%stage%4),%it%is%impossible%to%tell%whether%agent%1%is%playing%TFT%or% Grim%Trigger.%
% Thus,% in% determining% which% strategies% each% agent% employed,% some% simplifications% had%to%be%made%in%how%strategies%were%grouped.%%Specifically,%strategies%were%grouped%into% one%of%three%broad%categories%with%the%following%characteristics.% % Trigger5:%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
5%As%explained%above,%it%is%often%impossible%to%distinguish%between%TFT%and%Grim%Trigger%strategies%in% this%experiment.%%As%such,%I%chose%to%group%them%together%as%Trigger%strategies,%as%TFT%and%Grim%The%agent%cooperates%in%stage%1.%%However,%the%agent%also%has%a%“trigger”%that%he%or% she%will%activate%when%his%or%her%counterpart%defects%a%certain%number%of%times.%%Once%the% trigger%is%activated,%the%length%of%defection%can%be%anywhere%from%one%stage%to%all%stages%(in% which%case%it%is%a%“Grim%Trigger”).%%For%instance,%in%pairing%9%of%the%EB%treatment,%participant% 17% cooperated% in% the% first% three% stages% while% his% opponent% defected% in% those% first% three% stages% (See% Appendix:% Table% 1).% % In% stage% 4% and% thereafter,% participant% 17% defected.% % Thus,% participant%17%used%a%trigger%strategy%with%three%stages%of%“forgiveness,”%but%then%issued%a% severe%“punishment”%of%defection%for%the%rest%of%the%RPD.%
TitJforJTat% is% also% a% trigger% strategy,% and% was% thus% grouped% under% the% broader% umbrella%of%trigger%strategies,%as%it%was%almost%always%impossible%to%confidently%distinguish% between%a%TFT%strategy%and%another%type%of%Trigger%strategy.% % Always%Cooperate%through%Stage%9,%then%Defect%(AC%w/%D):%% This%strategy%was%ubiquitous%in%the%SB%treatment.%%An%agent%cooperates%for%the%first% nine%stages%of%the%game,%and%then%defects%in%the%final%stage.%%This%strategy%was%a%surprise%and% on%the%surface%may%seem%strange,%but%it%is%not%too%different%from%a%plausible%prediction%of%the% Li%and%Kendall%model%with%a%sufficiently%high%δ.%%That%is,%participants%continue%to%cooperate% because% they% believe% their% counterpart% will% cooperate% (after% all,% their% counterpart% has% cooperated%in%every%stage%to%this%point)%and%because%the%expected%value%of%cooperation%is% still%higher%than%the%expected%value%of%unilateral%and%then%bilateral%defection%(as%in%Li%and% Kendall,%both%players%are%assumed%to%be%TFT%players).%%% % Unsure:%%% As%explained%above,%there%are%many%instances%where%an%agent’s%strategy%could%not%be% deduced%from%the%history%of%the%RPD.%%These%strategies%are%labeled%“Unsure.”%%These%agents% may% have% had% strategies% that% dictated% their% actions% in% each% stage,% but% it% is% not% clear% what% they%were%and%thus%they%are%lumped%together%into%this%grouping.%
Using% a% chiJsquared% test,% there% was% a% significant% difference% at% the% .05% level% in% the% usage%of%the%AC%w/%D%strategy%between%treatments%(Table%4).%%The%greater%prevalence%of%this% strategy%in%the%SB%treatment%extols%the%value%of%the%EB%treatment,%which%is%that%it%prevents% this%final%stage%defection.%%But%somewhat%counterJintuitively,%the%fact%that%many%agents%used%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Trigger%are%both%elements%in%the%set%of%Trigger%strategies.%%However,%there%were%three%instances% where%it%could%be%definitively%said%that%a%strategy%was%Grim%Trigger%and%not%TFT.%%These%are%indicated% in%the%table.%the% AC% w/D% strategy% in% the% SB% treatment% is% actually% very% good% for% overall% levels% of% cooperation.%%In%fact,%if%all%agents%used%this%strategy,%there%would%be%a%90%%cooperation%rate.%% But%it%is%important%to%note%that%while%in%the%SB%treatment%AC%w/%D%was%used%12%times,%in%the% EB%treatment,%14%agents%over%seven%pairings%cooperated%in%every%stage.%%Thus,%while%ex%ante%
the% strategy% AC% w/% D% seems% like% it% would% positively% contribute% to% the% overall% levels% of% cooperation,%it%is%inferior%in%the%exJpost%comparison%versus%the%other%strategies%that%resulted% in%seven%pairs%having%100%%cooperation%in%the%EB%treatment.%%This%shows%that%the%increase% in%early%stage%payoffs%in%the%SB%treatment%would%have%been%better%spent%in%the%later%stages.% There%was%also%a%significant%difference%in%the%amount%of%“Unsure”%strategies%between% treatments,%but%it%not%possible%to%say%anything%meaningful%about%this%difference%because%the%% “Unsure”%grouping%is%heterogeneous.% ' The'Importance'of'the'Final'Stage% % The%largest%difference%in%cooperative%outcomes%in%a%single%stage%between%treatments% is%in%stage%10%(see%Figure%7);%the%difference%in%cooperative%outcomes%in%this%stage%alone%is%7,% which% is% more% than% half% of% the% total% difference% in% cooperative% outcomes% across% all% stages% (12).% % What% caused% this% large% difference% in% cooperation?% % I% believe% in% this% case% the% most% obvious%answer%is%the%most%correct%answer:%the%EB%treatment%was%active%in%this%stage%while% the% SB% treatment% was% not.% % However,% this% answer% is% not% fully% explanatory,% and% thus% it% is% worthwhile%to%take%a%closer%look%at%this%final%stage%of%the%EB%treatment%to%understand%more% precisely%how%such%high%levels%of%cooperation%were%reached.%
The%final%stage%of%the%EB%treatment%–%like%the%first%stage%of%the%SB%treatment%–%is%not%a% prisoner’s%dilemma%(T=R=140).%%This%modification%led%to%100%%cooperation%in%stage%1%of%the%
!
Strategy% End%Big% Start%Big% Chi2%
