Master Thesis Political Science – International Relations
Word count: 18,200 Tim Peute, 6286305 June 24, 2016
Supervisor: Farid Boussaid Second reader: Luc Franssen
2
Note: the interviewees are not anonymous in this version and is not supposed to be shared online.
3
ABSTRACT
Political goals and methods have increasingly been incorporated in the past fifteen years. As a result, development thinking has changed significantly in the past fifteen years. Politically smart methods focus on understanding hów development works, whereas the focus used to be on what type of development projects work. Political economy analysis (PEA) is one of these newly created political methods. The organizational implementation of PEA has, however, been met with a lot of resistance. Two early adopters of PEA are DFID and DGIS. These cases are studied by focusing on the differences in the process of implementing PEA and possibly generating hypotheses.
Three types of implementation are distinguished: intellectual, strategic and operational implementation. DFID has implemented PEA intellectually to a (much larger) extent than DGIS. However, DGIS obliges its embassies to incorporate a contextual political economy analysis at the start of every MASP – resulting in a larger degree of operational implementation. Finally, DFID has strategically implemented PEA to a larger degree.
Four hypotheses explaining why these differences exist can be formulated. First, political goals and political methods can conflict with each other when findings are politically sensitive. Secondly, the existence of a dense relationship between research institution and the aid agency are important for intellectual implementation. Third, a large degree of decentralization and non-mandatory PEA studies allow for revolutionary project implementation furthering the quality of PEA implementation. Finally, politically elected PEA champions are important for the first adoption of a PEA approach and later to allow for the policy space to improve PEA implementation.
4 Abbreviations
DFID Department for International Development DGIS Directorate General International Cooperation
DoC Drivers of Change
DDD Doing Development Differently
DMH Directorate Human Rights and Humanitarian MASP Multi-Annual Strategic Plan
MFA Ministry of Foreign Affairs
ODI Overseas Development Institute
PCA Power and Change Analysis PEA Political Economy Analysis
SGACA Strategic Governance and Anti-Corruption Analysis TWP Thinking and Working Politically
5
1 Introduction ... 8
2 Literature review ... 10
2.1 Politics & development ... 10
From good governance to political methods ... 10
A two-fold critique against a technocratic approach ... 11
2.2 Political economy analysis ... 12
What is PEA? ... 12
Two generations of PEA ... 13
Moving beyond PEA ... 14
Barriers of PEA implementation ... 16
Reflections ... 17
3 Theoretical framework ... 18
3.1 Institutional construction ... 18
3.2 The implementation of policy diffusion ... 19
4 Conceptual Framework ... 22
4.1 Definitions ... 22
4.2 Operationalization implementation ... 23
Intellectual implementation ... 23
Operational & strategic implementation ... 24
4.3 Possible explanations for differences in implementation ... 25
4.4 Overall framework theory & methodology ... 26
5 Methodology ... 28
5.1 Descriptive process tracing ... 28
5.2 Content analysis ... 29
6
6 Data analysis ... 31
6.1 Case selection ... 31
6.2 Documents ... 31
Types of documents ... 31
Obtaining the documents ... 32
Validating the documents ... 33
6.3 Interviews ... 34
6.4 Atlas analysis ... 35
7 Analysis ... 37
7.1 DFID Implementation ... 37
1999 – 2004: how PEA started ... 37
2002 – 2006: ad-hoc PEA implementation ... 38
First step of theorization ... 39
2009 – 2016: strategic & operational implementation ... 40
2016: theorization 2.0? ... 41
7.2 DGIS Implementation ... 42
2007: the adoption of PEA ... 42
2007: process of implementing PEA ... 43
2007 – 2010: first and last phase of SGACA ... 44
2010 – 2016: strategic & operational implementation ... 45
7.3 Comparative analysis ... 47
Onset of drivers of change & SGACA: policy diffusion ... 47
First phase of implementation: mandatory studies & consultants ... 48
Same problems, different uptake ... 49
Operational implementation & decentralization ... 50
Policy diffusion & neglected research institutions ... 52
7
8.1 Main findings ... 54
8.2 Possible explanatory patterns ... 56
9 Bibliography ... 59
10 Appendix A: Atlas Codes ... 65
8
Why do some aid projects fail and others succeed? This question is central for improving development aid. Scholars and practitioners who have sought to answer this question are concerned with central development questions, such as what constitutes a developed state and how should aid agencies tackle contemporary development issues. Post-WWII theories on development initially focused primarily on economic development. In the past decades political or institutional concepts such as ‘good governance’ have increasingly been incorporated in projects as development objectives, as cross-cutting goals or a necessary means for sectorial goals. However, these concepts only impact the question what kind of development projects are useful to implement in developing countries and do not address the question how we should approach development. The latter question has received increased attention by both scholars and aid practitioners in the last ten years. Two broad answers to this question exist. First, development can be approached deterministically and technocratic, which view development policies simply need to follow the path of more developed states. A second view argues development projects should include more political goals and/or political methods and aid practitioners should learn how to think politically instead of technocratic.
The starting point of my thesis is that providing development aid is essentially a political endeavour (Unsworth, 2003), for the donor as well as the recipient. This notion has been receiving more attention, by academics and aid agencies, but so far implementation of it has not yet proved successful. In academic circles, the inclusion of politics into development practices is not a question of yes or no anymore, but more focused on hów politics should be incorporated. Recent research argues political economy analysis (PEA) – one way to incorporate politics into development thinking – has hardly been operationalized by its most active proponents, the Department for International Development (DFID) and the World Bank (Yanguas & Hulme, 2015), suggesting new policy ideas rarely diffused into operational thinking of aid agencies. Several possibilities for future research arise with this notion. PEA has been adopted by the World Bank and several bilateral aid agencies such as DFID (United Kingdom) and Directorate General for International Cooperation (DGIS; the Netherlands). Although not identical in their PEA approaches, these three aid agencies have sought to increase awareness of the political aspects of aid.
9
The purpose of this comparative case study is to understand how the political economy analysis (PEA) approach has been adopted by aid agencies of the United Kingdom and the Netherlands. The PEA approach is defined as “the set of donor tools developed to analyse the drivers of political behaviour in specific [political economic] contexts where donors work and the impact this has on development interventions” (Dasandi, et al., 2016, p. 1). It is unclear how DFID and DGIS differ in their implementation process and in their degree of PEA implementation. So far, the differences in processes of implementation during a specific time period have not been analysed. Two questions are formulated to enhance our understanding of these processes. First, in what ways does the process of implementing the PEA approach, as adopted by the DFID and DGIS, differ? In order to understand these processes it is important to know how the two individual aid agencies have implemented PEA. The focus of this question is exploratory. The second question – how can these differences be explained? – aims at generating hypotheses explaining why these processes differ by using a number of aspects that could impact the length and quality of PEA implementation suggested by the literature.
The thesis will combine the current line of thinking in PEA literature with more theoretical foundations of how aid agencies implement new policies and use these insights to analyse the two cases individually using process tracing, compare the two cases and finally generate hypotheses possibly influencing the process of PEA implementation. Chapter 2 is aimed at to explaining how politics became incorporated in development thinking and how this fed into more current thinking on political economy analysis. Subsequently, chapter 3 proposes two separate mechanisms of PEA implementation. Chapter 4 combines these two chapters in order to develop a conceptual framework. Next, theoretical underpinnings of the three methodologies used in this thesis are explained (chapter 5) and applied to this thesis (chapter 6). The two cases are analysed separately and compared in chapter 7. Finally, in chapter 8 conclusions are formulated.
10
Literature outlined in the first section is presented in chronological order, aimed at illustrating four changes in development thinking important for the creation of political economy analysis (2.1). PEA is one of the newest development innovations focused on increasing aid effectiveness. The acknowledgement that the way aid is disbursed is important for the outcome of development is not new. However, the notion that politics is becoming an essential ingredient for thinking about how to deliver aid is relatively new. PEA is one of the approaches that uses this new insight. It is important to understand the context of PEA, meaning how it has developed in the past fifteen years and which barriers of PEA implementation have been suggested so far. This is highlighted in the second section of the literature review (2.2).
This section illustrates how governance became an important development goal in the 1990s, the increased usage of political goals and methods to achieve development goals (2.1.1) and finally, two important arguments against a technocratic approach were formulated in the mid-2000s (2.1.2).
In the mid-90s the role of economic policies was strengthened by an important study done by the World Bank (Burnside & Dollar, 1997) showing aid is positively correlated with growth in countries with good economic policies, and lower aid effectiveness in countries with bad economic policies. Although strictly limited to economic policies, the study increased the importance of good policies for aid conditionality and emphasized the role of good policies for aid effectiveness. The World Bank 1998 Report Assessing Aid further strengthened the idea of good policies as a precondition for effective aid (The World Bank, 1998). Partly due to these two publications good governance and institution building were incorporated more and more as means aimed at fostering economic growth and increase aid effectiveness or as precondition for distributing aid. Although aid became more political in terms of content, the way aid was perceived remained technocratic. In short, the 1990s were marked by a realisation that strengthening governance was a prerequisite for effective aid: it should serve as a way of measuring whether aid could be allocated effectively in a given country. In the 2000s good
11
governance became, besides an end, also a means to increase effectiveness of non-governance sectors (i.e. good governance became a cross-cutting issue) (Carothers & de Gramont, 2013). At the turn of the century, two strands emerged in development explicitly focusing on the importance of politics for development: political goals and politically smart methods (Carothers & de Gramont, 2013). Political goals refer to, for example, democratisation and improving the rule of law. In other words, development goals taking a clear political position. Politically smart methods are more designed to uncover what is ‘behind the façade’ and explain the historical background of the country and how institutions mediate between important agents and so-called foundational factors (e.g. history of a country). The specific focus of politically smart analyses differs from organization to organization.
The increased use of political goals and methods also marked a deviation from a technocratic approach. In order to understand why political methods are less technocratic, it is first important to understand what it means. Although the literature has offered few definitions for this approach, one possible definition was offered by Carothers (2016). He indicates a technocratic approach is a knowledge-based approach and neglects, for example, conflicting interests. In addition, a knowledge-based approach is more likely to prioritize existing theories instead of country contexts. A technocratic versus a more political approach lays bare some fundamental epistemological debates within development (not limited to scientific spheres). A technocratic perspective presumes we are able to find out how we can aid a developing country within a relatively short timeframe and implement this successfully. On the contrary, a political perspective holds development is incremental. Therefore, solutions are very difficult to find without detailed knowledge about the context of a certain country.
Two ideas were important for arguing against a technocratic approach. First, the seminal paper by Grindle (2007) made a strong case for including contextual factors in good governance programs; making an argument related to what a PEA proponent might say: “thus, analysis of the strengths and weaknesses of states, […] these are suggested here as ways that practitioners can increase the capacity to make decisions about what to do in particular countries” (Grindle, 2007, p. 572). In order to know how aid should be disbursed or projects should be managed, understanding the specific context one works in is important. Taking context into account is based on the idea there is no one-size-fits-all solution for development. Second, this notion also questions where development solutions come from and whether they take the specific context
12
into account. Rodrik (2008) made a strong against ‘best practices’, arguing so-called second-best institutions can lead to better outcomes. Following from the same perspective, Brinkerhoff (1996) argues the process of project implementation should be flexible and, in general, second or even third-best solutions are best to use in practice due to the non-linearity of policy implementation. The emergence of political economy analysis is part of the same ‘movement’ to make development more tailored to specific contexts instead of apply one solution across a continent or even the world (something reforms based on the Washington Consensus tended to do more). To sum up, these papers emphasized two arguments against a one-size-fits-all, knowledge-based approach. First, taking context into account improves aid effectiveness. Second, formulating development solutions based on ‘best practices’ implies a solution that works on side of the world works on the other side of the world as well whereas development solutions are implemented more successfully when tailored to a specific situation.
When acknowledging that taking context into account will improve aid effectiveness, the next step entails understanding the context. A myriad of different analyses are available to analyse the context of a country, politically smart methods being one of them. Scholars and practitioners generally agree the newest wave of thinking politically marks a significant deviation with the status quo. Some even call the increasing usage of political aspects (both goals and methods) an ‘almost revolution’ (Carothers & de Gramont, 2013).
This section will explain what political economy analysis entails (2.2.1), how it has matured (2.2.2) and increasingly incorporated objectives related to the (organizational and developmental) surroundings of PEA studies (2.2.3). Finally, possible organizational barriers, constraining the implementation of PEA, are explained (2.2.4).
Within the group of more politically oriented approaches, different politically smart methods still exist. The different kinds of political (economy) analyses adopted by aid agencies can be traced back to different academic backgrounds. As the name suggests, PEA tools are based on conceptions from political economy, emphasizing the interplay between politics and
1 Most of the publications mentioned in this paragraph come from reports published by research institutions such
as the Overseas Development Institute, Development Leadership Program and the Institute for Development Studies.
13
economics. In contrast, Scandinavian (the Norwegian and Swedish aid agencies in particular), more political analyses can be derived from critical theory and (political) sociology (Pettit & Acosta, 2014). Both types of analyses address the question of how development is done instead of what constitutes development. Political analyses imply a shift in focus in development thinking, “from “what” countries need to do to eliminate poverty, to “how” best to support the processes of change involved” (DFID, 2003, p. 5). Political economy analysis in particular usually focuses on analysing the historical characteristics, the institutions (formal and informal) and the interests and capacities of important actors of a given country. Almost fifteen years after the first PEA studies have been conducted, the scope, detail and surrounding development projects have been studied extensively by a small group of academics and development practitioners. As a result, the scientific field of PEA has matured. Two important debates are important to emphasize for this thesis. The first debate revolves around the question ‘what kind of political economy analysis works best’? Several generations of analysis can be distinguished, these are specified in the next sub-section. A second debate is centered around the organizational barriers of PEA implementation.
Two generations of political economy analyses can be distinguished. DFID’s Drivers of Change (DoC) studies and the Dutch SGACAs are considered to be first-generation politically smart methods (Menocal, 2014). After the first novelties of the first analysis wore off the first evaluations of the drivers of change as well as the SGACA approach were published, listing a number of deficiencies associated with the first generation of PEA studies. DFID and DGIS initially focused primarily on the systemic level (of countries), were too broad in scope, did not provide enough in-depth knowledge and were too focused on underlying constraints (Menocal, 2014). The first generation generally had more difficulty operationalizing PEA studies and translating them into clear policy objectives (Wild & Foresti, 2011) (Duncan & Williams, 2012). One of the evaluative comments on the SGACA exercise reflected this as well: “analysis of the political context is still regarded as important, but it is felt that this should be done in a more country-specific and demand-driven fashion” (Wild & Foresti, 2011, p. 3).
Building on the critique of the first generation, the second generation of PEA focused less on constraints and more on analysing specific problems (see, for example (Fritz, et al., 2014)) or sectors in order to facilitate the translation from study results to policy objectives. The second generation of PEA not only restricted itself to the type of PEA, but also move beyond PEA
14
itself by identifying conditions regarding development staff and the way projects are managed that enhance politically informed programming. In short, “the real challenge, they say, is to integrate political approaches into all types of development activity to achieve greater change” (Lucy, 2015, p. 44). The second generation of PEA took the meaning of a political perspective further than merely conducting country studies by arguing development practitioners should learn how to think and work politically (Menocal, 2014). Consequently, it was more concerned with the question ‘how donors should act differently’. In this regard, there have been more positive messages in an ‘act more political’ manner (Unsworth, 2008) but also more negative messages pointing out, for example, how PEA is becoming another product instead of in-depth analysis with the potential to change mindsets (Fisher & Marquette, 2014). Instead of a product, some proponents perceived PEA as an on-going analysis incorporated in aid programs as they are implemented, instead of a scoping analysis before an organization decides where and how to invest its money or even before aid programs are designed.
Thinking about PEA not only matured, but the PEA community has also grown. Several communities, dealing with politically informed approaches, have been formed. An international thinking and working politically (TWP) community of practice has been created in 2013. They have been joined by a doing development differently (DDD) community (Dasandi, et al., 2016), which emphasizes the problem driven iterative adaptation (PDIA) approach, designed by Andrews (Whaites, et al., 2015). In addition, the Governance Practitioner’s Notebook has been published by the OECD (Whaites, et al., 2015). This notebook brings many of the academics part of the DDD community or the TWP community together. Building on this idea, Booth & Unsworth’s (2014) widely cited study ‘politically smart, locally led development’ mentions “donor staff can work – directly or through partners – in ways that we call politically smart and locally led if the political and bureaucratic environment within their own agencies is supportive” (Booth & Unsworth, 2014, p. 1). Based on the seven success stories of how politically smart, locally led (an approach more far reaching than PEA) programs have been implemented, they conclude “donors can facilitate developmental change in very challenging contexts, but only if they are prepared to align their own thinking and practices with the uncomfortable reality that processes of developmental change are complex, unpredictable, mainly endogenous, and pervaded by politics” (Booth & Unsworth, 2014, p. 27). The two processes of iterative problem-solving and investment in building relationships with a wide range of stakeholders were vital for the successful implementation of these seven exceptional
15
cases (Booth & Unsworth, 2014). These findings point at a wider development problem; in order for politics to ‘maneuver’ around in development practices projects need to become more iterative, adaptive and long-term.
Besides the content of PEA studies or politically oriented projects, two points regarding the people involved in PEA studies and PEA implementation have been mentioned by PEA experts as well. Menocal (2014) has drafted a list of characteristics of successful implementation of ‘working and thinking politically’ mentioning the adoption of flexible and adaptive approaches but also finding the right people (‘champions’) are important. Brinkerhoff also identifies the presence of policy champions (i.e. individuals who are able to serve as leaders for change) as important for successful policy implementation (Brinkerhoff, 1996), but also for policy legitimation (Crosby, 1996). Secondly, it was questioned whether international experts and consultants should be involved in PEA studies (Menocal, 2014) because it would hinder the internal learning process especially if you consider PEA studies to be a way of achieving ‘thinking politically’, an endeavour essentially carried out by external staff when consultants are hired to conduct PEA studies.
Thinking and working politically is, however, not synonymous to political economy analysis, it focuses on more than a type of political (economy) analysis (Dasandi, et al., 2016). Examples of TWP uptake range from an evolutionary uptake – giving attention to politics within traditional aid approaches (Green, 2014) – and of a revolutionary uptake – shifting away from traditional aid approaches. If only formal PEA is conducted without changing the approach, it is not TWP, synthesized at a TWP Community meeting (Green, 2014). In other words, organizations or programs that use PEA to also learn to think more politically and adjust their programs to these insights, are part of the TWP framework. Where barriers of implementation exist for PEA, according to some due to structural elements of aid or “the political economy of aid” (Green, 2014), probably even more exist for revolutionary uptake of the TWP framework because it involves a break with all traditional approaches to development (e.g. a technocratic approach).
A wide range of aspects of the second generation has been listed. How these aspects are used for analysis is explained in the conceptual framework.
16
A second debate revolves around the implementation of PEA.2 Two broad aspects are important: the possible influence of the organizational culture and the consequences of organizational structure itself. A study similar in subject (to this thesis) published by Yanguas and Hulme (2015), but different in cases (they analyzed the World Bank and DFID) find several important organizational barriers that hinder (further) implementation of PEA. The PEA culture was found to be largely limited to a small amount of people from the Governance department at aid agencies and some specific country offices and isolated from the larger culture of development practitioners in general (Yanguas & Hulme, 2015), a number too small to constitute a broad cultural shift (Carothers & de Gramont, 2013). This small group may be reflected by the way of thinking in most development agencies, which was and perhaps still is largely technocratic (Unsworth, 2009) (Carothers & de Gramont, 2013). Hout emphasizes how only DMH (Directie Mensenrechten en Humanitaire Hulp) at DGIS pushed for the implementation of SGACA and received little support from other departments (Hout & Schakel, 2014). Finally, a good relationship between the people conducting the analysis and the operational staff is important because a high level of support among those who have to apply the studies (either embassy or country office staff) is key (Carothers & de Gramont, 2013). For this reason the extent of decentralization and the amount of money transferred directly to country offices could be important (Yanguas, 2016). The degree of decentralization might also affect the ()mandatory nature of PEA studies. Yanguas & Hulme (2015) found the non-mandatory nature of PEA studies results in conducting PEA studies only in areas individual managers saw the merits of conducting this kind of studies.
Aid agencies are sometimes (e.g. the Netherlands) part of the ministry of foreign affairs. This means what and how programs and designed is more related to bilateral relationship between the donor and recipient government (Carothers & de Gramont, 2013). Development assistance is more likely to serve a country’s larger foreign policy interests (Carothers & de Gramont, 2013). This might lead to a paradox of having more openly political goals but less politically smart methods (Carothers & de Gramont, 2013). Although, PEA studies being used by the embassy instead of a separate country office could also have its benefits. Embassies are, by default, concerned with the political environment of a country because embassies need to
2 Most of these scholars are proponents of PEA. Most studies seek to answer an underlying, more normative
question ‘how can we show them how valuable this kind of analysis is?’ It is important to keep this in mind during the analysis.
17
maintain a diplomatic relationship with the country in question as well. The organizational aspects listed in this subsection are further elaborated upon in the conceptual framework.
The literature covered in this review mostly argues from a perspective in favour of a political approach to development. I position myself within this group of proponents, but it is important to place two caveats. Most of the research is derived from the same research institutions, increasing the possibility the literature is biased in favour of PEA. Second, the link between higher aid effectiveness and the use of political (economy) analysis is still debatable (Routley & Hulme, 2013). Compared to fifteen years ago, the notion of taking the ‘how’ of politics into account is becoming common knowledge nowadays, but how this leads to improved aid effectiveness is predominantly based on individual case studies.3 The approach of this thesis differs in three respects from most research. First, these two specific cases have not been compared yet. The World Bank and DFID have recently been compared, but solely on organizational barriers and not regarding the process of implementation. Secondly, the difference of implementation processes of PEA between aid agencies has not been addressed by scholars yet. The difference in terms of implementation processes is still a gap in the literature. Cases have been compared with each other, but not yet compared in time. In order to analyse differences in process, not only development theories but also theories on organisational change and institutional change are used to make the research more ‘grounded’ in theory and raise expectations about how the process of implementation unfolds. Finally, the organizational barriers are mostly presented as systemic characteristics whereas a wide difference in uptake exists. A comparative analysis between cases and across time can provide information on what matters more in what time period. The last subsection (section 2.2.4) serves as basis for identifying the explanations for why the difference between cases exist.
3 Theoretical linking PEA with aid effectiveness across all cases may also lead to epistemological difficulties
because PEA as analysis uses a contextual approach. If results about the effectiveness are also context specific, it becomes difficult to generalize the effectiveness of PEA.
18
Where the previous chapter focused mainly on showing the rationale behind political economy analysis, this chapter will offer the theoretical underpinnings necessary to conduct a robust analysis on the implementation of PEA by adding two theoretical insights that emphasize the different phases of implementation and the interaction between organizations. This chapter provides a theoretical basis for comparing the differences between the DFID and DGIS in different time periods. The different time periods are explained in this chapter. The next chapter explains how the theories are applied to the cases. First, a theory proposing how a new policy idea is implemented is presented (3.1). This theory serves as basis for one of the ways implementation is operationalized, further elaborated in the next chapter. Subsequently, the possible impact the interaction between organizations could have on the process of implementation is explained. The second section serves as basis for one of the explanations of a difference in implementation (3.2).
One of the more comprehensive books on organizational sociology has been written by Scott (2014). Offering an overview of the theories on institutional construction and institutionalization, he adopts several broad perspectives, based on philosophical underpinnings similar to the different strands within international relations theory. The theories used here are largely based on the cultural-cognitive perspective using assumptions overlapping with constructivism.
The implementation of PEA by one aid agency is an instance of institutional construction. Institutional construction is studied “if attention is directed primarily to the processes and conditions giving rise to new rules, understandings, and associated practices” (Scott, 2014, p. 114). Three phases of institutional construction can be distinguished. Institutional construction starts with the “development, recognition, and naming of a recurrent problem to which no pre-existing institution provides a satisfactory repertoire of responses” (Scott, 2014, p. 126). Subsequently, when the problem is recognized and generalized to a certain extent, (several) ad-hoc solutions can be formulated. After ad-ad-hoc solutions have been implemented, it is possible to engage in “a more thoroughgoing ‘theorization’ of the situation – in other words, to formulate general accounts of how the system works and, particular, of which solutions are appropriate in
19
which contexts” (Suchman, 1995, p. 43). The latter part of this quote shows some degree of self-reflection and the ability to implement these reflections is important for the theorization of a new solution.
However, this theory views institutions from a demand-side perspective, meaning institutions are crafted in direct response to recurring problems, a view similar to the later generations of PEA, signalling PEA should specify how to tackle the problems identified by the study. In other words, institutions embody or channel solutions for these recurring problems. The overall institution of the aid agency is not only demand-driven. Although one of the main goals of an aid agency is related to development, e.g. eradicating poverty, it is still part of a national political body and a political agent itself. Aid agencies operate based on more than development goals because they are motivated by multiple interests, including self-interest. In order to implement a new idea, political will to implement this idea is necessary. Altogether, the steps of implementation are used for the operationalization of one type of PEA implementation (explained in 4.2.1).
The previous section distinguished three phases of how individual aid agencies implement a new policy idea. This section will further elaborate on different phases of implementation but focuses on how aid agencies can influence each other’s policy during different implementation phases. Partly because PEA has a strong international character of people and practitioners connecting and influencing each other, donors internally replicate ““what works” or at least “seems to work” for their peers” (Yanguas & Hulme, 2015, p. 210). Hence, aid agencies can be subject to institutional isomorphism. This concept is extensively studied by DiMaggio and Powell, explaining isomorphism involves a process making organizations more similar but not necessarily more efficient – defined as “a constraining process forcing one unit to resemble other units that face the same set of environmental conditions” (DiMaggio & Powell, 1983, p. 147).
A specific application of the ideas behind institutional mimicry or institutional isomorphism is policy diffusion, sometimes referred to as policy transfer (the term policy diffusion is mostly used in IR, whereas policy transfer by public policy researchers; (Marsh & Sharman, 2009)). Policy diffusion refers to a process where policy innovations are transferred from one government to another and can be defined as “concerned with the process by which knowledge
20
about policies, administrative arrangements, institutions and ideas in one political system (past or present) is used in the development of policies, administrative arrangements, institutions and ideas in another political system” (Dolowitz & Marsh, 2000, p. 5). Policy diffusion takes the notion of institutional mimicry as starting point and raises expectations about how institutional mimicry occurs.
In order to analyse when (meaning during which phase of implementation) policy diffusion occurred, it is important to make policy diffusion time-dependent. This is possible by integrating policy diffusion with a theory of institutionalization. Institutionalization is conceptually similar to institutional construction, focusing on the role of increasing objectification of shared beliefs, also highlighting how institutions influence each other (Scott, 2014). The first step encompasses the habituation of a new innovation. Once an institution habituates a new innovation as a result of interactions within and between organizations, innovations can become formally theorized. This involves formulating why, how and for who the innovation is effective (Scott, 2014, p. 148). Objectification follows theorization, encompassing “the development of some degree of social consensus among organizational decision-makers concerning the value of a structure, and the increasing adoption by organizations on the basis of that consensus […] the impetus for diffusion shifts from simple imitation to a more normative base” (original quote from Tolbert & Zucker, 1996, p. 182-183) (Scott, 2014, p. 148). During the second and third stage of implementation, theorization and objectification, the formulated solutions are more likely to diffuse to similar contexts.
In order to facilitate the analysis of policy diffusion in the case of DFID and DGIS, a specific set of questions is used, adopted from Dolowitz & Marsh (2000). Three of these questions are used as focus points in the analysis. First, four degrees of policy diffusion are distinguished: copying (completely transferring the lender country’s policy), emulation (the idea behind the policy is transferred), combinations of several policies are transferred and finally, inspiration which means policy from another jurisdiction inspires policy change. This answers the question to what degree the policy diffused.
Secondly, if PEA was indeed diffused, it is important to know who was involved. A distinction between external and internal actors is made. External actors are expected to have been actively involved in the process because of the possible existence of a PEA epistemic community (Yanguas & Hulme, 2015). Epistemic communities have been typified as the ‘go-between’ group in the process of policy diffusion (Graham, et al., 2013). Epistemic communities can be defined as being “responsible for generating new ideas and disseminating them among national
21
policy makers as well as others in the international community. Epistemic communities are networks of professionals and experts with an authoritative claim to policy-relevant knowledge, who share a set of normative beliefs, causal models, notions of empirical validity and a common policy enterprise” (Campbell, 2002, p. 30). At first sight, the PEA community could be called an epistemic community. It is a network of professionals and experts, they claim policy-relevant knowledge, have similar normative beliefs and causal models a link also argued by Yanguas & Hulme (2015).
Thirdly, policy diffusion may have had negative consequences for the implementation of PEA. The process of policy diffusion can lead to policy failure in three ways. First, because the borrowing country obtained insufficient information about the policy or institutional structure that was transferred. Secondly, because the borrowing country did diffuse crucial elements (essential for successful transfer) of the policy or institutional structure. Finally, because the economic, social, political and ideological contextual differences between the borrowing and lending country did not receive enough attention during the process of diffusion. Policy diffusion is one of the possible explanations for the difference in implementation, further explained in 4.3.
22
In order to use the theories described in the previous two chapters, it is important clarify how the concepts are defined, in some cases operationalized, and most importantly, how the theories relate to each other. This section defines the concepts PEA, adoption and implementation, the three important concepts mentioned in the main question (4.1): to what degree does the process of implementation of the PEA approach, adopted by DGIS and DFID, differ? In addition, a three-fold operationalization of implementation is described (4.2). Thirdly, five aspects identified in the literature review and theoretical framework as possibly impacting the process of implementing PEA – related to the second question how can these differences be explained? – are outlined (4.3). Finally, an overall framework explaining the relation between the main questions, the theory and the methodology is explained (4.4).
Political Economy Approach: “the set of donor tools developed to analyse the drivers of political behaviour in specific [political economic] contexts where donors work and the impact this has on development interventions” (Dasandi, et al., 2016, p. 1). This definition takes political behaviour into account, but PEA looks specifically at the interaction between political behaviour vis-à-vis the economic environment of a specific context and vice versa. Hence, political economic context is added to the definition.
Adoption (as described in purpose statement): a broad definition is adopted, considering the PEA approach adopted by an aid agency when the aid agency expresses possible significant impacts of the PEA approach in, for example, a strategic document introducing the agency’s usage of PEA but also when PEA is used on an ad-hoc basis without a common framework.4
Adoption is the simplest form of implementation. This definition is used to assess the total population of cases, further elaborated in the data analysis chapter. However, once an aid agency has drafted a common framework, a more complex definition and operationalization of implementation is necessary to analyse the two cases.
4 The reason this definition does not contain references is because the definition was formulated without using
23
The implementation of PEA occurs at several levels. The literature review and theoretical framework provide the basis for the operationalization of intellectual implementation of PEA. However, this type of implementation does not cover the full picture of PEA implementation. In order to provide a multi-dimensional view of PEA implementation, two more dimensions of PEA implementation are operationalized: operational and strategic implementation.
The intellectual implementation of PEA is operationalized using the different generations of PEA explained in the literature review and the three phases of implementation illustrated in the theoretical framework.
First it is important to justify why different theories can be integrated. This operationalization is based on two different theories from two different disciplines an displays an interdisciplinary view of the intellectual implementation of PEA. Since the theories are based on the same world view, constructivism – emphasizing the role and importance of ideas on decision-making processes – it is possible to integrate these theories quite straightforward as long as the theories are speaking of the same content.
Figure 1 combines the two concepts. The first generation of PEA is expected to be largely compatible with the ‘ad-hoc solutions’ phase identified by Scott (2014) because these were the first type of PEA tools that were used, were adjusted later by aid agencies themselves and
24
received substantial criticism by academics. The first generation entails PEA on a macro-level of a country as a whole and is analysis that feeds into the design of aid programs but is not embedded in the process of the implementation of an aid program. Criticism of the first generation of PEA fed into the design of the second generation of PEA. It is more difficult to capture the idea of the second generation of PEA in one thought because the landscape of politically informed thinking and PEA matured, became more complex and a myriad of different conceptions of PEA and its associated programs and projects emerged. The second generation of PEA contains three characteristics that are used for analysis: the different uses of PEA studies (country-level, sector-level and problem-driven), thinking and working politically (do PEA studies or frameworks also adopt the goal of changing the way staff thinks about development?) and the project characteristics associated with locally led, politically (iterative, adaptive and long-term projects). As mentioned before, a certain degree of self-reflection is also necessary to progress from ad-hoc solutions to theorization. Development practitioners as well as academics have argued the outlined ideas. What matters here is whether the aid agency was able to progress from the ‘ad-hoc solutions’ phase to the ‘theorization’ stage. In other words, was it able to accept the criticism associated with the first generation of PEA and enhance its PEA tools accordingly to progress to the second generation of PEA?
The intellectual implementation of PEA is important for the quality of PEA tools. However, it is not usable to assess whether the aid agency uses these (enhanced) PEA tools. As argued in the literature review, the implementation of PEA can stumble upon several organizational constraints, such as a prevailing technocratic culture (Carothers & de Gramont, 2013) (Unsworth, 2008). Academics have shown PEA is dealt with by a relatively small group of people within the Governance department of a given aid agency (Yanguas & Hulme, 2015) (Carothers & de Gramont, 2013). This suggests two additional dimensions of implementation ought to be measured – operational and strategic implementation – in order to fully understand the difference in implementation processes. Operational implementation measures the implementation on country level, questioning to what extent countries operationalized PEA. Operational implementation is measured by analysing country-level documents and whether (and how) PEA or political analysis is mentioned. Strategic implementation measures the organization-wide acknowledgement of PEA and the values behind PEA. Strategic implementation is measured by analysing organization-wide documents such as strategic
25
agendas but also sectorial organization-wide evaluation documents. Both types of documents are further specified in the data analysis chapter.
As mentioned in the introduction, the main question is descriptive but it might be possible to explain the differences between DFID and DGIS. Five aspects, explained in the literature review and theoretical framework, are expected to play some role in the process of PEA implementation and differ between the two aid agencies. These aspects are identified in order to the second question – focused on explaining why these differences exist – and to generate hypotheses that can explain why the processes of implementation differs at specific points. The five aspects are: whether PEA studies are mandatory or not, the degree of decentralization, whether the aid agency is part of the Ministry of Foreign Affairs or not, whether PEA Champions were involved, whether political will was present during the implementation and whether policy diffusion has influenced the implementation process of DGIS. The possible explanations for why the process of implementation differed between DGIS and DFID are operationalized below.
- Mandatory: PEA studies are either mandatory or not.
- Ministry of Foreign Affairs (MFA): the aid agencies are either part of the MFA or not.
- PEA Champions & political will: political will are PEA Champions who were politically elected. For the documents there are two codes related to PEA Champions. - Decentralization: the degree of decentralization depends on the autonomy of country
offices or embassies. Three aspects are important; to what degree do countries need approval for the main priorities of development (for the next years) and the way money is disbursed.
- Policy diffusion: It is important to elaborate further on the questions of when, why, what, how, to what degree, from where and who (see 7.3 for the application to this case). An epistemic community of PEA is expected to be important for the diffusion of PEA. Figure 2 visualizes how policy diffusion is expected to occur between two aid agencies.
26
The figure combines the concept of policy diffusion with the three phases of institutionalization, explained in section 3.2. Policy diffusion is expected to take place when the country who diffuses the policy has formulated its first criticism and seems to be progressing to the theorization phase of implementation. As hypothesized in the theoretical framework, the chance of policy diffusion increases as aid agencies progress to subsequent phases.
To sum up, three dimensions of implementation have been identified and five subsequent aspects possibly explaining why the process of implementation differs between DFID and DGIS. Now it is possible to connect the cases with the theory and methodology. The next section visualizes the overall framework.
Figure 3 explains the overall framework and serves as basis for the next three chapters. Each component contains a number that refers to the respective (sub)section information about that component can be found. The three types of implementation identified in this chapter each correspond to their own type of document, used to measure that particular type of implementation. Interviews can substantiate each of the three types of implementation, depending on the information the interviewees provided. Subsequently, two types of analyses are conducted: process tracing, analysing how one aid agency has implemented PEA and
27
comparative analysis, assessing what the differences between the two organizations are and how they can be explained.
28
This study uses a qualitative case study design. The cases analysed in this thesis serve three functions: describing the process of implementation of PEA, comparing the two cases and exploring possible explanations accounting for differences in PEA implementation. For that purpose three methods were used. First, Process tracing is well suited for analysing the process of implementation of new ideas by focusing on the sequence of events important for the implementation of PEA (5.1). Secondly, content analysis supports process tracing and comparative analysis by providing a robust way of finding detailed knowledge from different documents (5.2). The robustness comes from established criteria for how to categorize and interpret data from documents and criteria for the type of documents that are used. Thirdly, comparative analysis seeks to show the differences between DFID and DGIS. Based on these differences, possible patterns explaining the differences in implementation are formulated (5.3). The methodology and data analysis are divided in two chapters. This chapter explains what methods are used, why these were chosen and how they will feed into the analysis. The next chapter focuses on how these methods are applied to the two cases. The content analysis criteria are applied to the documents and codes identified during analysis are illustrated in the next chapter.
The unit of analysis is aid agencies. The methodological focus of this thesis is both descriptive and explanatory. Description as a methodology has been criticized on its value (Gerring, 2012), but as Gerring also argues descriptive analysis has a distinct value in relation to explanatory research. I plan to analyse the implementation of PEA with a combination of both approaches. Ultimately, (weak) relationships are analysed, but a significant amount of descriptive analysis is necessary to infer these relationships. Gerring (2012) identifies ‘what’ and ‘in what manner’ questions as descriptive.
Process tracing focuses on ‘what’ and ‘how’ questions, also allowing the exploration of causal mechanisms. For this thesis, the process tracing part of the analysis is descriptive. Process tracing is a within-case study method and is defined as “the systematic examination of diagnostic evidence selected and analysed in light of research questions and hypotheses posed by the investigator” (Collier, 2011, p. 823). Beach & Pedersen (2013) identify three types of
29
process tracing: theory-testing, theory-building (both theory-centric) and explaining outcome process tracing (case-centric). Theory-centric and case-centric differ in their ontological understanding of the world. The case-centric approach “contends that it makes little sense to distinguish between systematic and case-specific parts, given the impossibility of generalization in the complex social world” (Beach & Pedersen, 2013, p. 13), whereas theory-centric approaches ultimately seek to find causal relations. Following from the descriptive nature of process tracing, this thesis is an instance of case-centric process tracing. A case-centric within-case study does not offer the possibility of generalizing the findings. Because the within-cases are compared, generalization is possible. However, because the analysis is descriptive, the degree of generalization is, most likely, limited.
Content analysis is mainly used to validate the documents used and the categorization of the data analysed in Atlas.ti. Regarding the quality and type of documents, four standards have to be met: authenticity, credibility, representativeness and meaning (de Lange, 2016). Below I explain what each standard means.
1. Authenticity: is the origin of the documents unquestionable? Several ways to determine this are considering which department published the study and the kind of policy document.
2. Credibility: how much distortion do the documents contain (e.g. expressing other interests than providing information)?
3. Representativeness: are the documents representative in relation to the total population of documents?
4. Meaning: is the evidence comprehensible? This is most likely not an issue for this thesis. Furthermore, qualitative content analysis focuses on latent content, is interpretative (no quantification) and inductive (no coding scheme is designed beforehand), largely compatible with the standards of process tracing. Eventually, a coding scheme is necessary to systematically analyse the documents. Coding consists of three steps: condensation, categorization and interpretation (de Lange, 2016). Condensation tells what data is relevant and what is not, categorization focuses on reading the materials and trying to find labels for different categories based on arguments, narratives or frames. This process can be iterative. Finally, interpretation emphasizes what the patterns of the findings mean. Validity and intra-coder
30
reliability of the analysis can be ensured by reviewing how the codes were categorized and adjust codes if not valid. The validity naturally also depends on the validity of the coding scheme, being transparent about possible pitfalls is vital here.
Although the criteria for valid process tracing and content analysis largely overlap – meaning both methodologies allow for inductive analysis (Beach & Pedersen, 2013) – the scope of process tracing and comparative analysis is different. Using multiple methods serves two functions. Process tracing and comparative analysis complement each other because process tracing is a within-case study method, whereas comparative analysis requires multiple cases. Consequently, differences in times as well as between cases can be analysed (see Figure 3). Secondly, where methods do overlap they can be used for triangulation (e.g. interviews and document analysis).
The final part of the analysis – identifying patterns through which the process of PEA implementation is influenced – is focused on hypothesis generation. These type of case studies “contribute to the process of theory construction rather than to theory itself. Theory, defined as a logically interconnected set of propositions, requires a more deductive orientation than case studies provide” (Levy, 2008, p. 5). Although deductive insights (see 4.3) are used to guide the process of generating hypotheses, the overall orientation of this comparative case study is not deductive.
Next, it is important to consider how to combine process tracing and comparative analysis. In order to compare the two cases a substantial amount of knowledge about the individual process of implementation is necessary. Consequently, both cases are analysed separately first after which information is compared according to the three types of implementation and the five possible explanations.
31
The quality of analysis is determined by the justification of case selection and document selection, the validity of the data collection and by triangulating the data with information obtained during interviews. Five aspects are covered in this chapter: why DFID and DGIS were selected as cases (6.1), which documents are chosen (6.2.1), why these documents are chosen (6.2.3), who is interviewed (6.3) and how the Atlas.ti program was used and fits in with the design of this study (6.4).
The selection criteria for these cases is whether the aid agency has adopted a type of political (economy) analysis, defined in the conceptual framework. All aid agencies that are known to have adopted a type of political (economy) analysis are the World Bank, AusAid (Australia), USAID (United States), Sida (Sweden), Norad (Norway), UNDP, DGIS (the Netherlands), DFID (the United Kingdom), EUAid (European Union) and GIZ (Germany). DFID was the first to adopt a PEA approach, DGIS followed five years later. They were both among the earliest to adopt PEA. Within the total population of aid agencies to have adopted a political (economy) approach DFID and DGIS are both ‘most likely’ cases, similar to Mill’s method of agreement since these cases were among the earliest to adopt PEA (Levy, 2008). The two cases vary on the degree of intellectual, operational and strategic implementation (Levy, 2008, p. 15). In conclusion, the two selected bilateral aid agencies are: Directorate General for International Cooperation (DGIS; the Netherlands) and the Department for International Development (DFID; United Kingdom).
This section justifies the type of documents that are used, how the documents were found, to what degree the documents are authentic, credible and representative (see 5.2 on the criteria for content analysis) and finally, who were interviewed.
Three types of documents are used for analysis: strategic documents, country documents and PEA documents. Each type of document measures one kind of implementation and serves as
32
justification for the inclusion of the documents. PEA documents have terms associated with PEA in their title (e.g. Drivers of Change) and are only used to analyse the intellectual implementation of PEA. Strategic documents include multi-annual agendas which cover a larger subject but may tell something about the (type of) political approach the aid agency is pursuing as a whole. Strategic documents also include sectorial evaluation documents about fragile state policies, for instance, are included. In almost all cases strategic documents measure the strategic implementation of PEA. However, due to the low amount of direct documents, not all information (especially evaluative) about DGIS’ implementation of SGACA is covered in these documents. Hence, IOB evaluation documents, usually evaluating a specific development sector, are also used to measure the intellectual development of Dutch PEA. Thirdly, country documents are also included in the list. These are recent Operational Plans (for DFID) or Multi-Annual Strategic Plans (for DGIS). Both identify the trajectory or development cooperation with a specific country for the next four to five years.
In addition, documents come from two different sources: documents published by research institutions, either evaluating or advising the aid agencies about their PEA tools and aid agency (or Ministry of Foreign Affairs, in the case of DGIS) documents. Research institution publications are all typified as PEA documents whereas documents from aid agencies cover all three types. The research institutions include but are not limited to the Overseas Development Institute for DFID and Clingendael for DGIS. Research papers are not included in the analysis because of the analytical character.
PEA documents have been found by searching online or through interviews. In order to prevent randomness of documents, similar search queries have been used to generate the documents, mainly focusing on the websites of the aid agencies and research institutions. Case studies have been obtained through snowballing (based on other documents). Both aid agencies provide clear examples of countries PEA was or has been successful. The degree of implementation in these cases can be expected to be one of the highest – compared to the total population of partner countries the aid agency is engaged in – within that specific time frame. The Netherlands mentions Uganda, Tanzania and Ghana as cases of successful SGACA implementation. The United Kingdom mentions Nigeria as successful in 2003 and Pakistan, Malawi, Burma and the DRC as successful in a document from 2016. These cases are defined as ‘most likely’ cases for measuring the operational implementation. Finally, strategic documents were found by online
33
searches or through snowballing. The total population of strategic documents are all organization-wide documents published by the aid agency or ministry of foreign affairs strategic documents. The strategic documents used for this analysis constitute ‘least likely’ cases compared to all the other documents used for this analysis because they are organization-wide documents usually covering a organization-wide range of topics. For PEA to be included in these documents a large group outside the governance department of an aid agency need to be aware of PEA and find it useful. Note that the population of ‘least likely’ strategic documents (all strategic documents) is different than the ‘most likely’ case studies (all operational documents published by one aid agency).5
Three types of documents are distinguished. Table 1 provides an overview of the characteristics of each type of document. Below the three criteria of content analysis (see section 5.2) are applied to the three types of documents.
Authenticity of documents
- PEA documents are published by the department dealing with good governance but are in some cases published on personal title or by a research institution. When the intellectual implementation of PEA is measured, it is important to include documents that are officially published by the aid agency or MFA as well.
- Strategic documents, containing evaluation documents and strategic agendas. The authenticity is relatively unquestionable for this type of documents, because all agendas are department- (DFID) or ministry-wide (DGIS) documents.
- Country documents are published by country offices or embassies but the operational plans (DFID) or multi-annual strategic plans (MASPs; DGIS) are approved by the headquarters of the aid agency.
Credibility
- PEA documents most likely contain a bias towards legitimizing and promoting PEA. - Strategic aid agency-wide documents are more likely to contain a political message
biased towards a specific development priority.
5 Because a part of DGIS’ strategic documents are also used for the intellectual implementation, they do not
34
- Country documents are not expected to contain any significant biases that can be established beforehand.
Representativeness
- PEA documents are quite representative because the total population of aid agency PEA documents is not much larger than the ones already being used.
- Strategic documents are partly representative. Strategic agendas are only drafted every few years and several from both aid agencies have been included. The evaluative documents have been incorporated because they contain information about political (economy) analysis and are thus not very representative.
- The total population is all operational plans or multi-annual strategic plans. It is hard to establish the representativeness since these documents are typified as most likely cases. Thus, a superficial analysis of all operational plans and multi-annual strategic plans is included to assess the representativeness of these documents.
Type of document
Strategic PEA Country case
Type of implementation Strategic Intellectual Operational Way of obtaining Online & snowball Online, interviews Snowball
Source Aid agencies Research institutions,
aid agencies Aid agencies V ali dati o n
Authenticity Aid agency wide Research institutions Aid agency/
Personal title
Country offices/ Embassies
Credibility Political message Bias promoting PEA –
Representativeness Evaluative/agenda Yes Analysis
Table 1 Characteristics of the three types of documents. For a more elaborate explanation, see text. Source: author.
The use of expert interviews serves two functions. First, personal experiences from staff members are able to connect different documents from multiple years. Secondly, interviews are able to substantiate the data found in the content analysis. By triangulating the data, the validity of the findings increases. Two groups of interviewees can be distinguished.
35
The first group consists of three interviews conducted with academics: Thomas Carothers (Carnegie Endowment for International Peace), Wil Hout (International Social Studies) and Pablo Yanguas (The University of Manchester). These interviews were mainly suited for self-reflection, making sure the theoretical interpretations were correct. In addition, most interviews led to a more nuanced understanding of PEA and its organizational implementation. These interviews were not recorded and were cited in the literature review (if at all).
A second group of interviews was conducted with four (former) aid agency employees and an ODI employee. Unfortunately, only one employee from DFID was able to be interviewed which may skew data. Fortunately, one employee at ODI was also able to provide information on the DFID. These interviews were recorded (see Appendix C for the transcribed interviews) and were used to substantiate the analysis. The interviews were taken using semi-structured questions mainly revolving around the intellectual development of PEA and the role of the five possible explanations for why the process differed (see 4.3). However, two interview slightly deviated from this subject. One of DGIS’ employees was the only employee involved with SGACA at the time and still working at DGIS provided information on the operational implementation of PEA as well. Second, the ODI employee provided detailed information on the different types of PEA and the intellectual challenges that were overcome in the past years.6
Atlas.ti has been used to analyse the documents listed in Appendix B. The program facilitates the categorization and condensation of the data. If the documents would have been analysed without the program, the analysis would not have been as robust and would have probably resulted in a larger degree of confirmation bias (of the theory) and less data. In addition, data in Atlas.ti is easier to evaluate. However, reliability of the data is more difficult to safeguard because only one person has analysed the data. In order to ensure intra-coder reliability, all of the codes were reviewed. In the review, codes were removed, changed, merged together or further subcategorized, increasing the overall validity of the analysis.
Atlas.ti allows designing codes as you read the documents making it suited for both closed and open coding. In order to keep the analysis more iterative and inductive, open coding is used meaning no codes are defined beforehand based on the literature. In total 33 of the documents listed in Appendix B have been analysed and over 77 codes have been defined which has
