Agriculture in East Africa : partial pdentification analysis of the impact of farmer field schools on productivity

(1)

i

Agriculture in East Africa: Partial Identification

Analysis of the Impact of Farmer Field Schools on

Productivity

-Master Thesis Economics- August, 2016 University of Amsterdam

Maximilian Heyer

MSc Economics (Development Economics)

Prof. Dr. Hessel Oosterbeek Supervisor

Dr. Adam Sanoé Booij Second Supervisor

(2)

ii

Statement of Originality

This document is written by Student Maximilian Heyer who declares to take full responsibility for the contents of this document.

I declare that the text and the work presented in this document is original and that no sources other than those mentioned in the text and its references have been used in creating it.

The Faculty of Economics and Business is responsible solely for the supervision of completion of the work, not for the contents.

(3)

iii

Abstract

This paper uses a nonparametric approach to examine the effect of a Farmer Field School (FFS) project in East Africa on the crop productivity and the livestock production. Bounds were set around the Average Treatment Effect and tightened subsequently by applying different assumptions. The results assess a non-negative effect of the FFS project on the crop productivity and livestock production measures, though the range of the bounds differs clearly between countries. This differences show how sensitive the approach is in response to the data examined and the way how the assumptions are applied.

(4)

iv

List of tables

Table 1 - Participating countries, information about basic economic measures and

population’s activity in agriculture ... 5 Table 2 - Comparison of covariates and outcome variables at closure by treatment status, tables are divided by country ... 11 Table 3 - Results based on no assumption ... 17 Table 4 - Results based on no assumption and on the MTS assumption ... 20 Table 5 - Results based on no assumption, the MTS assumption and the MTS-MIV assumption ... 24 Table 6 - Results based on NO assumption, on the MTS assumption, on the MTS-MIV

assumption, on the MTS-MIV-MTR assumption and for the case of Uganda on the MTS-MTR assumption ... 26 Table 7 - Mean of outcome variables in USD by country with standard deviation and number of observations ... 27

(6)

1

1 Introduction

This thesis investigates how Farmer Field Schools (FFS’s) in East-Africa could improve productivity. FFS’s were first set up about 25 years ago and have the aim to extend the smallholders’ professional knowledge, particularly in a self-help context. Schools are set up in the developing world mainly by the Food and Agriculture Organization of the United Nations (FAO). The method for examination is called partial identification analysis. Its aim is to assess the impact of Farmer Field Schools on the farmers’ crop productivity and livestock production in three sub-Saharan countries (Kenya, Tanzania, Uganda). The partial identification approach with nonparametric bounds is used to isolate the effect of the FFS project based on multiple assumptions. After setting up an Average Treatment Effect (ATE) and arranging the so called no-assumption bounds around it, there will be three nonparametric and rather weak assumptions applied. Those assumptions are used to subsequently tighten the bounds around the ATE (Manski & Pepper, 2000).

Several articles (Diao, Hazell, & Resnick, 2007), (Davis, et al., 2010), (L. & Ulimwengu, 2015)1 emphasize the importance of agricultural productivity in mitigating poverty and enhancing welfare in developing countries. The agricultural sector is of crucial importance in most developing countries. Decisive factors are the dependence of the food security on agriculture and the extent of workforce employed in the agricultural sector. The development of the agriculture in sub-Saharan Africa is of particular interest in the field of development aid, as it is one of the least developed and poorest regions in the world. A majority of sub-Saharan Africa´s population lives in rural areas, which are mainly shaped by agriculture. There are many channels, through which it is possible to enhance agricultural productivity. To mention the most important influences on the agricultural system: agricultural-related policies, Research and Development, agricultural extension, rural infrastructure, connection to markets or credit markets, seeds and other inputs.

1_{The authors examined a positive relationship between health expenditures and the marginal}

productivity of agricultural inputs. This suggest that an increase in the agricultural productivity facilitates progress, which is not limited just on agriculture, enhancing welfare. They managed to calculate a counterfactual deployment of child mortality, which serves as welfare indicator.

(7)

2

There were major changes in agriculture worldwide over the second half of the twentieth century. The changes focused on the adoption of high-yielding varieties (HYV), along with investments in irrigation and fertilizer (Diao, Hazell, & Resnick, 2007). This led to substantial growth in the agricultural sector. Regarding the developing world, many countries in Asia could successfully adopt HYV, enhancing their agro-economic progress, whilst sub-Saharan Africa has been growing mostly due to agricultural expansion. Following Diao et al., the contribution of agriculture to the development process in Africa is of high significance. On the one hand proponents emphasize the role of agriculture as potential growth sector with linkages to non-agricultural sectors, on the other sceptics doubt whether agriculture can provide sufficient overall growth in a more integrated global environment. The conclusion of the research report is that agricultural growth is more effective at reducing poverty than non-agricultural growth, in particular broad-based agricultural growth is more pro-poor than export-led agricultural growth (Diao, Hazell, & Resnick, 2007). How is it possible, given the finite potential for land expansion and an ever increasing population, to ensure food security and stability in sub-Saharan Africa in a sustainable way? There is a consensus about the need of a “Green Revolution” in this region, though is not as easily enforceable as in Asia due to fundamental differences. (e.g. geographical, topographical, social) The main drawbacks for improving the productivity are the relatively small area under irrigation and the underdeveloped infrastructure leading to a rather extensive agriculture. Furthermore, there is broad heterogeneity between regions concerning environmental conditions, farming systems and crops planted. Nevertheless, the expected returns of R&D and extension in sub- Saharan Africa are very high. The return of such investments is estimated to be around 35 percent ((IEG), 2011).

There are several articles focusing on the impact of FFS on the agricultural production in developing countries (Feder, Murgai, & B.Quizon, 2004) (Godtland, Sadoulet, De Janvry, & Murgai, 2004) The previous impact evaluations of FFS’s differ greatly, according to the setting and the applied methods. Results are mixed. Godtland et al. used Propensity Score Matching to control the differences in observable covariates. They were able to indicate a positive correlation between improved knowledge about integrated pest management, gained through FFS participation, and productivity in potato production. Feder et al. used a modified difference-and-difference model to conclude that there was no significant impact on the performance of graduates and their neighbours in an Indonesian FFS set up. The authors present a comprehensive list of preceding studies

(8)

3

which evaluated FFS projects. All of them show positive effects on the performance of the smallholders. Though Feder et al. emphasize that the mentioned studies do not consider econometric problems resulting from non-random project placement. All in all, there were not many assessments of FFSs in Africa, applying an econometric approach and considering a possible selection problem. This thesis is exactly doing that.

The thesis tries to assess an impact of the FFS-project in East-Africa by using a method, which has not been used so far to examine an effect of FFS’s on smallholders’ productivity performance. The approach was established by Manski and Pepper 20 years ago, which leads to a topical way of project assessment. The nonparametric bounds approach does not assume a specified probability distribution of the outcome variables and does not give a precise estimate either. As the applied assumptions with the corresponding bounds are presented step by step, the results are tangible. In contrast to most of the other impact assessments of FFS’s, the nonparametric bounds approach does consider econometric problems resulting from non-random project placement.

The scopes of the final bounds around the Average Treatment Effect (ATE) differ clearly between the countries as well as between measures. The effect of FFS’s on livestock production is determined positively at an earlier stage than the effect of FFS’s on the crop productivity measure. The effect of FFS’s on the crop productivity cannot be determined as non-negative without assuming it. Based on the results, the FFS-project seems to be a successful way to enhance agricultural productivity and production.

The remainder of the paper continues as follows. Section 2 summarizes economic and agro-economic key measures to draw a picture of the East-African countries examined in the thesis. Section 3 describes the FFS approach in general and particularly the one we focus on in the thesis. In the fourth section, descriptive, country-specific tables are provided, presenting households’ mean endowment of covariates, divided by treatment status. Moreover, the basic productivity and production measurements are explained and included in the descriptive tables. Subsequently, in Section 5, the various methodological steps with the corresponding assumptions are incrementally presented and applied. The reader is able to observe gradually the tightening of the bounds, combined with the application of the different assumptions. Section 6 summarizes the conclusions.

(9)

4

2 Background

The countries of investigation are characterized by a high degree of poverty and a large agricultural sector. A general overview of the economic situation and the state of agricultural development is presented in the table below. The table information was taken from the CountryStat database. It is a database for food and agriculture statistics. It was initiated by the Food and Agriculture organization of the United Nations (FAO) and includes different sources. It harmonizes scattered institutional-generated statistical information by several countries, including Tanzania, Uganda and Kenya. Kenya was categorized as a lower middle income country, whereas the other two countries are listed as low income countries by the World Bank. Table 1 lists the HDI index and the GDP per capita for the year 2014 as key parameters of the countries’ economic development (http://www.countrystat.org/Default.aspx, 2016). In addition, it provides some agricultural related measures in the project-participating countries.

The percentage of inhabitants, who were economically active in agriculture in 2014, emphasizes the importance of agriculture in the East-African region. The rough distance to the level in the countries of the Western world is significant. A possible comparison to the development status in the industrial nations contrasts the measure above with the 2014’s percentages of employment in agriculture (% of total employment), which range from 1,1 to 3,5 for the six founding nations of the European Union, respectively Italy, Germany, France, the Netherlands, Belgium and Luxembourg (http://data.worldbank.org/, 2016). As you can see in Table 1, the percentage of the population economically active (% of total population) is much higher for the project-participating countries, though the comparison is not made between identical measures. The high numbers of agricultural employment indicate the low level of development and the scope for a productivity improvement.

The annually demographic growth rate of 2012 as benchmark of the increasing population marks the importance of a sufficient food supply. An adequate organisation of the food security, facing the increasing number of people, demands for an inclusion of the domestic agricultural production. Hence, an adaption of agricultural productivity and production would help to solve the region-specific and omnipresent problem of food provision and lead the way to an assured broad nutrition level, which counts as a key determinant of growth.

(10)

5

Table 1 - Participating countries, information about basic economic measures and population’s activity in agriculture

Variable Kenya Tanzania Uganda

Inhabitants 2014 46,749,000 52,291,000 37,578,900

Human Development Index 0.535 0.476 0.484

Population economically active in agriculture 20,577,000 31,013,000 12,197,000 Ratio of population economically active in

agriculture 44% 59% 32%

Ratio rural population 79% 75.2% 83.6%

Annually demographic growth rate 2012 2.7% 2.7% 3.3% GDP per capita (current US Dollar) 994 966.5 571.67 Agriculture value added (% of GDP) 30.00% 29.00% 25.02% Source: CountryStat Database

3 Farmer Field Schools

Farmer Field Schools were set up as an alternative to the conventional top-down test and verification extension approach, which partly failed to provide solutions for more complex and counter-intuitive problems. It was developed by the Food and Agriculture Organization (FAO) and partners in Southeast Asia about 25 years ago. A typical FFS class has 20-25 members and meets once per week in a local field setting under the guidance of a facilitator. Divided into groups of five, the participants investigate the development of test plots. The plots are partly cultivated by conventional methods and partly by experimental methods, which are considered as the “best practices”. Together with the trained facilitator, participants observe agro-ecological key elements and discuss their findings at a conclusive plenary session each week. The learning-by-doing approach is perceived to enhance organizational skills, decision-making processes and farm-based experimentation (http://www.fao.org/agriculture/ippm/programme/ffs-approach/en/, 2016). Graduates are allowed to organize FFS’s as facilitators and are encouraged to

spread their knowledge and experiences. This is seen as cost-effective way to enhance knowledge dissemination and financial sustainability (Feder, Murgai, & B.Quizon, 2004).

The general extensional content, conveyed by FFS, tries to increase productivity while protecting the environment. A decentralized network of Farmer Field Schools, enhancing farmers´ participation, allows for an implementation at different locations.

(11)

6

This is important in case of agro-ecological differences, in particular for the establishment of new seed varieties. If the testing and selection of seeds take place on the farms, it enables an involvement of the farmers in the decision-making and the research. So besides the better adaptation to location-specific circumstances, there is an extension effect for the farming community. The trainer is chairing the meetings and interacts with the participants rather communicatively than instructively. FFS’s aim to develop farmer’s analytical skills and critical contemplation. The approach is created to enhance creativity instead of simply transmitting facts (Feder, Murgai, & B.Quizon, 2004).

Although the effectiveness and advantages of participatory development are ambiguous in the overall context, in case of R&D for agriculture in sub-Saharan Africa, it seems appropriate. As there is a variety of variables affecting the agricultural outcome, it is difficult to prove the effectiveness of enhancing farmers’ education. Kristin Davis et al. conclude in their paper that the impact of FFS participation on productivity and poverty alleviation is potentially higher for those farmer with a lower level of education (Davis, et al., 2010). Generally, education is expected to drive the agricultural development. Davis et al.’s results emphasize the significance and the potential lying beneath accelerated education for farmers.

Organized by the International Food Policy Research Institute (IFPRI), Farmer Field Schools were set up in various districts in Tanzania, Kenya and Uganda. The FFS project was established by the Food and Agriculture Organization in 1999 and focused on the implementation of Farmer Field Schools in eight pilot districts in Kenya, Tanzania and Uganda. In addition to the initiator FAO, the International Fund for Agricultural Development and the Ministry of Agriculture and Livestock Development supported the project. The project was separated into two phases. Integrated production and pest management were the topics in the first period, which started in 1999 and ended in 2002. In the first period, the initiators focused on an improvement of the responsiveness to local specifics, the setup of a network for exchanging empirical content and the evaluation of the FFS approach.

The second period of the project had a different substantial objective. It started in 2005 and ended in 2008. According to Davis et al., the topics of the project were devising self-financing mechanisms, broadening the scope of extension services, encouraging demand-driven and market-oriented services and strengthening farmer organizations and networks. Devising self-financing mechanisms in developing countries is of particular importance, as many regions suffer from an insufficient credit and saving system. In the

(12)

7

context of the low development level, it is even more useful to finance by own sources instead of trusting the alternatives if those exist. The other topics mentioned focus on informational extension. On the one hand, it is a distinct goal of the FFS to broaden the farmers’ knowledge, which seems to be expandable, considering the level of development and the corresponding level of education in rural Sub-Saharan Africa. On the other hand, the project tends to set up a network between the FFS and farmers as well as between the farmers themselves to create a viral base of informational exchange. All in all, the project’s setup aims on the creation of self-help mechanisms.

4 Data

The longitudinal dataset contains a household survey including information about participation in FFS, socio-economic characteristics and access to agricultural services (markets, extension, credit). It is important to mention that the areas of data collection were divided into agro-ecological zones. Moreover, households were stratified by the density of FFS in districts and types of zone. Due to the structure of the data, it is not possible to evaluate long-term effects of the project. However, there is a basis for comparison between countries. One issue in evaluating the dataset is to draw a comparison between households which participated in the FFS-project and those which did not. As already mentioned participation in the project was not randomized; there was a self-selection process for participants, which resulted in a selection bias. The underlying dataset was already evaluated in the previously mentioned paper by Davis et al. (2011). The article draws on a combination of econometric methods to assess the effect of farmer field schools in East Africa. Davis et al. examined the effect on crop productivity, livestock production and agricultural income. A correction for non-random placement of treatment groups by using an instrumental variable (IV) was not applied. The determination of a valid IV in the underlying dataset is difficult, as there is no valid, simple variable to find.

Kristin Davis et al. used in their evaluation a double-difference estimator in combination with various matching procedures, including propensity score matching (PSM). The matching methods aim to clarify the issues resulting from the non-randomized design. Literature, which focuses on matching methods, is mixed. There are

(13)

8

papers which question the reliability and representativeness of the results of the matching methods (LaLonde, 1986), (King & Nielsen, 2016). Critics claim that too few observations could lead to increased error-proneness in case of the implementation of PSM. King and Nielson state that the application of PSM increases imbalance, inefficiency, model dependence and bias. Matching techniques assume unconfoundedness, which is a rather strong assumption.

As a first step, two measures were created to be able to assess the performance of the households. The measures are done on aggregated household level, values of production and the corresponding costs, all of which are summarized for each household.

The following formulae were used to calculate the crop productivity and the livestock production: Crop productivity: (𝑇𝑜𝑡𝑎𝑙 𝑣𝑎𝑙𝑢𝑒 𝑜𝑓 𝑐𝑟𝑜𝑝 ℎ𝑎𝑟𝑣𝑒𝑠𝑡 – 𝑇𝑜𝑡𝑎𝑙 𝑜𝑓 𝑖𝑛𝑝𝑢𝑡 𝑐𝑜𝑠𝑡𝑠 𝑖𝑛𝑐𝑙𝑢𝑑𝑖𝑛𝑔 𝑠𝑒𝑒𝑑 𝑐𝑜𝑠𝑡𝑠, 𝑒𝑥𝑐𝑙𝑢𝑑𝑖𝑛𝑔 𝑓𝑒𝑟𝑡𝑖𝑙𝑖𝑧𝑒𝑟 𝑐𝑜𝑠𝑡𝑠) / 𝑐𝑟𝑜𝑝 𝑎𝑐𝑟𝑒𝑎𝑔𝑒 (1) Livestock production: 𝑇𝑜𝑡𝑎𝑙 𝑣𝑎𝑙𝑢𝑒 𝑜𝑓 𝑙𝑖𝑣𝑒𝑠𝑡𝑜𝑐𝑘 𝑜𝑤𝑛𝑒𝑑 – 𝑇𝑜𝑡𝑎𝑙 𝑜𝑓 𝑙𝑖𝑣𝑒𝑠𝑡𝑜𝑐𝑘 𝑐𝑜𝑠𝑡𝑠 (2)

The crop productivity and livestock production measures were calculated for the years 2005 and 2008. In 2005, the second part of the FFS-project was initiated. This second stage is the one we want to examine. A follow-up survey was performed by the project organisers in 2008. The two points in time were chosen to examine the effects on the agricultural productivity and production between the treated and untreated. We abandoned an examination by a difference and difference approach. So, merely the baseline data is used for a verification of the assumptions by investigating patterns in the data and then drawing conclusions. The time-specific effect on the measures is omitted, as it is not relevant for this particular investigation.

The crop productivity and livestock production measures in the different countries as well as between treatment and control group clearly differ. In Kenya the non-participating households achieve higher crop productivity-levels on average, whereas in Tanzania and Uganda it is the other way around (See Table 2). The level of livestock

(14)

9

production is balanced on average in the Kenyan sample at baseline. Moreover, the average of livestock production is higher for non-participating households in Tanzania and lower for non-participating households in Uganda. After the treatment, the average level of livestock production is higher for FFS-participating households in all of the three countries in 2008. (See Table 2) Without an application of any econometric approach, this fact supports the positive effect of the FFS-project on the livestock production in the participating sub-Saharan countries.

Unfortunately, the available dataset did not include information about the amount of fertilizer used. This made a comprehensive consideration of all costs in the crop productivity impossible and constitutes a flaw in the dataset. As a result, the absolute values are not entirely representative. Setting the changes in the measures in relation to their absolute values is also not advisable, as the incomplete measurement of costs and the consequential possible unreliability of the absolute values mean that relative measures could be defective. It is necessary to assume a similar amount of fertilizer used by the treatment and control group to make a comparison valid, but a different amount of fertilizer may have been used. As an example, the FFS-participants could be advised to use high yielding varieties with a potentially high need of fertilizer. At the same time, non-participants could still drill with traditional seed varieties. The cultivation of traditional seeds potentially requires less fertilizer, resulting in a different usage between test groups. The differences between the two subgroups would lead to a bias in the results. As the initiators focused more on setting up networks and mechanisms to support self-help instead of the introduction of new seed varieties, we assume a balanced usage of fertilizer over the test groups.

The value of livestock products was calculated net of the livestock-related expenses; information about both are included in the datasets. It is possible to divide the value added in the crop production through the area tilled by the farmers. The standardisation could be distorted, due to differences in soil fertility or in local circumstances for different areas or for different households. We assume a balanced level of soil fertility as most households are close together, leading to a homogenous agro-ecological structure. This assumption is supported by the fact that the households were chosen in regard of similar agro-ecological circumstances, including biophysical (rainfall, topography, etc.) and socioeconomic (farming systems, etc.).

The crop productivity measure is standardised, whereas the livestock production cannot be standardised. It is possible to calculate a number of livestock units owned per

(15)

10

household, though it is advisable to divide the net value of livestock production through it. Livestock units are not suited as a unifying denominator, as those are not precise and vulnerable to generalize between the kind of animals.

Before applying the methodology and taking a look at the results, there are some descriptive statistics presented, including outcome and control variables. For a couple of control variables, there are statistically significant differences between participants and non-participants. There was a mean comparison t-test applied to detect differences on the common significance levels for the listed variables. The test considers the subgroups’ unequal variances and uses Welch’s approximation. The Welch’s t-test is based on the student t-test and allows for unequal variances and sample sizes. As the Student t-test, the Welch t-test assumes normality of the samples. The test was done for each country separately.

There are statistically significant differences for all countries, though the kind of variables differ. As you can see in the tables below, Kenyans treatment households have more favourable characteristics on average. Education levels of the household head and his/her spouse are significantly higher, as well as the proportion of members in credit/savings organizations or in any other organization or group. The circumstances for Tanzania are similar, as the part of members in other organizations of both types, savings/credit and general is also significantly bigger for the treated. While the education levels are rather balanced, the distance to major market towns is on average shorter for the treated. The treated households in Uganda have also a higher portion of members in other organizations, though the spouse’s level of education is lower on average. Furthermore, non-participating households have a significantly bigger acreage and are in shorter distance to the next major market town. The crop productivity measure and livestock production were included to give an overview of the mean performance of the test groups at closure.

(16)

11

Table 2 - Comparison of covariates and outcome variables at closure by treatment status, tables are divided by country

Kenya Variable Mean Non-FFS S. D. Mean FFS S. D. Sig. Diff.

Education level, household head 2.40 0.75 2.19 0.71 ***

Education level, household head's spouse 2.16 0.61 2.02 0.62 *

Age of the household head 48.48 12.43 50.57 11.86

Acreage of farm 3.27 3.44 3.40 3.07

Member to credit or savings organization 0.59 0.49 0.29 0.46 *** Distance to nearest all season-road in km 9.66 7.91 9.40 7.07 Distance to next market town in km 5.93 4.90 5.21 4.71

Member to any organization or group 0.16 0.37 0 0 ***

Off- farm income 0.16 0.08 0.13 0.06

Occupation of the household head 0.76 0.43 0.72 0.45

Family size 7.00 3.09 6.50 2.51 *

Sex of household head 0.82 0.39 0.84 0.36

Crop productivity 2005 in Kenyan Shilling 14,517 199,198 -17,130 49,890 * Crop productivity 2008 in Kenyan Shilling 27,926 115,882 -22,586 290,131 * Livestock production 2005 in Kenyan Shilling 8,544 19,102 8,516 20,282 Livestock production 2008 in Kenyan Shilling 18,788 115,566 28,486 47,928

Member to credit or savings organization, Member to any organization or group and Off- farm income: No = 0, Yes = 1

Occupation of the household head: 0 = no farmer, 1 = farmer Sex of household head: 0 = female, 1 = male

The statistical significance is described as follows: * significant at the 10%-level, ** at the 5%-level, * at the 1%-level Tanzania Variable Mean Non-FFS S. D. Mean FFS S. D. Sig. Diff.

Education level, household head 2.07 0.59 2.06 0.50

Education level, household head's spouse 2.00 0.36 1.95 0.38

Age of the household head 48.54 14.81 45.44 13.43 *

Acreage of farm 1.99 1.81 1.81 1.61

Member to credit or savings organization 0.81 0.39 0.64 0.48 *** Distance to nearest all season-road in km 9.85 15.35 12.43 16.91 Distance to next market town in km 15.43 21.25 11.63 15.14 * Member to any organization or group 0.07 0.26 0.01 0.10 **

Off- farm income 0.14 0.08 0.13 0.09

Family size 5.84 2.90 6.10 2.76

Crop productivity 2005 in Tanzanian Shilling -83,537 1,621,587 173,177 796,693 Crop productivity 2008 in Tanzanian Shilling -1,938,345 3,018,572 280,343 890,644 Livestock production 2005 in TZS 1,442,057 9,962,108 244,854 2,228,966 Livestock production 2008 in TZS 548,385 2,719,440 717,179 3,780,017

(17)

12

The statistical significance is described as follows: * significant at the 10%-level, ** at the 5%-level, * at the 1%-level Uganda Variable Mean Non-FFS S. D. Mean FFS S. D. Sig. Diff. Education level, household head 2.50 0.73 2.38 0.76 Education level, household head's spouse 2.15 0.86 1.96 0.86 **

Age of the household head 44.88 12.65 45.48 11.80

Acreage of farm 5.93 6.42 4.82 4.98 *

Member to credit or savings organization 0.63 0.49 0.32 0.47 *** Distance to nearest all season-road in km 2.29 1.73 3.90 15.18 Distance to next market town in km 3.94 4.68 5.06 7.04 ** Member to any organization or group 0.24 0.43 0.02 0.13 ***

Off- farm income 0.72 0.45 0.71 0.45

Family size 7.46 4.06 7.75 4.53

Crop productivity 2005 in Ugandan Shilling 72,348 315,514 582,451 5,102,011 Crop productivity 2008 in Ugandan Shilling 234,523 793,054 938,898 6,899,384 Livestock production 2005 in UGX 108,787 777,288 223,734 1,333,114 Livestock production 2008 in UGX 237,219 1,742,981 753,889 2,801,873 **

The statistical significance is described as follows: * significant at the 10%-level, ** at the 5%-level, * at the 1%-level

Incidentally, the impact of the listed characteristics on the outcome variables is not straightforward. In particular, agricultural production is influenced by numerous variables, which cannot be defined in detail. Table 2 is useful, because the fact that there are statistically significant differences in control variables between test groups suggests further distinctions of unknown or unobserved ways between those groups. Concerning the differences between control and treatment groups, we have to be aware of the possibility of unobservable differences with a potential to bias the results.

The baseline measurements are also expected to be correlated with the project-following measurements. Differences in the later taken outcomes could be more convincingly attributed to project-specific changes in case of balanced baseline outcomes between test groups. Nevertheless, the nonparametric approach to identify the project’s

(18)

13

effect allows for differences between test groups. Eventually the distinction between test groups is also advantageous, because it enables the application of the Monotone Treatment Selection assumption and the Monotone Instrumental Variable assumption. Those assumptions require a distinguishable classification.

5 Empirical approach and findings

We will focus on the production measures of the year 2008, after the FFS-project was finished. The average treatment effect (ATE) between the treatment group and the control group, respectively between the FFS-participating and the non-participating households, serves as foundational approach to estimate an effect attributable to the project. The measures are based on the country-specific datasets of Kenya, Uganda and Tanzania.

𝐴𝑇𝐸 = 𝐸[y(t = 1)] − E[y(t = 0)] (3)

The Average Treatment Effect describes a difference between mean potential outcomes for each of the country-specific samples. To indicate the effect of the Farmer Field School projects in the three East-African countries, we define the treatment variable t. The treatment is described as participation of a farming household in the farmer field school project. Hence, t = 1 describes the state of school attendance of one of the household members and t = 0 denotes the state, in which nobody in the household visits a farmer field school. So the first mean potential outcome treats the sample as a comprehensively treated one, whereas the second assumes all units as untreated. The mean potential outcomes always include the whole sample, which is defined per country. Hence, we receive for each measure, crop productivity as well as livestock production, the average effect by the treatment.

Concerning formula 3, Y denotes the outcome variable, whereas the E is set for expectations. The ATE describes the difference between the mean potential outcome of the sample that would emerge if all households would be treated and the mean potential outcome of the sample that would emerge if no household received the treatment. This difference can be read as the causal effect of treatment, as the only change between the

(19)

14

samples is the switching of t from 0 to 1. The evaluation of the ATE suffers the problem of missing outcomes (Manski C. F., 2003).

As already mentioned, the non-randomized design of the project complicates a reliable evaluation of the project and describes the main impediment in identifying the effect of the Farmer Field School project on the economic activity of the farmers. Due to the self-selection process in choosing the participating farmers, there is the threat of a selection bias. The possibility of a selection bias hampers a straight difference-in-difference estimation of the treatment effect. The threat of selection bias is supported by the statistically significant differences in some of the control variables, presented in table 2.

It is not possible to observe a treated and an untreated outcome for one household. Either the household received a treatment or it did not. There cannot be a comparison between a treatment and a control outcome for the very same household. If we would like to estimate an effect of the treatment to draw a conclusion or maybe to infer policy implications, an average treatment effect is the best-suited.

A randomized experimental design would allow for the assumption that the observed difference between the treated and untreated corresponds approximately with ATE. As this is not the case, it is necessary to use an alternative estimation method, one, which considers the non-randomized experimental design, that led to observable statistically significant differences and potentially unobservable ones between the treated and control group.

There are many factors influencing the outcome variables. Attributing changes in the productivity variables solely to the FFS project itself would be negligent. Several methods were developed over the last decades for measuring impacts of treatments in non-randomized projects. The goal is to consider other sources of influence, observable as well as unobservable, on the outcome variables. For instance, there could be a change in market prices with a potential impact on the prevailing farming practices. If those changes affect the defined productivity measures or possibly lead to general adaptions, which were solely expected for the treated, the effect of the project gets blurred. We are able to observe some distinctive differences and try to draw inferences from those to give a lower and upper bound of the treatment effect.

Let’s denote d as a dummy variable, describing the realized treatment status of the individual household. If there was somebody in the household, observably attending a Farmer Field School 𝑑 = 1. If there was no participant in the household 𝑑 = 0. D as a

(20)

15

variable is different to the variable t, as a hypothesized state of a household is not expressed by d. Using the law of iterated expectations and including the realized observations, we split the formula for the ATE into the following into the following terms:

𝐴𝑇𝐸 = 𝐸[𝑦(𝑡 = 1|𝑑 = 1)] ∗ 𝑃(𝑑 = 1) + 𝐸[𝑦(𝑡 = 1|𝑑 = 0)] ∗ 𝑃(𝑑 = 0) − [𝐸[𝑦(𝑡 = 0|𝑑 = 1)] ∗ 𝑃(𝑑 = 1) + 𝐸[𝑡 = 0|𝑑 = 0] ∗ 𝑃(𝑑 = 0) (4)

The terms P(d=0) and P(d=1) are known. P describes the ratio of the treated and untreated respectively to the whole sample. The terms, describing an outcome for t=d are also known, as those which are observed. Hence, the only unknown terms are 𝐸[𝑦(𝑡 = 1|𝑑 = 0)] and E[𝑦(𝑡 = 0|𝑑 = 1)]. To estimate a range for the ATE value we replace the unknown terms with upper and lower values to construct bounds. Gradually, taking different assumptions, we tighten the bounds to receive more expressive results. The following bounds are limiting the counterfactual outcomes to be in the range of the observed values and are subsequently used to define bounds for the ATE.

It is important to mention that the ATE and all related following equations might not be representative in a comprehensive manner. Though the ATE tries to internalize the counterfactual outcomes for all observations, the informative inclusion is limited to the sample size. This leads to a limitation of the external validity of the results and the corresponding inferences, in particular for scenarios in different environments. The fact that the sample’s household and village structures can be met in most of the African developing countries speaks for a broader external validity of the findings.

𝑦_𝑚𝑖𝑛≤ 𝐸[𝑦(𝑡 = 1)|𝑑 = 0] ≤ 𝑦_𝑚𝑎𝑥 (5) 𝑦𝑚𝑖𝑛≤ 𝐸[𝑦(𝑡 = 0)|𝑑 = 1] ≤ 𝑦𝑚𝑎𝑥 (6)

The bounds are called “no assumption” bounds (NOA) by Manski, because the probability that the mean counterfactual outcomes are actually out of the NOA range is not distinguishable from zero (Manski, 1989).

NOA bounds with an example for the Kenyan participants’ crop productivity in Kenyan Shilling (KES):

𝑈𝑝𝑝𝑒𝑟 𝑏𝑜𝑢𝑛𝑑: 𝐸[𝑦(𝑡 = 1|𝑑 = 1)] ∗ 𝑃(𝑑 = 1) + 𝑦_𝑚𝑎𝑥 ∗ 𝑃(𝑑 = 0) −

(21)

16 = (−22586,37) ∗281 398+ 2004155 ∗ 117 398− (−1085424) ∗ 281 398− 27925,86 ∗ 117 398 = 1.331.347,19 𝐿𝑜𝑤𝑒𝑟 𝑏𝑜𝑢𝑛𝑑: 𝐸[𝑦(𝑡 = 1|𝑑 = 1)] ∗ 𝑃(𝑑 = 1) + 𝑦_𝑚𝑖𝑛∗ 𝑃(𝑑 = 0) − [𝑦𝑚𝑎𝑥 ∗ 𝑃(𝑑 = 1) + 𝐸[𝑡 = 0|𝑑 = 0] ∗ 𝑃(𝑑 = 0) (8) = (−22586,37) ∗281 398+ (−1085424) ∗ 117 398− 2004155 ∗ 281 398− 27925,86 ∗ 117 398 = −1.758.231,81

As it can be seen, the observed outcomes are included in this equation. The observations are summarized and averaged out by treatment status. Subsequently, mean outcomes were weighted by their relative number of observations. The min/max values are also included, using the same weights as for the observable productivity measures, as the counterfactual outcomes are sample-specific and follow the observed outcomes. Replacing the mean counterfactual terms with the largest and smallest observed values makes the bounds as extreme as possible, just considering the information out of the sample. It is a conservative way of calculation, because it is unlikely that the unobserved mean potential outcomes are around the extreme values of the observed maximum or minimum value.

There are assumptions, suited for a tightening of the bounds. First, the least strong assumption is applied. Afterwards, we gradually continue with the stronger ones. This procedure enables the reader to trace the development of the bounds. Step by step, it is possible to observe how the bounds are tightened by the assumptions made. Readers can decide how many assumptions to believe in and which implementation of bounds is still credible, if not all of them are. The bounds are uniformly translated into equivalents to the respective standard deviations. This makes the results comparable.

(22)

17

Table 3 - Results based on no assumption2

Crop productivity bounds

NOA

Kenya [-9.49; 7.18]

Tanzania [-7.75; 11.91]

Uganda [-9.40; 5.62]

Livestock production bounds

NOA

Kenya [-12.69; 5.56]

Tanzania [- 9.11; 3.75]

Uganda [- 6.76; 4.88]

As you can see in table 3, applying no assumptions results in very broad bounds around the different average treatment effects. Bounds at a level between four to twelve times of the respective standard deviation are very large.

Monotone Treatment Selection Assumption (MTS)

The Monotone Treatment Selection Assumption is applicable, if it is credible that the treated group has a higher or lower mean potential outcome than the control group’s hypothesized mean potential outcome under an identical treated status. The same inequality relationship assumed under treatment should hold under no treatment. The comparison of outcome-related characteristics between the treated and untreated households exposed statistically significant differences. If we want to investigate differences between test groups concerning the empowerment, which is theoretically linked to the farming household´s level of productivity, it is advisable to take a look at empowerment-describing control variables.

As the crop productivity measures out of 2005 contradict the findings, it is more promising to base the MTS empirically on the observed baseline productivity measures. We assume that the group with the lower mean productivity measure at baseline, treated or untreated, is expected to have the lower mean potential productivity outcome at closure. The comparison is hypothesized, presuming an identical treatment. The impossible observability of the inference leads to an unproven assumption. To

2_{Figures are shown as multiples of the standard deviations, which were calculated separately for}

(23)

18

substantiate this assumption, I drew a regression of the final crop productivity measure and the livestock production measure to the equivalent measures at baseline. Definitely, it is not a regression between the baseline measure and the counterfactual end point measure, which would serve as evidence for the assumption. But the regression describes the relationship between actual realized outcomes of the very same households at different points in time. A positive relationship supports the assumed consistency in the groups’ classification. Certainly, it is kind of conflicting to draw this regression, as the goal of the thesis is investigating an effect of the FFS project, which is definitely influencing this particular regression. Nonetheless, we use it as a hint. So, the assumption is at least supported, though it is not proven.

For all countries, the coefficients of the drawn regressions are significantly different from zero in a statistical manner. This supports the assumed proximity between the same measures at different points in time. We set up distinct inequality assumptions by country and by measure, respectively for the crop productivity and livestock production measure in Kenya, Tanzania and Uganda. Distinct inequality assumptions are essential, because the connections between treated and untreated differ. Moreover, we apply a t-test to see if the difference between treated and control group is statistically significant. A statistically significant difference would rather maintain distinguishable groups and support the reliability of the MTS assumption. Though a statistically insignificant difference between test groups at baseline does not prevent a trustworthy application of the MTS assumption. As we will see, most distinctions between test groups at baseline are statistically insignificant, except for the Kenyan and Tanzanian crop productivity measure. For Kenya, the treatment group has the lower level of crop productivity on average and the distinction to the untreated is significant at the 5%-level. As an example, we apply the following two formulae for the Kenyan crop productivity measure in US Dollar:

𝐸[𝑦(𝑡 = 1)|𝑑 = 1] ≤ 𝐸[𝑦(𝑡 = 1)|𝑑 = 0] (9) −337,96 ≤ 𝐸[𝑦(𝑡 = 1)|𝑑 = 0]

𝐸[𝑦(𝑡 = 0)|𝑑 = 1] ≤ 𝐸[𝑦(𝑡 = 0)|𝑑 = 0] (10) 𝐸[𝑦(𝑡 = 0)|𝑑 = 1] ≤ 417,86

(24)

19

Equation (9) states that if all households were treated the average crop productivity measure of the 𝑑₀ households would be not lower than the average crop productivity measure of the 𝑑1 households. The same relationship applies for the scenario

of no treatment for any household in equation (10). To generalize the made assumptions, the average crop productivity measure of the untreated households is higher in case of a hypothesized identical treatment. Hence, the MTS assumption is not about the treatment response of the participating households, it is about the selection into treatment and control group and the expected relation of the outcome variables between the groups on average.

For Tanzania and Uganda, it is the other way around. The untreated households have the lower mean crop productivity measure and are subsequently expected to have the lower closing mean productivity measure. For the Tanzanian sample, the distinction is statistically significant, whereas the difference between the Ugandan groups is distinct, but not statistically significant. Hence for Tanzania and Uganda, the same formulae (9, 10) are applied, but the arithmetic operators are reversed.

For the livestock production measure in Kenya and Tanzania, treated households have the lower baseline measurement on average, while it is the opposite for the Ugandan sample households. No country is able to claim a distinction at the 5%- significance level between test groups.

Below, there is an example of the calculation of the MTS bounds. The bounds are for the crop productivity measure of the Kenyan households. Compared to the NOA-bounds, there is a new lower bound for the counterfactual outcome of the untreated and a new upper bound for the counterfactual of the treated applied. Hence, we get a new lower bound for the ATE on the crop productivity measure of Kenyan farming households. Depending on the assumptions, the tightening of the bounds differs between countries and measures. 𝑈𝑝𝑝𝑒𝑟 𝑏𝑜𝑢𝑛𝑑: 𝐸[𝑦(𝑡 = 1|𝑑 = 1)] ∗ 𝑃(𝑑 = 1) + 𝑦_𝑚𝑎𝑥 ∗ 𝑃(𝑑 = 0) − [𝑦𝑚𝑖𝑛∗ 𝑃(𝑑 = 1) + 𝐸[𝑡 = 0|𝑑 = 0] ∗ 𝑃(𝑑 = 0)] 𝐿𝑜𝑤𝑒𝑟 𝑏𝑜𝑢𝑛𝑑: 𝐸[𝑦(𝑡 = 1|𝑑 = 1)] ∗ 𝑃(𝑑 = 1) + 𝐸[𝑦(𝑡 = 1|𝑑 = 1)] ∗ 𝑃(𝑑 = 0) − [𝐸[𝑡 = 0|𝑑 = 0] ∗ 𝑃(𝑑 = 1) + 𝐸[𝑡 = 0|𝑑 = 0] ∗ 𝑃(𝑑 = 0) = 𝐸[𝑦(𝑡 = 1|𝑑 = 1)] − 𝐸[𝑦(𝑡 = 0|𝑑 = 0)]

(25)

20

Table 4 - Results based on no assumption and on the MTS assumption

NOA MTS

Kenya [-9.49; 7.18] [-0.27; 7.18]

Tanzania [-7.75; 11.91] [-7.75; 0.27]

Uganda [-9.40; 5.62] [-9.40; 0.12]

NOA MTS

Kenya [-12.69; 5.56] [ 0.10; 5.56]

Tanzania [- 9.11; 3.75] [ 0.05; 3.75]

Uganda [- 6.76; 4.88] [-6.76; 0.21]

Applying the MTS assumption noticeably tightens one bound of each specific ATE. It depends on the direction of the inequality sign in the specific MTS assumption, how the bounds are tightened. Assuming a higher mean potential outcome for the treated group lowers the upper NOA bound, whilst an assumed higher mean potential outcome for the untreated lifts the lower NOA bound. This way to tighten the bounds is unilateral, as it pulls solely one bound closer to the zero effect level and leaves the other one unchanged. The MTS bounds are wide, considering the fact that there is one unchanged bound for each ATE, which is still situated in a far distance from zero, several times higher than the respective SD.

Monotone Instrumental Variable (MIV)

For a further tightening of the bounds, we can use the Monotone Instrumental Variable approach (Manski & Pepper, 2000). The Monotone Instrumental Variable is different from the Instrumental Variable. The MIV allows for a weakly monotone relation between the instrumental variable and the outcome variable. An independence between instrumental and outcome variable as in the general IV approach is not imperative. More precisely, the independence of the mean response over different values of an observed covariate, required for the application of the IV, is weakened in the MIV approach. It is replaced by the requirement of weak inequalities between the subgroups with different realizations of the covariate. We denote v as MIV in the following equations. Formally, the following relation must hold with 𝑣1 < 𝑣2 < 𝑣3 as realizations of the MIV:

(26)

21

𝐸[𝑦(𝑡 = 1)|𝑧 = 𝑣₁] ≤ 𝐸[ 𝑦(𝑡 = 1)|𝑧 = 𝑣₂] ≤ 𝐸[ 𝑦(𝑡 = 1)|𝑧 = 𝑣₃] (11) and

𝐸[𝑦(𝑡 = 0)|𝑧 = 𝑣₁] ≤ 𝐸[ 𝑦(𝑡 = 0)|𝑧 = 𝑣₂] ≤ 𝐸[ 𝑦(𝑡 = 0)|𝑧 = 𝑣₃] (12)

In theory, the MIV is expected to have an impact on the productivity measures. The three possible variables for the selection of the MIV: education level of the household head, membership to any credit/savings organization and membership to any organization or group. In theory, the education level of the household is positively correlated with the degree of the outcome variables. The household heads, who achieved a higher education level, are expected to make more favourable decisions concerning farming procedures. Due to their better education, they should have a better knowledge about the empirically investigated performance of farming procedures. Based on this, they should choose the better ones. Moreover, they are expected to adapt faster to new findings and to be rather inquisitive about those. Though agricultural performance in itself is a rather unpredictable measure, which cannot be linked solely to education levels. Whereas the membership to any credit/savings organization theoretically leads to an improved liquidity of the members and a potentially informational advantage due to communicative exchange through the group. The same applies for the third variable, membership to any organization or group also leads to communication between farmers and thus exchange of information as an advantage of the group member.

To substantiate the possible MIV’s potential effect on the productivity measures, simple regressions were drawn between the selected variables and the crop productivity measure of 2005 respectively the livestock production measure of 2005 (See Appendix A1). Before drawing the regressions between the livestock production of 2005 and the selected variables, the households with no livestock production were dropped out of the samples. Reasons for households for not owning livestock could be various. Including those observations in the regressions and ascribing the level of their livestock production on the chosen describing variable would not be valid. The inclusion of the households with no livestock production in the regressions would bias the coefficients and the p-values. There is no regression with a coefficient different from zero at the 5%-significance threshold for any of the variables in any of the three countries. Nonetheless, the variable with the lowest p-value seems as the most appropriate MIV. Hence, the variable membership to any credit/savings organization was chosen as MIV for Kenya. The

(27)

22

Kenyan households with a member in a credit/savings organization are expected to have a higher level of crop productivity than the ones with no membership.

For the Ugandan farming households, it is the other way around. The direction of the expected relationship is the opposite of the one in Kenya. Members to any credit/savings organization as well as to any organization or group do have lower baseline measures of the crop productivity measure on average. The findings in the Ugandan data and the assumptions about the data’s composition are contradictory. Attempts to apply the MIV assumption on the crop productivity ATE bounds by using the household head’s education level or membership to a credit/savings organization led to inconsistent results. In case of the application of the household head’s education level, the range of the MTS-MIV bounds was broader than the MTS-bounds. I did not apply the MTS-MIV approach for the crop productivity measure in the Ugandan sample, as the possible MIV’s could not be reasonably applied. Though, we used the covariate education level of the household head as MIV for the Ugandan livestock production. Taking different MIV`s for the remaining country-specific samples is necessary, as based on the selection process there is no variable suited as universal MIV for all three countries.

For Tanzania, the household head`s education level was chosen. The household head’s education is categorized into no education, primary, secondary and tertiary education. There are six Tanzanian farming households in the Tanzanian dataset, whose household head has the level of tertiary education. An inclusion of those data points as a discrete MIV subsample would be rather invalid, because the low amount of observations is insufficient to ensure representativeness. There are three treated and three untreated households with a head, whose education level is tertiary. The group’s means of the outcome variable in the respective subsamples, we would use in the calculation of the MIV bounds, would respectively be based on only three observations. The six observations represent a small portion of the Tanzanian dataset (<2%), leading to a potential biasing effect on the MIV bounds in case of an inclusion as a whole MIV subsample. Otherwise, an exclusion means to drop households with a distinctive property, leading to a distortion of the results. Hence, we included those households in the MIV group of farmers with a secondary education. The FFS approach was not particularly established for farming households, led by a person holding a tertiary education title. Considering this fact, summarizing the observations of different higher kinds of household heads’ education levels in one MIV subsample seems less bold. The

(28)

23

participation in the FFS aims on improving the professionality of those farmers, who did not achieve higher education levels.

We will construct MIV bounds around the ATE by calculating MIV bounds around the mean potential outcomes of the treated and untreated. To receive those MIV bounds, we do a further refinement and create subsamples for each value of v for both treatment statuses. As there are just two possible implementations of being a member of credit or savings organization, we receive four respective subsamples in Kenya. In case of Tanzania’s MIV, concerning the household head’s level of education, there are three possible MIV implementations, leading to six subsamples. There are eight MIV subsamples for the bounds around the ATE on the Ugandan livestock production.

For each of the defined subsamples, we calculate the highest and lowest possible values for the mean potential outcomes. We use the observed mean outcomes, calculated for each implementation of the MIV in consideration of the treatment status, and the min/max values for the unobserved counterfactual outcomes. The max/min values of the country-specific samples were used to estimate the mean potential outcomes (Platzhalter1). For the creation of the subsample-specific bounds, we use equations (9) and (10), which are related to the MTS assumption. Afterwards, we apply equations (11) and (12) to tighten those bounds. Taking a look at equation (11) and (12), we can state that the upper bound of the subsample 𝐸[ 𝑦(𝑡 = 1)|𝑧 = 𝑣₂] can’t be higher than the upper bound of the subsample 𝐸[ 𝑦(𝑡 = 1)|𝑧 = 𝑣3]. Similarly, lower bound of the subsample

𝐸[ 𝑦(𝑡 = 1)|𝑧 = 𝑣₂] can’t be lower than the lower bounds of 𝐸[ 𝑦(𝑡 = 1)|𝑧 = 𝑣₁]. This logic applies for all inequality relationships between the different MIV realizations, but with the same treatment status. If an inequality is breached, we can replace the upper (lower) bound of 𝐸[ 𝑦(𝑡 = 1)|𝑧 = 𝑣] with the lowest (highest) value of any mean potential outcome with higher (lower) value of z.

To get the aggregate MIV bounds for the treated and untreated, we take the subsample MIV bounds, weight them with the relative number of subsample observations and add them up. After the adding up, we get an upper and lower bound for 𝐸[𝑦(𝑡 = 1)] as well as for 𝐸[𝑦(𝑡 = 0)]. Those weighted averages of the MIV bounds are taken to construct the final MIV bounds around the ATE.

(29)

24 𝐴𝑔𝑔𝑟𝑒𝑔𝑎𝑡𝑒𝑑 𝑢𝑝𝑝𝑒𝑟 𝑏𝑜𝑢𝑛𝑑 𝐸[𝑦(𝑡 = 1)] − 𝑎𝑔𝑔𝑟𝑒𝑔𝑎𝑡𝑒𝑑 𝑙𝑜𝑤𝑒𝑟 𝑏𝑜𝑢𝑛𝑑 𝐸[𝑦(𝑡 = 0)] ≤ 𝐴𝑇𝐸 ≤ 𝐴𝑔𝑔𝑟𝑒𝑔𝑎𝑡𝑒𝑑 𝑙𝑜𝑤𝑒𝑟 𝑏𝑜𝑢𝑛𝑑 𝐸[𝑦(𝑡 = 1)] −𝑎𝑔𝑔𝑟𝑒𝑔𝑎𝑡𝑒𝑑 𝑢𝑝𝑝𝑒𝑟 𝑏𝑜𝑢𝑛𝑑 𝐸[𝑦(𝑡 = 0)]

Table 5 - Results based on no assumption, the MTS assumption and the MTS-MIV assumption

NOA MTS MTS-MIV

Kenya [-9.49; 7.18] [-0.27; 7.18] [-0.22; 5.14] Tanzania [-7.75; 11.91] [-7.75; 0.27] [-7.84; 0.26] Uganda [-9.40; 5.62] [-9.40; 0.12]

NOA MTS MTS-MIV

Kenya [-12.69; 5.56] [ 0.10; 5.56] [0.13; 3.73] Tanzania [- 9.11; 3.75] [ 0.05; 3.75] [0.09; 3.64] Uganda [- 6.76; 4.88] [-6.76; 0.21] [0.06; 0.17]

Estimates of the MIV-bounds suffer from finite-sample bias (Manski C. F., 2009). As a consequence, the estimated bounds tend to be narrower than the true bounds. Manski & Pepper used Monte Carlo estimations to spot, that the finite-sample bias decreases with the number of observations. Moreover, the bias increases with the variance of the variable of interest and the number of subgroups, determined by the number of possible realizations of the MIV. The country-specific datasets are divided differently by the different MIV choices. There are just two subgroups for the dummy variable of being in a credit or savings organizations, while there are three for the household head’s level of education. Though the level of the variances is very high, we assumed that the number of subgroups is small enough and the size of the country-specific datasets is big enough to compensate for the high variances.

Monotone Treatment Response (MTR)

As a last step to apply, we make use of the MTR assumption. The application of the MTR assumption and its combination with the two other assumptions correspond to

(30)

25

the approach used by (Manski & Pepper, 2000). It can be reasonably applied in the context of the underlying setup. To be able to apply this assumption, the possibility of a negative impact of the FFS project on the outcome measures has to be credibly denied. It is important to state that the exclusion of a non-negative effect does not rule out a zero impact of the project. Whether the FFS concept is enhancing agricultural productivity and creating positive effects on rural development is ambiguous. Assuming a non-negative effect is less questionable. This specific program is solely aiming on rather simple positively connoted measures. Those measures are precisely an improvement of self-financing mechanisms, broadening the scope of extension services, encouraging demand-driven and market-oriented services and strengthening farmer organizations and networks. The following formula states the non-negative impact of the FFS project:

𝑦(𝑡 = 0) ≤ 𝑦(𝑡 = 1) (14)

Out of formula (9) the two following two are formed:

𝐸[𝑦(𝑡 = 0)|𝑑 = 1] ≤ 𝐸[𝑦(𝑡 = 1)|𝑑 = 1] (15) 𝐸[𝑦(𝑡 = 0)|𝑑 = 0] ≤ 𝐸[𝑦(𝑡 = 1)|𝑑 = 0] (16)

The MTR assumption is applied in two ways. Firstly, and not too far-fetched, it is used to exclude a lower bound below zero for the crop productivity measure and the livestock production in all of the participating countries. The positive lower MTS-MIV bounds of the ATE on the livestock production support the exclusion of the negative effect range. Whilst at the same time, the exclusion of the negative ATE-effect range for the crop productivity measure can’t be supported by the level of the lower MTS-MIV bounds. A further way to apply the MTR assumption is by combining it with the MTS and MIV assumptions. During the process of setting up lower and upper bounds for the mean potential outcomes of the MIV-subsamples, we are able to use the formulae (10) and (11) together with the formulae (15) and (16). Hence, we can apply the MTR assumption in each MIV-subsample. If the final lower MTS-MIV-MTR bounds of the ATE-effect are still negative, we set them to zero, as negative values are excluded by the MTR-assumption.

In case of the Ugandan livestock production, we did not apply the MTR assumption in none of the two possible ways. The lower MTS-MIV bound was already

(31)

26

above zero. An application in combination with the two MTS and MIV assumptions, as explained above, could not be applied, because the inequality relations contradict the findings in the Ugandan data. The range of the mean counterfactual outcomes of the households with a head, who has no education, was empty.

Table 6 - Results based on NO assumption, on the MTS assumption, on the MTS-MIV

assumption, on the MTS-MIV-MTR assumption and for the case of Uganda on the MTS-MTR assumption

NOA MTS MTS-MIV MTS-MIV-MTR MTS-MTR

Kenya [-9.49; 7.18] [-0.27; 7.18] [-0.22; 5.14] [0; 5.14] Tanzania [-7.75; 11.91] [-7.75; 0.27] [-7.84; 0.26] [0; 0.26]

Uganda [-9.40; 5.62] [-9.40; 0.12] [0; 0.12]

NOA MTS MTS-MIV MTS-MIV-MTR

Kenya [-12.69; 5.56] [ 0.10; 5.56] [0.13; 3.73] [0.13; 3.73] Tanzania [- 9.11; 3.75] [ 0.05; 3.75] [0.09; 3.64] [0.09; 3.64] Uganda [- 6.76; 4.88] [-6.76; 0.21] [0.06; 0.17]

The high value of the lower MTS-MIV-MTR bounds of the ATE on the livestock production in Kenya and Tanzania indicates that there is a positive impact of the FFS on the performance of the participating farmers in those countries. The lower bounds are about 13 % and 9 % of the respective 2008 livestock productions’ standard deviations. Though the level of the upper bounds is not explanatory. It corresponds to about 4 equivalents of the specific standard deviation. The actual impact of the project is most likely not detectable in those degrees. The project’s effect on Kenyan livestock production is more precise.

The span of the Kenyan MTS-MIV-MTR crop productivity bounds is as wide as [0; 5.14]. The range of the ATE bounds is large and does not express a precise effect. The same bounds for Tanzania and the Ugandan MTS-MTR bounds are more centred.

The scope of the final ATE bounds differs distinctly. How can we explain the differences in the magnitudes of the final effect ranges between countries and measures? The reasons for that could be diverse. One possible reason is the different directions in

(32)

27

the MTS assumptions or the different choices of the MIV in connection with the datasets structure. It is noticeable that the bounds of the ATE for the crop productivity are the tightest for Uganda, though there was no MIV assumption applied. As mentioned before, Manski & Pepper stated that the application of the nonparametric bounds approach could suffer from finite sample bias (Manski & Pepper, 2000). The authors stated that due to the bias the estimated bounds are tighter than the true ones. Different limitations of the estimated bounds could potentially be traced back on different severities of the finite sample bias. It increases with the variance of the variable of interest and the number of MIV subsamples. Moreover, the bias decreases with the number of observations. Below, in Table 7, I presented the mean country-specific crop productivity measure as well as the mean level of livestock production in US Dollar. The magnitude of the standard deviations in relation to their respective measures differ between countries, but it is not possible to find a pattern, explaining an increased threat of a finite sample bias and justifying the high magnitude of some bounds. The same applies for number of MIV subsamples as well as the number of observations. Those are not differing substantively between countries. It is not possible to detect a relation between the differences in the bounds’ magnitudes and the constellation of the bias-explaining properties. Nevertheless, the finite sample bias could be a reason for the different scopes of the estimated bounds, though we could not find a pattern as a possible hint

.

Table 7 - Mean of outcome variables in USD by country with standard deviation and number of observations

Kenya Tanzania Uganda

mean crop productivity 2008 $ -110.64 $ 124.56 $ 388.68 SD $ 130.13 $ 75.22 $ 169.00 mean livestock production 2008 $ 358.22 $ 550.50 $ 320.66 SD $ 70.43 $ 148.27 $ 74.21