The impact of an Early Childhood Education and Development intervention on primary school test scores : empirical evidence from Indonesia

The impact of an Early Childhood Education and Development

intervention on primary school test scores

Empirical evidence from Indonesia

Daniëlla Vellinga

The aim of this thesis is to evaluate the impact of a large Early Childhood Education and Development (ECED) project that was meant for children between 0-6 years of age and was implemented in 3,000 villages in Indonesia from 2009 to 2012. Using a sample of 310 villages, the impact of the project on test scores of children aged 6-9 in the first four classes of primary school in 2013 will be assessed. Within the project there is variation in age of the children and in the timing of implementation in the villages. Using a difference-in-differences analysis that exploits the differences across classes due to the age of children and differences in the duration of exposure to the program, makes that the causal effect of the program on test scores can be identified. The results indicate that the impact of the program on the attendance to the ECED project increases with the amount of exposure and decreases for children in higher classes. Exposed children that were in class 1 when the test was taken in 2013 and thus were 2-3 years old when the project started, experience the greatest effect: one year of exposure results in 1.1 months of significant extra attendance compared to children that were in class 4. This effect decreases for children from higher classes: 0.9 and 0.3 months of extra attendance for children in class 2 and 3 respectively. No significant impact of the program on the test scores of the children was found.

MSc Thesis

University of Amsterdam, Faculty of Economics and Business MSc Economics, specialization Development Economics Student number: 10654720

Supervisor: Prof. Dr. M.P. Pradhan Second reader: Noemi Peter July 2014

1. Introduction

Early childhood is a period of major importance for further development. Research has established strong associations between cognitive and psychosocial skills, nutrition and health status measured at young ages and later educational attainment, earnings and employment outcomes (Armecin et al., 2006). Children in developing countries are

particularly vulnerable in this sense, since they are exposed to multiple risks such as poverty, poor health and nutrition and deficient care (Grantham-McGregor et al., 2007).

Not only will this affect these children’s subsequent levels of cognition, education and earnings, it can also play an important role in the intergenerational transmission of poverty. Moreover, the failure of children to fulfil their developmental potential and achieve

satisfactory educational levels can have a serious impact on the development of countries as a whole. Disadvantaged children in developing countries who do not reach their developmental potential are less likely to be productive adults, because they tend to have fewer years of schooling and less learning per year in school (Grantham-McGregor et al., 2007). There are economic costs in the form of foregone productivity and wages. If the damaging effects of children’s underdevelopment subsequently negatively affect later outcomes such as

employment, criminality and social integration there may be social costs too (Pradhan et al., 2013).

These links suggest that it is important for both the children in developing countries and for these countries as a whole, to intervene early in life already. In order to help children in ensuring a smooth transition to primary school, a better chance of completing education and a route out of poverty, strong foundations are necessary (UNESCO, 2006). These foundations should be holistic and should include good health, nutrition, cognitive stimulation and a nurturing environment (UNESCO, 2006; 2007). Early Childhood Education and

Development (ECED) programs incorporate such foundations. ECED programs include services for children of ages 0-6 years. Typically, such services may include group programs, home-based day care programs and home visiting or parent education programs and may aim at different aspects of children’s development, such as education, physical care, health or nutrition (Pradhan et al., 2013).

The case for ECED programs is compelling, especially for the most disadvantaged. There is substantial empirical evidence that there are significant benefits of ECED interventions, both

in the short and the long run (e.g. Barnett, 1995; Currie and Thomas, 1995; Currie, 2001; Garces et al., 2002; Schweinhart et al., 2005; Armecin et al., 2006; Berlinski et al., 2008; Berlinski et al., 2009). Research in neuroscience, psychology and cognition has shown that learning is easier in early childhood than later in life and that nutrition and cognitive

stimulation early in life are critical for long-term skill development (Berlinski et al., 2009). In economics, basic human capital theory suggests that children should start formal learning as soon as possible. Early childhood is not only the most effective, but also the most cost-efficient time to ensure that all children develop their full potential (Engle et al., 2011). Carneiro and Heckman (2003) give evidence for a very high rate of return to investments in human capital early in life compared to low returns of investment in later stages of the life cycle. Also it is more cost-effective to support children early on in a preventive manner than to compensate for their disadvantage as they grow older (Currie, 2001).

Despite convincing arguments and evidence, ECED program coverage is still low (Engle et al., 2007). In that sense it is essential to evaluate the impact and strengths and weaknesses of such programs. This is important to inform and advice governments on the effectiveness of the ECED programs they have implemented. Moreover, it can serve to advocate for

expansion of ECED interventions by intervening governments and have a demonstration effect on governments that did not intervene yet.

This thesis will evaluate the impact of a large ECED project that was implemented in

Indonesia from 2009 to 2012. This project targeted an estimated 738.000 children aged 0 to 6 and their parents/caretakers living in approximately 6000 poor communities in 3,000 villages within 50 districts in Indonesia. It was funded by the World Bank, The Netherlands and Indonesia. The overarching aim of the project was to improve poor children’s overall development and their readiness for further education (Hasan et al., 2013; Pradhan et al., 2013).

In this study the focus will be on the latter, by looking at the impact of the intervention on test scores of children aged 6-9 in the first four classes of primary school in 2013. Primary test scores are commonly used to measure educational performance and can give an indication of whether the ECED intervention succeeded in improving school readiness and subsequent attainment. Early test scores are notable predictors of the variation in later educational

achievement, employment and wages (Almond & Currie, 2011).1 Moreover, due to an extensive endline survey held in 2013 there are recent data available now on primary test scores of different age groups and panel data on the time of exposure these children have had to the different ECED services. With these data it can be evaluated whether the Indonesian ECED project served one of its main aims by improving short run educational outcomes and whether time of exposure and age are of influence in this. The results of this evaluation are not only of importance to Indonesia, they can also be meaningful to other developing countries. Indonesia is an example of a country that has begun to achieve middle-income status; it is currently classified as a lower-middle-income economy (World Bank, 2014). Yet, it still faces many challenges including the persistent poverty and its effects on children’s well-being. These challenges are also akin to those faced by many other middle- and low-income countries (Hasan et al., 2013).

Most studies that evaluate ECED interventions do not address whether and how impacts differ depending on age and/or duration of exposure to the intervention (Armecin et al., 2006). In this study, the identification strategy rests exactly on those two features. Within the program, there is variation in age of the children and in the timing of implementation of the project in the different villages since this was done in several waves. The impact of the program depends on whether children had the suitable age to attend the project at the moment it was implemented and on the amount of exposure children have had. Using a difference-in-differences analysis that exploits the difference-in-differences across classes for children of different age and the differences in duration of exposure, makes that the causal effect of the program on test scores for children can be identified.

The results indicate that the impact of the intervention on attendance to the ECED program is positive and increases with the amount of exposure and decreases for children in higher classes. The effect is the greatest for children that are in the class 1 at the time the outcome was measured in 2013: one year of exposure to the program results in 1.1 months of significant extra attendance compared to children that were in class 4 at that time. These children were 2-3 years old at the time the program started. The effect decreases for children from higher classes who were thus older than 3 when the project started: 0.9 and 0.3 months of extra attendance for children in class 2 and 3 respectively. The results indicate as well that the program had no significant impact on the test scores of the children.

1_{Almond and Currie (2011) define reading and math test scores at age 7. Including background variables such}

The remainder of this thesis is organized as follows. Section 2 gives an overview of the existing literature about the impact of ECED interventions on educational outcomes in developing countries, that use a similar identification strategy as this study does. Section 3 first gives some background information on how the current climate in Indonesia is for children and on the design of the Indonesian ECED intervention. Second, it elaborates on the data, identification strategy and model that will be used. In Section 4 the results are presented after which in Section 5 some robustness checks are done to verify the found results. Section 6 concludes and discusses the findings.

2. Literature Review

There have not been many evaluations of ECED programs in developing countries that focus on educational outcomes and account for non-random selection into such programs. There is evidence of positive impacts of programs in the United States2, but while indicative this evidence cannot be generalized to developing countries because of factors such as lower program expenditure, less well-trained service providers, more malnourished children and more constraints on reaching and accessing services (Behrman et al., 2004). There is growing evidence though that ECED programs can be effective in developing countries too.

This section will summarize findings from previous literature in which, similarly to this thesis, the identification strategy rests on age and duration of exposure to determine the effect of ECED programs on educational outcomes.

Behrman et al. (2004) evaluated Bolivia’s PIDI program that provide nutrient inputs and systematic learning environments for poor children aged 6-72 months. They used an

identification strategy based on age and exposure by comparing children in the program for short and longer durations. Propensity score matching methods were used to control for any bias due to the selection into PIDI. The results showed positive effects on children’s growth and psycho-social development which are highly dependent on age and duration of exposure (Behrman et al., 2004).

Berlinski et al. (2009) examined the returns to pre-primary education by taking advantage of a large infrastructure program aimed at increasing pre-school attendance for children between

2

For example the Perry Preschool program (Schweinhart et al., 2005), the Home Instruction Program for Preschool Youngsters (HIPPY) (Baker et al., 1998) and Head Start (Currie & Thomas, 1995; Garces & Currie, 2002).

ages 3 to 5 in Argentina. The intervention was targeted towards poor areas with low pre-primary school enrollment rates. Since the program was non-randomized, the authors

exploited the variation in treatment intensity across regions and cohorts to estimate the effect of expanding pre-primary school facilities on subsequent achievements in primary school. Treatment intensity was measured as the number of rooms constructed. By conditioning on region and cohort fixed effects, the construction program generated plausible exogenous variation in the supply of pre-primary school facilities. The data used did not contain

systematic information on pre-primary school attendance. Therefore, in principle, Belinski et al. (2009) could only estimate the net effect of the supply of pre-primary public school on subsequent school outcomes of children. This reduced-form estimate, or the intention-to-treat effect (ITT) sheds light on the impact of the policy. However, since the take-up rate of the newly constructed places was not significantly different from one, these estimates are also estimates of the treatment-on-the-treated parameter. This analysis sheds light on the academic returns to pre-primary education. Performance was measured based on standardized test scores in Spanish and Mathematics. The evidence suggests that the expansion of pre-primary education was effective in improving academic performance: one year of pre-primary school attendance increased average third grade test scores by 23% of a standard deviation. The gains were larger for students living in more disadvantaged municipalities (Berlinski et al., 2009).

Using a similar identification strategy, Duflo (2001) exploited a major school construction program in 1973 in Indonesia to estimate the effects on educational attainment and earnings. Although this was not an ECED intervention per se, Duflo’s identification strategy and outcomes are of interest for this thesis and are therefore elaborated on. The exposure of a child to the program was determined both by the number of schools built in its region of birth and by its age when the program was launched. More schools were built in regions where enrollment rates were low. Using a difference-in-difference analysis, the causal effect of the program could be estimated under the assumption that in the absence of the program, the increase in educational attainment would not have been systematically different between high and low-intensity regions. After controlling for region of birth and cohort of birth effects, interactions between the age of the individual in 1974 and the intensity of the program in its region of birth are plausible exogenous variables and used as instruments in the wage equation. Each primary school constructed per 1000 children led to an estimated average increase of 0.12 to 0.19 years of education and a 1.5 to 2.7 percent increase in wages, implying a return to education of 6.8 to 10.6 percent (Duflo, 2001).

Armecin et al. (2006) estimated the impact of an ECED initiative by the Philippine government on children’s well-being in the form of better cognitive, social, motor and

language development and health and nutrition. Again the program was targeted towards high risk regions. A difference-in-difference propensity score matching approach was used that allowed obtaining control samples that were as similar as possible to the treatment group in order to control for the non-experimental assignment of the program. Due to variance in the timing of implementation of the project, the duration of exposure to the program across regions was different. Intent-to-treat impacts by child age were examined, and also the length of exposure to the program is assessed. The results indicate that the program has had

important positive impacts on children’s status with some suggestion that duration of

exposure increases the impacts. The effects are particularly important for younger children of below age four (Armecin et al., 2006).

3. The Indonesian Program, Data and Empirical Strategy

3.1 Indonesian Setting

Indonesia has shown positive economic growth in the past few years and despite challenges, it is expected to continue to do so (World Bank, 2014). The poverty rate has declined from 14.2% in 2009 to 11.4% in 2013, but this still means that out of a population of 234 million, almost 27 million people live below the poverty line. Furthermore, almost half of all

households remain clustered around it and are subject to severe risks (World Bank, 2013; 2014). Inequality is rising and pronounced regional differences remain. Educational quality lags behind what is required by a growing economy and there are wide variations in

educational outcomes (World Bank, 2012; Pradhan et al., 2013). An achievement is that primary school enrolment is near 100 percent for both boys and girls. However, dropout rates are high and children fail to progress to higher levels of education, especially those from poorer and rural households (UNESCO, 2012; Hasan et al., 2013). A possible explanation for this is that these children lack proper foundations. This explanation is compelling, since the same disparities can be found in enrolment in any type of early childhood education services prior to primary school: of the 28 million children aged 0-6 years in Indonesia, enrolment was estimated to be only 8%, which is far behind the global average of 24% for low income

countries. Enrolment of children from poorer households and rural areas in such services lags behind significantly compared to children from richer families (Pradhan et al., 2013, Hasan et al., 2013).

The government of Indonesia recognized the problems young children in its country are exposed to and acknowledged the strong case for early intervention. It has implemented several ECED programs and policies since 2001 (Hasan et al., 2013). There are several types of ECED services available in the country, all emphasizing different aspects of child

development. Integrated Health Service Units (Posyandu) give families the opportunity to let their young children to be weighted and measured and for mothers to receive some

information on health, nutrition and child development. Another service provided to the caregivers are the toddler family group (BKB) sessions in which parenting skills are taught. In playgroups (KB) the emphasis is on preparing children for primary school by learning through play. In contrast, kindergarten (TK) usually uses a more formal and ‘academic’ way of teaching to prepare children for primary school. The Ministry of Religious Affairs

provides Islamic kindergarten (RA and TPQ). The government expects that, before entering primary school at age 6, children attend kindergartens (TK or RA) between age 4 and 6 (Hasan et al., 2013).

3.2 The Program

The ECED project in Indonesia that is evaluated in this thesis was modeled on the project in the Philippines as evaluated by Armecin et al. (2006) which was discussed in Section 2. The main goal of the program was to improve poor children’s overall development and readiness for further education within a sustainable quality ECED system. In 2009, block grants of about US$9,000 were given to 3,000 selected villages within 50 districts. The program was rolled out in three waves (batches), creating variation in the time of exposure across villages thus differences in the amount of time that the intervention could have had an impact. The selection of districts was done according to certain criteria, such as low current participation rates in ECED services and poverty. The focus was on rural districts. Villages were selected then based on the highest need for ECED services, by looking at the number of children between ages 0-6 and poverty rates. The project started with community facilitation to guide the communities in using their funds and to raise awareness on the importance and benefits of ECED. Facilitators mostly were well-educated local residents with some experience in

community development. After this, the block grants were provided, followed by a teacher training for staff of the ECED centers (Hasan et al, 2013).

Villages could decide themselves how to allocate the grant, provided that the funds were used for services that complied with a set of essential standards. There also needed to be a strategy to increase the number of poor children served and a plan to improve the quality of

community programs. To assist in and manage this process, the project provided a manual to help villagers. This manual focused on playgroups (KB) and related outreach services with an emphasis on children aged 2-4. Indeed, 90% of the services that were made available under the project were center-based playgroups (KB), mainly new established ones. Children enrolled in these playgroups were between ages 3 and 6. Children are expected to attend formal kindergarten (TK or RA) by age 4, however in the real world this distinction is less clear-cut because of local conditions (Hasan et al., 2013).

Besides the grant given to the villages, funds were made available within the project for monitoring, supervision and evaluation of the program. Part of these funds are used to perform an impact evaluation of whether the project improves children’s development and readiness for primary school (Hasan et al., 2013). In order to do this, a randomized controlled trial of three rounds of data collection (2009, 2010 and 2013) was held at village-level among a sample of 310 villages (100 allocated to Batch 1, 20 to Batch 2, 100 to Batch 3 and 90 as matched control group). A matched control group was constructed because of concerns that the randomization of the sampling didn’t work well enough. The data collection included several internationally recognized standard scale instruments to measure child development and several questionnaires that included access to and use of services (Pradhan et al., 2013; Hasan et al., 2013).

The evaluation is still in progress and has the objectives to establish the short-run impact of the ECED program on early childhood development outcomes and to obtain greater insight into the patterns of this development. The subject of this thesis is embedded in this impact evaluation. The impact evaluation is focused on the entire scope of child development using a broad range of dimensions and is also concerned with the possible effect of quality of ECED services and the possible pathways through which the impact might go (Pradhan et al., 2013). This thesis though will focus on a more quantitative outcome on which ECED services might have an impact by looking at primary test scores.

3.3 Data

The three rounds of data collected from the sample of 310 Indonesian villages in 2009, 2010 and 2013 will be used. Besides data on several features of the children including age and attendance history to the different ECED services for each year, the dataset also includes test scores of children aged 6-9 in the first classes of primary school taken at endline in 2013. The test was taken based on the age of children. Two different tests were given to the age groups 6-7 and 8-9, although there is substantial overlap in questions. The test for the 6 and 7 year olds consisted of 52 questions, the 8 and 9 year olds got 64 questions. The questions were focused on language skills, mathematics and number recognition and included a Raven Test. A total of 12,972 children were tested. Of these observations, several had to be removed because of missing data or because of concerns with the validity and representativeness of the data. This results in 12,453 children in the sample of which 5,287 made the test of the type for the 6-7 year olds and 7,166 made the test for the age group of 8-9 year old3.

The implementation of the project in the subsequent batches and the collection of the data were not synchronized well. Not all villages ended up in the batch as they were intended to. Therefore data was collected on the actual moment villages implemented the project. This resulted in Batch 2 to be omitted as a category and an increase in the number of children that received the project together with either Batch 1 or Batch 3. In the analysis the moment of actual implementation is used. Consequently, 4,219 children in the sample live in villages that were in Batch 1 and were the first to receive the project; 4,747 children were in Batch 3; and the final 3,487 children were in control villages (Batch 5) where the project was not implemented. Table 1 lists the number of children in each batch by age group and gender.

3 _{Actually there are 5,285 children in the age group 6-7 and 7,168 children of age 8-9. This means that two}

children who were 8-9 years old did in fact make the test for the age group 6-7 or made the correct test and got their age registered incorrectly. Since it concerns only two children out of a total sample of 12,453, this error is disregarded in the analysis.

Table 1 – Number of children per batch by age group and gender

Batch 1 Batch 3 Batch 5 (Control) Total Boys Girls Total Boys Girls Total Boys Girls Total

Number of 6-year olds 286 296 582 323 341 664 222 229 451 1,697 [34.30] [39.13] [26.58] [100.00] Number of 7-year olds 630 600 1,230 663 660 1,323 522 513 1,035 3,588 [34.28] [36.87] [28.85] [100.00] Number of 8-year olds 815 784 1,599 927 899 1,826 643 680 1,323 4,748 [33.68] [38.46] [27.86] [100.00] Number of 9-year olds 423 385 808 483 451 934 355 323 678 2,420 [33.39] [38.60] [28.02] [100.00] Total 2,154 2,065 4,219 2,396 2,351 4,747 1,742 1,745 3,487 12,453 [33.88] [38.12] [28.00] [100.00]

Notes: Percentages based on totals by age group and by batch are in brackets.

Table 1 indicates that the sample is well balanced with respect to gender and ages per batch. When comparing the different age groups to each other though, one can see that the number of 6-year olds in the sample is particularly low compared to the amount of children of other age groups in the sample. Data collectors went to schools in the sample villages to take the test. In some villages though, not all schools had a class 1 since not all 6 year-olds are enrolled in primary school yet. Consequently some age groups are better represented in the sample than others. Since this is the case for both the program and control villages it is not considered to be a problem for the analysis.

3.4 Identification strategy

Since the majority of the grants were used to facilitate center-based playgroups (KB), the analysis will focus on the attendance to these playgroups specifically. Thus, when analyzing the impact of the program, in fact the impact of the project playgroups is evaluated. It is important to note that many other non-program ECED services such as kindergartens and regular playgroups were present in both program and non-program villages during

implementation. Some children in the sample could have attended these services instead. There could also be substitution effects present. Therefore an intention-to-treat (ITT) effect

is measured, that is the impact of having the ECED program on average for all children in the age range 6-9 in the project villages, whether or not all these children actually received the treatment. Even though data is available on whether the children attended a project playgroup or a non-program ECED service, the impact of this attendance cannot be evaluated. It is unknown then whether the outcomes would be an effect of the program attendance, or of other essential child or parent characteristics. The characteristics of the children who would enrol in the project services are not observed and could significantly differ from

characteristics of children that do not enrol in project services. If these characteristics are of influence on the outcome variable, the results would be biased. Also, there could be spillover effects to children that did not participate in project services because these children could learn from children that did attend the project.

Characteristics of the samples in the different batches are presented in Table 2. Table 2 – Observed characteristics of the sample by batch

Notes: Standard deviations are in parentheses. * Mean difference between samples of treated (Batch 1 and 3 combined) and control groups is significant at the 10% level. ***Mean difference between samples of treated (Batch 1 and 3 combined) and control groups is significant at the 1% level.

Batch 1 Batch 3 Control Total Mean months of exposure to project at endline (N=12,453)*** 51.01 39.59 0.00 32.37 (1.65) (3.07) (0.00) (20.87) Mean householdsize (N=10,643)*** 4.53 4.54 4.61 4.56

(1.39) (1.38) (1.53) (1.43) Proportion of mothers working (N=10,643)*** 0.44 0.40 0.34 0.39

(0.50) (0.49) (0.47) (0.49) Mean number of females in the household (N=10,643) 2.29 2.32 2.34 2.33

(1.06) (1.07) (1.12) (1.08) Mean number of males in the household (N=10,643)* 2.23 2.24 2.27 2.24

(1.10) (1.06) (1.12) (1.09) Mean of order in birth (N=2,673) 2.02 2.05 2.09 2.05

(1.09) (1.13) (1.14) (1.12) Mean standardized wealth (N=10,709)*** 0.01 0.07 0.23 0.10

(0.96) (0.92) (0.90) (0.93) Mean years of mother's completed education (N=9919)*** 16.84 16.47 13.69 15.81 (26.59) (26.02) (21.97) (25.18)

As can be noted, apart from the mean of order in birth and mean number of females in the household, all other observables differ significantly between the sample of treated children (Batch 1 and Batch 3 combined) and the sample of control children. As discussed before, even though it was intended to use a randomized controlled trial to collect the data, there were some issues with the randomization. Therefore a matched control group was

constructed. However, based on Table 2 one can conclude that the matching did not work properly enough for the control group to be used in a direct comparison. The simple

difference between treated and control group in this case would not only reflect the effect of the treatment, but probably also the differences in characteristics between these groups. Therefore, this paper will not rely on the experimental research design but instead uses a difference-in-differences analysis that rests on the differences in effect for children of different age and with different amounts of exposure. The difference-in-differences method calculates the effect of the treatment on the outcome variable by comparing the average difference in the outcome variable for the treatment group, to the average difference for the control group.By taking a double difference the impact of the significantly different

characteristics between treatment and control group are offset (Stock & Watson, 2011).

The fact that the project was implemented in batches, makes that there is variation in the time of exposure across villages thus differences in the amount of time that the intervention could have had an impact on the children. As can be seen in Table 2, the time of exposure between the two batches and the control group indeed differs significantly. Figure 1 gives an

indication of a positive association of the amount of exposure and the attendance to the project: children who live in Batch 1 have a longer average attendance than children in Batch 3. Children in control villages could not attend the project.

Besides the time of exposure to the project, the age of the child also determines the strength of the effect. The ECED services are meant for children between ages 0-6, after which children are supposed to start primary school. Thus, when being tested in 2013, children of age 6 who were 2 when the program started could have had at most 4 years of attendance to the program since the start of the intervention in 2009. This gradually falls for children of older age until a maximum of 1 year for 9 year olds, who were already 5 years old when the program started. This pattern of a graduate fall in the attendance to the ECED project when age increases is indeed observed, as can be seen in Figure 1.

-0,5 -0,4 -0,3 -0,2 -0,1 0 0,1 6 7 8 9 A ve rag e z -sco re Age 0 1 2 3 4 5 6 7 8 6 7 8 9 A ve rag e att e n d an ce in m o n th s Age Batch 1 Batch 3 Control Figure 1 – Average attendance to project playgroup

Treated children who are of lower age are thus expected to experience the largest impact of the ECED program. However, this doesn’t imply higher test scores for these children will be found: test scores increase with absolute age as well. Standardized total test scores of the Indonesian children are presented in Figure 2.Since the two age groups 6-7 and 8-9 made different tests, test scores are standardized based on scores of children in control villages for these two tests separately. The graph indicates that the older children score better compared to the younger children of the age group of their test: 7-year olds do better than 6-year olds and 9-year olds have higher scores than 8-year olds.

Together, the variation in age and exposure allow a difference-in-differences analysis that can be interpreted as the causal effect of the program on test scores. The key identification

assumption is that there is a common trend in test scores. That is, it needs to be assumed that the initial differences between the treated and control children affect younger children in the same way as they affect older children. For example, the fact that the mean years of

completed education of mothers is higher in the treated sample than in the control sample should have the same effect, if any, on the test scores of younger and older children within the treated sample.

The general idea is to compare children who were in Batch 1 or Batch 3 and thus were

exposed to the project, to children in Batch 5 who were in control villages and did not receive the project. Next to this, children of ages 6-8 who had the suitable age to attend the ECED project during the time of implementation are compared to children aged 9 for whom the attendance to the project should be negligible. Since the emphasis of the program was on children between ages 2 and 4, these children were already too old to attend the program services by the time of implementation. However, in fact the attendance of 9-year olds is not negligible: 2.07 months on average over Batch 1 and Batch 3, see also Figure 1. This latter comparison will therefore be harder to identify.

To circumvent this issue, class is used as an alternative for age. The data show that more than 50% of the sample of 9-year olds is still in class 3. Because of this, there is a large likelihood that these 9-year olds were exposed to the project for a longer period of time before they formally started primary school. Consequently, they have a higher chance on having attended the ECED project services compared to their peers who are already in class 4. Children who were in class 4 in 2013 thus can be assumed to have had less impact of the project compared to children in class 1-3 with more certainty than when comparing children of age 9 to

children of ages 6-8.

A necessary condition to be able to use this variation, is that the fact that more than half of all 9-year olds is still in class 3 must be a general pattern and not something specific to the program villages. It could be that children in program villages in fact postponed the moment they formally started primary school because they wanted to attend the program longer and that therefore so many 9-year olds are still in class 3. This could bias the effects.

In Table 3 the average age in years for each class is shown for both Batch 1&3 (program villages) and Batch 5 (control villages). In each class the average age in the samples of program and control villages are very close to each other. In class 3 the difference in average

age between program and control villages is significant, but the magnitude is very small and the average age in program villages is lower than in control villages. This justifies the assumption that the program did not affect the school starting age of the children4.

Table 3 – Average age in years per class by Program and Control villages

Batch 1& 3 (Program villages) Batch 5 (Control villages) Total Average age Average age Average age

Class 1 6.69 6.71 6.70 (0.68) (0.65) (0.67) Class 2 7.64 7.68 7.65 (0.67) (0.65) (0.66) Class 3*** 8.30 8.37 8.32 (0.60) (0.59) (0.60) Class 4 8.88 8.89 8.88 (0.33) (0.31) (0.33) Total 7.63 7.64 7.63 (0.95) (0.94) (0.95)

Notes: Standard deviations are in parentheses. ***Difference in means between program and control villages is significant at the 1% level.

Table 4 and 5 illustrate the basic idea behind this identification strategy. In Table 4 one can see that the average attendance to the project for children in class 4 is 1.36 months, which is indeed lower than the 2.07 average attendance for all 9-year olds. Children who lived in Batch 1 or 3 were able to attend the project playgroups, leading to a positive difference compared to those who lived in control villages and could not attend. Children who were in class 1-3 in 2013 attended the project for a longer period than children of class 4.

Consequently, there is a positive difference-in-difference estimator of 3.26 more months of attendance to the project for children who lived in the program villages and were in class 1-3 at the time of the test in 2013. Hence, attendance increases with higher amounts of exposure and decreases with higher age. Attendance to established playgroups that were not part of the ECED project is displayed as well. Again a low amount of attendance for children in class 4 can be observed. But here, the children in the control villages attend more months compared

4 _{Another concern that might arise when using the variation among classes, is that children might have to}

repeat or skip classes. Since the test was given based on age of children this issue should be circumvented. Moreover, the repetition rate in primary education in Indonesia was only 3.4% in 2009 and 2.9% in 2010 and 2011 (UNESCO, 2012).

to the children in program villages. This is plausible, since the children in the control villages did not have a choice in which type of playgroup to participate and consequently had to attend the non-project one. This leads to a negative difference-in-difference estimator of 0.80: children who lived in the program villages and were in class 1-3 in 2013 attended the non-project playgroups less. This is probably a substitution effect in the sense that children in the program villages went to the project playgroups instead of the regular ones.

Table 5 shows the standardized average total test scores by class and batch. It can be seen that children who were in class 4 and were in control villages, score better on the test than

children who were in class 4 and in program villages. This is counterintuitive since one would expect a positive effect of the amount of exposure to the program. Instead, this result implies that 1.36 months of attendance to the project playgroup lowers the performance of children on the test by 0.23 standard deviation of a test score. As could be seen in Table 2, the children from program and control villages are significantly different when it comes to

certain observables. These differences are probably reflected in this result. Thus, the simple difference does not only capture the effect of the treatment, but also the effect of the

differences between the two groups. This affirms the use of a difference-in-differences strategy so that this can be accounted for.

Table 5 also reveals that children in class 4 do better than children in class 1-3. This is expected as it reflects the absolute age effect. In the end there is a positive difference-in-difference estimator: children who were in class 1-3 in 2013 and lived in villages of Batch 1 or 3 during the project, on average scored 0.05 standard deviation better. Under the

assumption that the initial differences between the treated and control children affected children who were in class 1-3 in 2013 in the same way as they affected children who were in class 4 in that year, this can be interpreted as the causal effect of the program. This positive effect is driven by the fact that children in class 1-3 in program villages caught up with the older children in class 4 faster than children in class 1-3 in control villages did. Apparently the younger children do experience a positive effect of the program on their scores, but this effect is very small.

Table 4 – Means of attendance in months to project and non-project playgroup by class and batch

Notes: Standard deviations are in parentheses.

Table 5 – Standardized average total test scores by age and batch

Standardized average total test scores

With intervention (Batch 1 & 3) Control (Batch 5) Difference

In class 1-3 in 2013 -0.22 -0.04 -0.18 (1.05) (1.00) (0.02) In class 4 in 2013 0.38 0.61 -0.23 (0.85) (0.76) (0.01) Difference -0.60 -0.65 0.05 (0.03) (0.03) (0.00)

Notes: Standard deviations are in parentheses.

Attendance to project playgroup Attendance to non-project playgroup With intervention (Batch 1 & 3) Control (Batch 5) Difference With intervention (Batch 1 & 3) Control (Batch 5) Difference In class 1-3 in 2013 4.69 0.07 4.62 1.72 2.90 -1.18 (7.94) (0.97) (0.02) (5.05) (6.16) (0.11) In class 4 in 2013 1.36 0.00 1.36 0.50 0.88 -0.38 (3.56) (0.00) _(0.04) (2.18) (2.84) (0.05) Difference 3.33 0.07 3.26 1.22 2.02 -0.80 (0.12) (0.01) (0.00) (0.07) (0.10) (0.00)

3.5 Model

To further analyze and evaluate the impact of the ECED program, the identification strategy can be generalized into a reduced-form regression. The following model will be estimated to analyze the impact of the program on attendance:

( ) ∑

∑ ( )

where is the attendance to ECED service j of child i, is a dummy that indicates in which class the child was at the time of the test in 2013 where class 4 is left out as a reference group, is the number of months of exposure to the program children in village k had based on the batch the village was in, is a vector of controls and is an error term. is either a district fixed effect to control for unobserved factors that differ per district but not over time, or a village fixed effect to control for unobserved factors that differ per village but not over time. Districts are uncorrelated with the time of exposure the children in the different batches had and therefore the parameter on can be identified when using district fixed effects. When using village fixed effects, this parameter cannot be identified since villages are collinear with the amount of exposure. However, the parameters of interest are , which will still be identifiable. These estimates can be interpreted as the impact of the program on attendance to the ECED service for a given class. The coefficients can be interpreted as a trend term: regardless of the class a child is in, in 2013 it will have had a fixed amount of attendance to any ECED service before it went to primary school. is expected to be positive for the ECED services that are part of the project, since a longer amount of exposure to the project should result in more attendance. For the project playgroup the parameters are supposed to be decreasingly positive compared to the reference category class 4: the higher the class, the less positive the impact is. The positive effect of the program on the attendance to the project playgroup will be the largest for children in class 1, since they were in the appropriate age range when the program was implemented. The higher the class, the smaller this effect will be, since these children reached the school starting age at an earlier point in time during implementation and therefore have had less chance to attend the project.

The impact on test scores is evaluated using a similar model: ( ) ∑ ∑ ( ) ∑

Here is the standardized test score of child i. To control for the effect absolute age has on the test score, is a dummy that indicates the age of child i at the time of the test in 2013 where age 6 serves as reference category. As in regression (1), is a dummy that indicates in which class the child was at the time of the test in 2013, is the number of months of exposure to the program children in village k had based on the batch the village was in, is either a district fixed effect or a village fixed effect, is a vector of controls and is an error term.

The coefficients will be decreasingly negative for higher classes. Given age, children in higher classes will score better on the test, but worse than the reference category class 4 which will score the best. is supposed to be positive, since a longer amount of exposure to the project should be beneficial for the educational results of the children. The parameters of interest are , which capture the impact of the program on test scores for a given class. This impact is expected to be decreasingly positive for children who are in a higher class in 2013 compared to the reference group class 4. Children in class 1 are expected to experience the highest amount of impact. Children who are in higher classes in 2013 had a shorter available period to attend the project and will therefore have a lower impact of the program on their test scores compared to the children in lower classes.

4. Results

4.1 Effect on attendance

The results of regression (1) for the playgroups that were part of the Indonesian ECED intervention are in Table 6.

Table 6 – Effect of the program on months of attendance to project playgroup

Months of attendance to project playgroup

(1) (2) (3) Class 1 0.081 -0.146 0.250 (0.515) (2.413) (0.484) Class 2 0.052 0.667 0.270 (0.504) (2.346) (0.476) Class 3 0.178 0.741 0.171 (0.518) (2.406) (0.486) Exposure 0.047*** 0.117** (0.012) (0.046) Class 1 * Exposure 0.092*** 0.096* 0.090*** (0.013) (0.051) (0.012) Class 2 * Exposure 0.075*** 0.060 0.069*** (0.013) (0.053) (0.012) Class 3 * Exposure 0.029** 0.015 0.029** (0.013) (0.053) (0.012)

District fixed effects Yes Yes No

Village fixed effects No No Yes

N 10,688 7,678 10,688

Notes: Standard deviations are in parentheses. *Significant at the 10% level. **Significant at the 5% level. ***Significant at the 1% level. Column (2): sample of treated children (Batch 1 and 3).

Column (1) depicts regression (1) using district fixed effects only. An extra month of exposure to the project results in nearly 0.05 months of significant extra attendance to the project playgroup. Since the average months of exposure to the project was 51.01 months for children in Batch 1 and 39.59 for children in Batch 3, on average they attended the playgroup for respectively 2.4 months and 1.9 months extra compared to children that were not exposed. The estimates on the interaction terms between class and exposure that depict the impact of

the program for the different classes, show a significant decreasingly positive impact as expected: the higher the class a child was in in 2013 compared to class 4, the less positive the impact. This is consistent with the idea that children in higher classes had less opportunity to attend the ECED services.

The significant estimates on the interaction terms in Column (1) indicate that children that were in class 1 in 2013 and lived in villages of Batch 1 which had an average exposure of 51.01 months, attended the project playgroup for 4.7 months longer compared to children that lived in Batch 1 but were in class 4. For children in class 2 and 3 this amount decreases to 3.8 and 1.5 months respectively. In general, compared to children in class 4, one year of exposure to the project results in a significantly longer amount of attendance to the project playgroup of 1.1, 0.9 and 0.3 months for children that were in class 1, 2 and 3 in 2013 respectively. Column (2) shows the results of the same estimation but only for the sample of children in program villages (Batch 1 and 3). The estimated effect of exposure is even higher now: 0.12 months more attendance to the project for each extra month of exposure. The parameters on the interaction terms are decreasingly positive again, although they lose significance

compared to estimation (1). The results in Column (3) are those of an estimation where village fixed effects are used to control for unobserved factors that differ per village but not over time. Since villages are collinear with both districts and the amount of exposure, the parameters on these estimates cannot be identified in this regression. Using village fixed effects results in estimates of the impact of the program that are of similar order and magnitude as with using district fixed effects.

4.2 Effect on test scores

According to the results in Table 5, children that were in control villages during the project appeared to perform better on the test than children that were in program villages. To

illustrate this outcome, Figure 3 shows T-scores5 of the different classes by batch. Indeed it is clear from this figure that in all classes, children that were in control villages had higher test scores than children in program villages.

5 _{T-scores are used instead of standardized scores (Z) since they depict the relations between class and test}

scores and exposure and test scores in a more appealing way. T-scores are calculated using the formula: T= (Z*SD)+mean

Figure 3 – Total test scores of children by class and batch (T-scale)

This effect can be found in the regression results of regression (2) as well. Table 7 shows that the amount of exposure has no significant effect on the standardized average total test score. The estimates for the interaction terms that depict the impact of the program are of very small magnitude and not significant. Using the sample of children in the intervention villages only (Column (2)) or using village fixed effects instead of district fixed effects (Column (3)) does not change these results. Thus, children that are exposed to the ECED program attend the project significantly longer, but do not experience any effect of this on their test scores. The estimates on the class variables are significant and in the expected direction. Holding age constant, children in higher classes perform better with class 4 having the highest test scores. The estimates on the age variables cannot be interpreted, since classes consist of children of different ages and both age and class are included in the model.

0 5 10 15 20 25 30 35 40 45 1 2 3 4 T-sco re Class Batch 1 Batch 3 Control

Table 7 – Effect of the program on standardized average total test scores

Standardized average total test score

(1) (2) (3) Class 1 -1.436*** -1.584*** -1.548*** (0.072) (0.267) (0.070) Class 2 -0.860*** -0.904*** -0.925*** (0.068) (0.269) (0.066) Class 3 -0.407*** -0.389 -0.400*** (0.068) (0.276) (0.065) Exposure -0.002 -0.006 (0.002) (0.005) Class 1 * Exposure 0.002 0.006 0.001 (0.002) (0.006) (0.002) Class 2 * Exposure -0.001 0.001 -0.001 (0.002) (0.006) (0.002) Class 3 * Exposure -0.000 -0.001 -0.001 (0.002) (0.006) (0.002) Age 7 0.233*** 0.255*** 0.170*** (0.028) (0.033) (0.027) Age 8 -0.428*** -0.372*** -0.555*** (0.033) (0.039) (0.032) Age 9 -0.474*** -0.409*** -0.630*** (0.039) (0.047) (0.039)

District fixed effects Yes Yes No

Village fixed effects No No Yes

N 12,453 8,966 12,453

5. Robustness of results

The fact that the regression results show that the ECED program had no impact on the test scores of children, appeals for further verification of this result and for analysis into the mechanism behind it.

5.1 Adding controls

First, it is verified whether adding control variables to the estimation changes the results. Table 2 indicated that certain characteristics of the household differed significantly between program and control villages. These characteristics are: the mean size of the household; the mean number of males in the household; mean standardized wealth; the mean years of completed education of the mother; and the proportion of mothers that are working. These household characteristics could be of influence on the decision of parents to let their children enrol into early childhood education and/or on the test scores of the children. Since these characteristics differ significantly between households in program and control villages they could bias the results, which was why a difference-in-differences strategy was used. To account for this in the reduced form regressions as well, all these characteristics will be added as control, except for the proportion of mothers that are working as that may be a bad control. The fact that more mothers are working in program villages (see Table 2) could mean they send their children to the program ECED services more, but it could also be true that because of the higher exposure of ECED services mothers get the opportunity to supply labor because they do not have to take care of their children any more.

The results of regression (1) for attendance to the project playgroup and regression (2) for standardized total test scores in which this vector of controls is included are in Table 8a and 8b.

Table 8a – Effect of the program on average months of attendance to project playgroup including vector of controls

Months of attendance to project playgroup

(1) (2) (3) Class 1 0.064 -0.204 0.274 (0.515) (2.415) (0.484) Class 2 0.051 0.718 0.289 (0.504) (2.346) (0.476) Class 3 0.176 0.756 0.177 (0.518) (2.406) (0.486) Exposure 0.047*** 0.117** (0.012) (0.046) Class 1 * Exposure 0.093*** 0.098* 0.090*** (0.013) (0.053) (0.012) Class 2 * Exposure 0.076*** 0.060 0.069*** (0.013) (0.051) (0.012) Class 3 * Exposure 0.029** 0.015 0.029** (0.013) (0.053) (0.012)

District fixed effects Yes Yes No

Village fixed effects No No Yes

Vector of controls Yes Yes Yes

N 10,688 7,678 10,688

Note: Standard deviations are in parentheses. *Significant at the 10% level.**Significant at the 5% level. ***Significant at the 1% level. Column (2): sample of treated children (Batch 1 and 3).

When comparing Table 6 with Table 8a, one can see that the inclusion of controls does not change the estimates of the effect of the program on attendance. The estimates on the

interaction terms are all in the expected direction: the amount of impact again increases with exposure, but decreases for children in higher classes.

Table 8b – Effect of the program on standardized average total test score including vector of controls

Standardized average total test score

(1) (2) (3) Class 1 -1.329*** -1.317*** -1.456*** (0.070) (0.268) (0.069) Class 2 -0.795*** -0.746*** -0.872*** (0.067) (0.261) (0.065) Class 3 -0.373*** -0.281 -0.383*** (0.066) (0.267) (0.064) Exposure -0.001 -0.002 (0.001) (0.005) Class 1 * Exposure 0.001 0.002 0.001 (0.002) (0.006) (0.002) Class 2 * Exposure -0.001 -0.002 -0.001 (0.002) (0.006) (0.002) Class 3 * Exposure -0.001 -0.003 -0.001 (0.002) (0.006) (0.002) Age 7 0.251*** 0.268*** 0.187*** (0.027) (0.033) (0.026) Age 8 -0.388*** -0.341*** -0.519*** (0.032) (0.038) (0.031) Age 9 -0.381*** -0.329*** -0.544*** (0.039) (0.046) (0.038)

District fixed effects Yes Yes No

Village fixed effects No No Yes

Vector of controls Yes Yes Yes

N 12,453 8,966 12,453

Notes: Standard deviations are in parentheses. ***Significant at the 1% level. Column (2): sample of treated children.

The results for standardized total test scores are robust to the inclusion of controls as well. Table 8b indicates that the direction and magnitude of the estimates are similar to those found in the general estimation results in Table 7. There is still no impact of the program on the test scores of the children.

5.2 Heterogeneity in effects

Second, it may also be that the effects of the implementation of the ECED program vary across subgroups in the sample of treated children. To assess whether this is the case for the amount of attendance to the project playgroup, Table 9a shows regression results of the general model using district fixed effects, for different subgroups within the subsample of children in program villages only (Batch 1 and Batch 3).

Table 9a – Effect of the program on average months of attendance to project playgroup for different subgroups of the sample of treated children

Notes: Standard deviations are in parentheses. *Significant at the 10% level. **Significant at the 5% level. Wealth is generated based on the indicator of standardized wealth, where a standardized wealth value below zero is regarded as low wealth and values of zero or higher are marked as high wealth.

In particular children of families with relatively high wealth, children in rural villages and boys experienced a positive impact of the amount of exposure on their months of attendance. The pattern of a decreasingly positive impact of the program for higher classes can be

observed again, although most of the estimates are not significant. Both children from Months of attendance to project playgroup of children in program villages Low wealth High wealth Boys Girls Rural Urban Class 1 -0.506 -0.632 1.282 -1.583 0.210 -9.300 (3.818) (3.151) (3.587) (3.302) (2.507) (9.922) Class 2 -0.207 0.692 2.674 -1.080 0.919 -7.510 (3.737) (3.046) (3.511) (3.186) (2.435) (9.856) Class 3 0.654 0.517 4.338 -3.060 0.884 -9.152 (3.821) (3.133) (3.573) (3.300) (2.498) (10.011) Exposure 0.094 0.120** 0.138* 0.098 0.114** -0.008 (0.073) (0.061) (0.071) (0.062) (0.048) (0.220) Class 1 * Exposure 0.084 0.125* 0.060 0.134* 0.088 0.301 (0.082) (0.070) (0.079) (0.072) (0.055) (0.234) Class 2 * Exposure 0.067 0.071 0.011 0.105 0.057 0.211 (0.080) (0.068) (0.077) (0.070) (0.053) (0.233) Class 3 * Exposure 0.010 0.027 -0.063 0.099 0.011 0.247 (0.082) (0.070) (0.079) (0.072) (0.055) (0.236)

District fixed effects Yes Yes Yes Yes Yes Yes

wealthier families and girls that were in class 1 at the time of the test do experience a significant positive impact.

In Table 9b estimates of the same regression but with standardized test scores as outcome variable are displayed.

Table 9b – Effect of the program on standardized average total test score for different subgroups of the sample of treated children

Standardized average total test score of children in program villages

Low wealth High wealth Boys Girls Rural Urban Class 1 -1.420*** -1.589*** -2.067*** -1.091*** -1.492*** -4.072*** (0.471) (0.373) (0.422) (0.364) (0.285) (1.260) Class 2 -1.061** -0.753** -1.081*** -0.717** -0.871*** -2.924** (0.459) (0.359) (0.413) (0.352) (0.277) (1.243) Class 3 -0.557 -0.311 -0.664 -0.136 -0.305 -2.756** (0.469) (0.367) (0.421) (0.364) (0.284) (1.265) Exposure -0.010 -0.001 -0.011 -0.002 -0.005 -0.050* (0.009) (0.007) (0.008) (0.007) (0.005) (0.027) Class 1 * Exposure 0.003 0.008 0.016* -0.003 0.004 0.060** (0.010) (0.008) (0.009) (0.008) (0.006) (0.029) Class 2 * Exposure 0.004 0.000 0.005 -0.002 -0.000 0.049* (0.010) (0.008) (0.009) (0.008) (0.006) (0.029) Class 3 * Exposure 0.001 -0.000 0.006 -0.006 -0.003 0.055* (0.010) (0.008) (0.009) (0.008) (0.006) (0.029) Age 7 0.303*** 0.241*** 0.328*** 0.199*** 0.258*** 0.133 (0.054) (0.049) (0.047) (0.047) (0.035) (0.125) Age 8 -0.266*** -0.408*** -0.336*** -0.377*** -0.366*** -0.646*** (0.062) (0.058) (0.055) (0.056) (0.041) (0.146) Age 9 -0.245*** -0.412*** -0.405*** -0.353*** -0.405*** 0.729*** (0.072) (0.071) (0.065) (0.068) (0.049) (0.173) District fixed

effects Yes Yes Yes Yes Yes Yes

N 3,517 4,174 4,550 4,416 8,322 644

Notes: Standard deviations are in parentheses. *Significant at the 10% level. **Significant at the 5% level. ***Significant at the 1% level. Wealth is generated based on the indicator of standardized wealth, where a standardized wealth value below zero is regarded as low wealth and values of zero or higher are marked as high wealth.

The most notable result is that the estimates of the impact of the program are significantly positive only for children in the treatment group that live in urban villages, even though the

focus of the project was on rural villages. As expected, this effect is the greatest for urban children who were in class 1 at the time of the test. Also boys that were in class 1 in 2013 experience a significant positive impact of the ECED program.

These results give an indication that there was some positive impact of the ECED project on test scores for certain subgroups only. However, one should be reticent in interpreting this result as a cause for the lack of finding an overall impact. The magnitude of the estimates is small. Moreover, since the project was focused on rural areas the sample of children that live in urban villages is small, especially compared to the total sample of children that made the test.

5.3 Substitution effects

Another explanation might be that because of the intervention, parents chose to keep their children in the project playgroups longer instead of letting them enrol into kindergarten by the time they were 4. Parents might even have let their children switch from kindergarten to project playgroup. Even though it is expected by the government that prior to primary school children are enrolled in kindergarten, parents may have chosen to do otherwise, for example because they had the feeling that due to the project the quality of the playgroups was better compared to the established kindergartens that were not part of the project.

Playgroups are focused on learning through play while kindergartens have a more formal style of preparing children for primary school. If attending the more formal kindergartens rather than playgroups results in better test scores, this could explain the fact that children in control villages perform better than children that were in the program. The correlations between attendance to the project playgroup or kindergarten and the standardized total test scores indicate that this link might be present: attendance to kindergarten is positively correlated with test scores (0.20), while the correlation between attendance to the project playgroup and test scores is negative (-0.07).

To investigate whether any substitution effects are present, regression (1) has been done for the attendance to ECED services that did not belong to the project, but do focus on early

childhood education as well6. The results for attendance to the non-project playgroup and kindergarten during the period of the intervention are in Table 10.

Column (1) is again the regression with district fixed effects only. In Column (1) it can be seen that the amount of exposure decreases the amount of attendance to the non-project playgroup, but this effect is not significant. There is a significant negative impact of the project on the attendance to non-project playgroups for children that were in class 1 at the time of the test though. This is probably a substitution effect. Since these children were 2-3 years old when the project was implemented, it could be that compared to older children from higher classes, for these children it was still worthwhile to switch from non-project

playgroups to project playgroups. For the subsample of children in program villages only (Column(2)) the project did not have any impact nonetheless. Using village fixed effects as is done in Column (3) does not change the results compared to using district fixed effects. For the attendance to kindergarten there is no longer any impact of the program. All the estimates on the interaction terms are insignificant. There is a small negative effect of the amount of exposure on the months of attendance to kindergarten that is significant when using district fixed effects (Column(4)). This might be a substitution effect in that children that were exposed to the project longer, chose to attend the project playgroups instead of the kindergartens.

Hence, there are some indications that substitution effects were present from non-project services to project-services. Also, attending the more formal non-project kindergartens rather than project playgroups is associated with better test scores. However, one should be cautious in inferring any conclusions from this. If the implementation of the project meant that parents chose to let their children switch from kindergarten to project playgroups and therefore children in program villages performed worse on the test, one would expect a negative impact of the program on attendance to kindergartens. Yet, no such significant effects are found. Also there are no significant negative estimation results for the subgroup of children in program villages for which the greatest substitution effects should be expected.

6 _{The Islamic ECED services that are regulated by the Ministry of Religious Affairs are disregarded in the}

Table 10 - Effect of the program on months of attendance during intervention to non-project early childhood education services: non-project playgroup and kindergarten

Months of attendance to non-project playgroup Months of attendance to kindergarten

(1) (2) (3) (4) (5) (6) Class 1 3.162*** 1.475 3.226*** 4.002*** 2.654 3.902*** (0.412) (1.550) (0.392) (0.600) (2.421) (0.555) Class 2 1.875*** -0.660 1.798*** 4.237*** 6.281*** 4.230*** (0.403) (1.507) (0.386) (0.587) (2.353) (0.545) Class 3 1.045** 0.070 1.060*** 2.951*** 3.345 2.949*** (0.415) (1.546) (0.394) (0.604) (2.413) (0.557) Exposure -0.012 -0.010 -0.025* -0.021 (0.009) (0.030) (0.014) (0.046) Class 1 * Exposure -0.034*** 0.003 -0.035*** -0.010 0.021 -0.015 (0.010) (0.034) (0.010) (0.015) (0.053) (0.014) Class 2 * Exposure 0.013 0.042 -0.010 -0.013 -0.057 -0.022 (0.010) (0.033) (0.010) (0.015) (0.052) (0.014) Class 3 * Exposure 0.0034 0.018 -0.004 0.012 -0.020 -0.016 (0.010) (0.034) (0.010) (0.015) (0.053) (0.014)

District fixed effects Yes Yes No Yes Yes No

Village fixed effects No No Yes No No Yes

N 10,695 7,684 10,695 10,695 7,684 10,695

Notes: Standard deviations are in parentheses. *Significant at the 10% level. **Significant at the 5% level. ***Significant at the 1% level. Column (2): sample of treated children (Batch 1 and 3).

6. Conclusion and Discussion

Evaluating the impact of Early Childhood Education and Development (ECED) programs is of major importance. First, early childhood is a period that is essential for further

development. Children in developing countries are especially vulnerable in that sense, since they are exposed to multiple risks such as poverty, poor health and nutrition and deficient care. Despite convincing arguments and evidence that it is important to intervene early in life already and that ECED interventions have significant benefits, program coverage is still low. Second, there have not yet been many evaluations of ECED programs in developing countries that focus on educational outcomes and that account for non-random selection into such programs.

This thesis has evaluated the impact of a large ECED project that was implemented in Indonesia from 2009 to 2012, on test scores of children aged 6-9 who were in the first four classes of primary school in 2013. The program was implemented in different batches, creating variation in the amount of time the program could have had an impact. Next to this, the impact depended on whether children had the suitable age to attend the playgroup at the moment the project was implemented. Most studies that evaluate ECED interventions do not address whether and how impacts differ depending on age and/or duration of exposure to the intervention. In this thesis, a difference-in-differences strategy was used that exploits the difference in effects across classes due to the age of children and the difference in effects due to the duration of exposure to the program. Under the assumption that there is a common trend in the test scores of children from different classes, this makes that the causal effect of the program on test scores for children could be identified.

The results indicate that the impact of the intervention on attendance to the program increases with the amount of exposure and decreases for children in higher classes. For children who were in class 1 at the time of the test in 2013 and who thus were between 2 and 3 years old when the project started, one year of exposure to the program resulted in 1.1 months of significant overall attendance to the project extra compared to children that were in class 4. This positive impact decreases for children that were in higher classes at the time the test was taken: 0.9 and 0.3 months of extra attendance to the project for children who were in class 2 and 3 respectively. This is consistent with the idea that children in higher classes had less opportunity to attend the ECED services compared to children that were in lower classes. In higher classes children are of higher age and thus when the project was implemented, these