PADev guidebook: participatory assessment of development

(1)

UvA-DARE is a service provided by the library of the University of Amsterdam (https://dare.uva.nl)

Dietz, T.; Bymolt, R.; Bélemvire, A.; van der Geest, K.; de Groot, D.; Millar, D.; Obeng, F.;

Pouw, N.; Rijneveld, W.; Zaal, F.

Publication date

2013

Document Version

Final published version

Link to publication

Citation for published version (APA):

Dietz, T., Bymolt, R., Bélemvire, A., van der Geest, K., de Groot, D., Millar, D., Obeng, F.,

Pouw, N., Rijneveld, W., & Zaal, F. (2013). PADev guidebook: participatory assessment of

development. KIT publishers. http://www.padev.nl/other_output/PADev_guidebook_2013.pdf

General rights

It is not permitted to download or to forward/distribute the text or part of it without the consent of the author(s) and/or copyright holder(s), other than for strictly personal, individual use, unless the work is under an open content license (like Creative Commons).

Disclaimer/Complaints regulations

If you believe that digital publication of certain material infringes any of your rights or (privacy) interests, please let the Library know, stating your reasons. In case of a legitimate complaint, the Library will make the material inaccessible and/or remove it from the website. Please Ask the Library: https://uba.uva.nl/en/contact, or a letter to: Library of the University of Amsterdam, Secretariat, Singel 425, 1012 WP Amsterdam, The Netherlands. You will be contacted as soon as possible.

(2)

(3)

Ton Dietz Roger Bymolt Adama Bélemvire Kees van der Geest Dieneke de Groot David Millar Francis Obeng Nicky Pouw Wouter Rijneveld Fred Zaal March 2013

PADev Guidebook

Participatory Assessment of Development

(4)

University of Amsterdam,

Tamale University for Development Studies

Expertise pour le Développement du Sahel ICCO Alliance

African Studies Centre Leiden

Royal Tropical Institute Amsterdam

(5)

In collaboration with:

Joyce Ahenkorah, Anika Altaf, Genevieve Audet-Bélanger, Bassimbo Larba Jean Donald, Fred-erick D. Bebelleh, Wim Blok, Sanne Böhmer, Samuel Z. Bonye, Compaoré Nadège, Saa Dittoh, Kini Janvier, Henk Jochemsen, Christiana Kangsangbata, Agnieszka Kazimierczuk, Martha Lahai, Lengani Abdoul Karim, Akudugu A. Mamudu, Aurélien Marsais, , Jerim Obure, Jolien Oosterheerd, Oubda Lucien, Bruno Ouedraogo, Caroline Ouédraogo, Richard Yeboah Nartey, Zjos Vlaminck, Conrad A. Weobong, Yambré Alain, Ziba Balibio.

We would like to thank all contributors so far, including Robert Chambers, Irene Guijt, and members of the reference group (Antonie de Kemp, Sonja Ooms, Leo de Haan) for their many useful remarks.

Funded by:

ICCO Alliance, Prisma, Royal Tropical Institute, Universiteit van Amsterdam, Woord en Daad, and the African Studies Centre Leiden, 2007-2013

Amsterdam, March 2013

(6)

(7)

Preface

In recent times, development aid agencies have been confronted with ongoing skepticism as to their eff ectiveness. Aid agencies are increasingly expected to demonstrate ‘proof of success’, as measured by sophisticated monitoring and evaluation tools.

In turn, the methods used to evaluate the eff ectiveness of development interventions have also received their share of criticism. Th ey are oft en costly, time-consuming, have a bias towards quantitative data, oft en fail to capture the complexity of development and frequently require the input of expensive professional evaluators. Th e so-called ‘gold standards’ in impact evaluations are experimental or quasi-experimental econometric exercises, with random-ized control trials (RCT’s), and ‘diff erence in diff erence’ measurements. Th e overall aim of RCT’s is fi nding dominant patterns on the basis of ‘average’ behavior. As such, it pays rela-tively less attention to heterogeneity and dynamic interactions with other development inter-ventions and processes of change. While some studies have shown the potential of such an approach, they also highlight the downside of its use: their applicability tends to be restricted to certain domains of social change (like education, health care and water sectors), where this type of measurement is relatively easy, and they very much tend to be expert driven. PADev is an approach to development assessment that is holistic and participatory, and at the same time rigorous, and frequently makes use of both qualitative and quantitative data. It is intended as an additional impact measurement tool, rather than as a replacement for RCT’s. It aims to add both context and depth by building up a big picture of development and change in an area over time. Th is is done through a more inclusive approach, which gives a clear voice to the benefi ciaries of development interventions. Additionally it can be used as a set of tools for participatory history writing at local levels of scale.

PADev is a fi ve-year experimental research project to develop and test a new holistic and par-ticipatory methodology for assessing development. In 2007 three Dutch NGOs (ICCO Alliance, Prisma and Woord en Daad) made the decision to fund this project, which will be continued through to mid-2012. As of March 2012 all fi eldwork components of the PADev project are com-plete. To date, eleven workshops have been organized:

◆ 2008: Langbinsi, Sandema (both Ghana) and Tô (Burkina Faso), in areas with ongoing intensive collaboration between local NGOs and Dutch funding agencies

◆ 2009: Lasia Tuolu, Nandom (both Ghana) and Silly (Burkina Faso), in areas with past collaboration between local NGOs and Dutch fi nancing agencies

◆ 2010: Wulensi, Daboya (both Ghana) and Niabouri (Burkina Faso), in areas with no collaboration between local NGOs and Dutch development agencies.

◆ 2012: Tô II (Burkina Faso) and Langbinsi II (Ghana), to test the fi nal methodology that was developed based on experiences during earlier rounds, update the fi ndings for the last four years of development and change, and discuss the fi ndings with the local agen-cies in order to hear their feedback and viewpoints.

(10)

A number of additional theses have been written by (former) Master’s students at the University of Amsterdam and the University of Antwerpen:



◆ Jerim Obure: Participatory Monitoring and Evaluation: A Meta- Analysis of Anti-Pov-erty Interventions in Northern Ghana (2008)



◆ Agnieszka Kazimierczuk: Participatory Poverty Assessment and Participatory Evalua-tion of the Impact of Development Projects on Wealth Categories in Northern Ghana (2008/2009)



◆ Martha Lahai: Participatory Evaluation: Perceptions of Local People on Long-Term Impact of Development Interventions in Northern Ghana (2008/2009)



◆ Aurelien Marsais: Participation in the Land of the Righteous: Between Discourse and Development Reality in Burkina Faso (2009)



◆ Jolien Oosterheerd: Perceptions of the impact of migration on the development of the sending communities, Dondometeng and Kogle, Northwest Ghana (2009/2010) 

◆ Sanne Böhmer: “Th at’s how it is” Local perceptions of the notion of education-for-development and its impact on people’s livelihood strategies to improve their lives in Nandom, Ghana (2009/2010)



◆ Genevieve Audet Bélanger: Participatory Assessment of Environmental Projects: Con-cerns and Realities of Villagers and Development Organisations in the East Mamprusi District, Ghana (2010)



◆ Roger Bymolt: HADev - Holistic Assessment of Development: Assessing the ‘big pic-ture’ of development in Nanumba South District, Ghana (2010)



◆ Zjos Vlamick: PADev:The way forward: An Assessment of the Utilisation and

Empow-erment Capability, based on Fieldwork in East-Mamprusi, Northern Region, Ghana.

MA thesis, University of Antwerpen (2011) Two further studies were also undertaken



◆ Agnieszka Kazimierczuk: Langbinsi/Gbangu follow-up, with a focus on the poor (2010) 

◆ Anika Altaf: Perceptions about Islamic NGOs in Northern Ghana (2010).

In 2012 the eleven workshop reports were fi nalized and (combinations of) Ghanaian, Burkinabé and Dutch participants will write additional concluding papers.

See the PADev website for all available reports, data sets, and excel fi les formats used for record-ing data: www.padev.nl .

Please contact the PADev team through the website to provide feedback on the guidebook or to share experiences with the methodology.

(11)

Introduction

Overall goal

PADev is an innovative toolbox of tools and methods that have been designed to get a bottom-up assessment of development and change in a particular area over a period of time based on the value systems of the population. PADev is thus a holistic and participatory approach to develop-ment assessdevelop-ment.

PADev can be diff erentiated from conventional assessment methodologies, which oft en focus on a single agency, project/programme or sector with quite a ‘technical’ expert-driven approach to evaluation of output, eff ect and sometimes impact. In PADev, participants assess a wide range of changes, projects and agencies based on their perceptions. Further, PADev assessments typically look back at development and change over the past twenty to thirty years. Th is yields extremely valuable information for development agencies in the area: they learn about their own impact vis-à-vis other actors, and in addition, they fi nd out which types of projects have been regarded as most eff ective and relevant in that particular geographic and cultural setting and more impor-tantly: for whom and why, according to people’s own experiences and criteria. Th is can be an important lesson for future interventions. PADev should not be considered as a replacement for conventional approaches, but rather as an alternative approach that enables construction of a ‘big picture’ of development and change for an area.

Th is guidebook has been written specifi cally for those interested in using the PADev methodol-ogy. It begins with an introduction to the PADev workshop setup, before describing the aims and methods of each exercise. Th e complete PADev methodology comprises nine participatory exercises that can be run in a workshop format over three days. Proponents interested in employ-ing PADev in their assessments can do so in its entirety, or by usemploy-ing selected exercises and com-ponents to complement their existing assessment approach. It is entirely possible to implement PADev as it is described in this guidebook. Nevertheless, it should be stressed that this guidebook is not intended to be prescriptive. While it does describe what has worked for us, we do encour-age further experimentation and adaptation as circumstances require.

Up to this point we have employed PADev only in ex-post assessments. However, as Robert Chambers has pointed out during a recent debate about the PADev approach and preliminary fi ndings1_{, there is also potential for PADev to be extended for use in the development of}

commu-nity action plans. PADev can also be an excellent way for local governments, NGOs and devel-opment-oriented banks and businesses to improve the design of their interventions and their communication with supposed benefi ciaries.

(12)

Th e most recent workshop round in early 2012 showed that PADev also promises to be a valuable set of tools to enable local people to write their own development history. We encourage you to

share your experiences with us2_.

(13)

Workshop setup

Scale and scope

PADev exercises can be done at various levels of scale. To date, PADev workshops have been run in northern Ghana and southern Burkina Faso. Each research area had around 50,000 inhabit-ants, which oft en translated to an area of roughly twenty by twenty or thirty-by-thirty kilometres. Generally this meant an area with a central village/town of around 10,000 inhabitants, and ten to twenty villages and hamlets around that service centre. Th e fi eld team leader in northern Ghana or southern Burkina Faso would pay a visit to the area a few months in advance to have discus-sions with the local authorities (and get their informal approval), fi x a date for the workshop, fi nd a workshop venue and start preparations for selection of participants, catering and accommoda-tion.

We have experimented with using the same method at the level of small villages (150-300 houses), and even at the level of some secondary schools (14-16 year-old students). We have also attempted to specifi cally target people who are locally regarded as belonging to the poor. Some of this experimentation with scale and composition has been undertaken in Master’s thesis projects attached to PADev. Th is work has enriched the PADev method and has informed this guidebook. Th e level of scale chosen has certain implications. We have found that the smallest local market area makes an ideal level of scale. Th is is because most of the workshop exercises require shared knowledge of the same area by most participants.

A greater level of scale will likely produce data that is more general and less comparable at a project level because participants would need to be divided into geographical groups while the ‘local market level’ would not require subdivision in geographical groups. Participant groups may know a greater number of diff erent projects, but fewer participant groups will know of, and be able to assess, the same projects. Th e same is true for assessments of changes and agencies. Th e level of scale chosen will depend on the research objectives. Also, while it may be possible to apply PADev in a (peri)urban environment, this has not been tested yet by the PADev team and the methodology may require some modifi cations to accommodate this level of scale and the diff erent setting. Some experiments have taken place to do PADev-type work for specifi c sectors only, and for relatively limited time periods3_.

Th is guidebook describes the methodology for the level of scale as we applied it and will therefore indicate for which exercises geographical groups are needed.

3 e.g., Dengerink, Just. 2011.Experiences with renewable energy in Uganda: a participatory evaluation of the

(14)

Participants

To be suffi ciently representative, a workshop should consist of participants from all relevant cat-egories of the local population. In practice, this means sampling participants in a way that does justice to the demographic, socio-cultural and socio-economic composition of the community. To do so requires a listing exercise beforehand and some kind of prior stratifi cation (offi cials/ non-offi cials; and: men/women; old/young; relevant cultural categories; relevant socio-economic categories). Unfortunately there is typically little prior information available that could be used to select such a sample. It would be preferable to select participants in such a way that they are representative for all these diff erent characteristics. However, in using an approach that leads to a statistically representative sample, listing and selection is oft en quite time-consuming and could stretch the fi nancial and organisational capacity of many researchers and develop-ment agencies who want to apply this method. A pragmatic approach to sampling participants is suggested, although be aware that the selection of participants can infl uence the results of the workshop, so do consider this carefully.

In practice we select participants to include specifi c groups of people: old/young, female/male, offi cials/villagers, people from the central place and people from hamlets, Muslims/ Christians/ Traditionalists or any other relevant criteria, to get a proxy representation.

It is wise to include ‘offi cials’4_{and respected elders (e.g. the traditional authorities) in the}

work-shop as these participants can off er great historical insight, albeit from a particular perspective. Care should be taken not to exclude locally infl uential people as exclusion may jeopardise the acceptance of the research activity. However, care should also be taken that these local opinion leaders do not dominate the workshop or that they prevent others to express their thoughts. We have addressed this by treating them as one or two separate groups during the workshop, facili-tated by the most experienced or senior facilitators. Chiefs, government staff , NGO leaders and church leaders are all welcome, but preferably they should number only around ten or fi ft een out of about fi ft y or sixty participants in total.

A local, independent researcher who has a good knowledge of the area should preferably do the selection of workshop participants. Th is researcher can use the ‘snowball’ method starting with a man and a woman from each village, who should invite people from all groups within the com-munity. It is not advisable to seek the help of NGOs to invite participants because they may bias the sample (intentional or unintentionally) by selecting those they have worked with them in previous projects.

Th e PADev method is intended to combine knowledge about the area’s recent history with an assessment of people’s perceived ‘valuations’ of changes and interventions. Th e views of local spokespeople and offi cials should be separated from the views of ‘common’ villagers. Again: if a fully representative ‘valuation’ is important, the choice of workshop participants should be based on a representative sample, either of the population as a whole, or based on stratifi ed sampling. In both cases a local ‘census’ is oft en needed, as recent and complete lists of people or households are mostly not readily available.

(15)

Participant group sizes

We have observed that the size of the participant group aff ects the extent to which participants feel free to engage in the workshop exercises. Th e larger the group size, the less opportunity there is for each participant to speak. Participants may ‘switch off ’ as they get bored, or ‘hide’ behind more outspoken group members. In contrast, the smaller the group size, the more participants are involved and engaged, which may result in higher quality data. Participant groups of between six and eight members seem to be ideal. Diff erent subgroup confi gurations can be used, and these are suggested in the description of each exercise (below).

Avoiding bias

Looking at the experiences so far we can say that it is very unlikely that the richest and most infl uential business people in a particular research area will attend the workshops. Oft en they are simply too busy. Even if they say they will come, they either don’t turn up, or if they do, they tend to go in and out continuously disrupting their group.

It is also very unlikely that the very poor (and oft en despised or pitied) members of the local community will participate. Even if they have been explicitly invited, many will shy away from any formal meeting, let alone a three-day workshop. Th e same can be said about some minority groups (in our research area: Fulani herders for instance), who tend to live separate lives, and who oft en exclude themselves from meetings (and sometimes there is an element of explicit exclusion).

In practice, it is likely that workshop participation will be biased in favour of the relatively rich, average and poor people, and in favour of the local leaders and social organisers. Th e very rich and very poor tend to be the least well represented, and it is not uncommon that they choose to self-exclude themselves. Nevertheless, if care is taken to include peripheral villages, both men and women, and old and young, a (much) wider representation can be achieved than is typical of evaluation exercises. Th e methods used in PADev further facilitate triangulation between the information from diff erent exercises and between diff erent participant groups. Th e ‘story of local development’ that evolves is an amalgam of sub-stories, which together forms a locally acceptable reconstruction of the area’s development history, including diff erences of opinion and judgment. If one would really want to hear the stories and opinions of the very poor, it is wise to organise separate activities for them, and take special care to encourage them to talk.

Workshop timing

Th e timing of the workshop is very important. It should not be organised during the busiest peri-ods of the agricultural season, nor at a time when many people are away. It is important to avoid days of festivities, market days, or the holy days in the week. Organising the workshop during the rainy season causes problems, both of transport (facilitators and participants get stuck on the road, and arrive late, dirty, or not at all), and of illness (malaria, coughs). In many areas with sea-sonal agriculture, the rainy season is the hunger season and a period of stress; not the best period

(16)

for refl ection. Good timing should be discussed during the preparatory visit.

In cases when it is impossible (or too costly) to get all participants together at the same time, it is also possible to apply the PADev method in a sequential way; one group aft er the other. Particu-larly students who want to apply the PADev method for their studies can do this. Th e students working with the PADev method in our project have positive experiences with this sequential approach.

Organisation, catering and accommodation

Generally the workshop will be organised by a trusted local organiser (maybe a former student of the participating university). Accommodation is necessary for the team of facilitators, but not for the participants: Most go home at the end of the aft ernoon and come back the following morn-ing (although sometimes transport for those livmorn-ing in faraway communities should be provided). In general it is important to make sure that there are refreshments and lunch for all participants and facilitators. Providing frequent refreshments and a decent lunch keeps participants happy and their minds alert. When participants see that facilitators are sharing the same food with them they may feel a spirit of togetherness that enhances the participatory experience. Of course care should be taken to provide food and drinks that are locally acceptable. It is good to have a sepa-rate logistics offi cer, who deals with all food, transport and accommodation issues.

Workshop participants did not get formal payment, but we decided to give participants a small token of appreciation to compensate for their time, which they otherwise might have invested in income generating activities. Also, participant’s transport costs were paid. Th e main workshop facilitator mentioned a few times (including in his welcome speech) that attendance lists would be made for all three days, and that during the last day he would ask all participants to come to him in private. As a payment guideline, one could use the equivalent of what a local teacher would get for three days of work. Since the payment was not known beforehand and also not mentioned in the open, we assume that this didn’t play a role in the agreement to participate.

Training of facilitators

Workshop facilitators play a crucial role in PADev. Hence, it is important that they are fully aware of the ins and outs, the potential and the pitfalls of the approach. Oft en the teams of facilitators consist of people with diff erent cultural and disciplinary backgrounds5_{, and it is important to}

come to a common understanding, while at the same time making use of the wealth of cul-tural and disciplinary diff erence. Training should be provided before the actual workshop and a focussed refresher training and questions session the night before the workshop. Evening

ses-5 Th e PADev team of facilitators consisted of people with cultural backgrounds from Belgium, Burkina Faso, Canada, Ghana, Kenya, the Netherlands, New Zealand, Pakistan, Poland, Sierra Leone, and Spain, and they had been trained as anthropologists, (agricultural) economists, geographers, historians, nutritionists, political scientists, sociolo-gists, in international development studies and in natural resource studies.

(17)

sions evaluating the process is also recommended. An overall coordinator should be available for on-the-spot checks and follow up. Some of the issues that should be part of training exercises will be discussed in the following sections.

On language and interpretation

In our workshops to date, few facilitators spoke the local language(s), and among themselves English or French were the languages of communication. To be able to communicate with work-shop participants of whom few would speak/understand English or French good translation is very important, and sometimes in more than one local language. We always made use of work-shop participants, who would volunteer to play the role of translator. Th ere are issues with this, and facilitators should be trained to deal with these. One of the obvious problems is that some of the volunteers may not be fl uent in English or French. Another problem is that oft en these volunteers are young men or women, who have gone to school, and do not yet have a high status among the uneducated elderly people. Volunteers sometimes want to dominate the discussion and valuation, willingly or unwillingly manipulating the outcome of the groups in which they translate. Finally, some of the exercises require analytical skills to translate quite diffi cult and abstract concepts that may not be known to the interpreters or could be wrongly understood. Facilitators should be aware of these problems and use periods in between sessions to discuss with interpreters about these concepts. Bringing together outcomes of conceptual discussions in plenary sessions (e.g. about wealth categories) oft en brings to the fore that also in the local lan-guages there are major diff erences of opinion about concepts like ‘rich’ and ‘poor’, and this may even be more pronounced when there are diff erent languages present in the workshop.

On religion

In Ghana and Burkina Faso religion is very much part of daily life, and so care was taken in the various workshops to include prayers as part of the plenary sessions. A respected local reli-gious leader was asked to pray for everyone; care was taken to have diff erent rounds of prayers during the workshop, to enable religious leaders from a variety of faiths to lead prayers. In our workshops it has never caused a problem that an imam led prayer sessions in groups that also consisted of Catholic or Protestant members and leaders, or the other way around. Also, foreign and local ‘freethinkers’ always joined without any problems. However, there might be a problem if there are severe tensions between religious groups,. And in general one may say that it is not wise to have a workshop in a building or with an organisation that clearly belongs to one of the identity groups in the community (a church, or a mosque).

Workshop introductions

Th e workshop as a whole starts with a plenary round of introductions, in which the facilitators explain who they are and what the purpose is of the three-day gathering. In general it is wise that the overall coordinator presents him/herself fi rst and leads the round of introductions. At some

(18)

point the most senior among the facilitators from the country and from abroad should also get a chance to explain who they are and why they play a role in the workshop.

Th e introduction may last between one and one-and-a-half hours (depending on the type of introductory rituals, prayers, formal addresses, etc.). Brief introductions of the workshop partici-pants are best done at the start of the group exercises due to the time it would take to introduce everyone during the introductory plenary. Participant groups are then formed for the fi rst exer-cise of the day.

At the end of the fi rst plenary session and aft er all participants have been introduced, the facilita-tor who is responsible for the so-called personal fi les explains about the personal fi les and gives everyone a copy.

Aft er this fi rst plenary session all workshop facilitators go to specifi c rooms (or outside, ‘under a tree’) and ask all group participants to fi ll in lists with their correct names. Th ese participants’ lists will be compared later during the day with the list provided by the workshop organiser(s).

Data management and workflow

In PADev workshops, a senior facilitator is typically paired with a junior who is responsible for data recording for each of the participant groups. Facilitators should be provided with work-sheets in which they can fi ll in participant responses. Some facilitators prefer to record par-ticipant responses directly into digital worksheets, while others prefer recording responses with pen and paper and transcribing later. Whichever method is used, the worksheet for recording responses needs to be easy to understand and properly tested prior to use. Providing printed or digital worksheets the day they are to be used gives facilitators no time to gain familiarity or raise questions, which may induce basic recording errors, so this should be done the day before the workshop. At the top of each worksheet should be basic instructions of how to run the exercise. Th is should include the exact phrasing of exercise questions, ensuring that all facilitators ask the same questions in the same way to all participant groups.

Working with digital PADev data templates (available at padev.nl) makes it possible to get an overview of raw data while the workshop is on-going; to collect comparable data across the dif-ferent sub-groups in a systematic manner; and to do data cleaning and improvements while the workshop participants are still around. It also enables a fi rst ‘fast analyses’ and present ‘fi rst results’ very soon aft er (or even during) the workshops.

It is crucial that data documents and fi les are properly named in a consistent format. It should always be clear which fi le represents which group, and for what exercise. When necessary, partic-ipant group names should always be specifi c to the gender of the group. In practice this requires a dedicated workshop coordinator who clearly explains what is expected from facilitators and regularly makes checks. Th e excel template fi les on the PADev website are designed in such a way that it is easy to compile the data from diff erent participant groups per exercise or even across workshops.

(19)

Breaks between workshop sessions are important opportunities for facilitators to check and fi nalise reporting, and to recover from a very exhausting task. In the evenings the facilitators should come together to compare notes and briefl y discuss the working process of the exercises - how did participants understand the questions? What were the hitches? Are there suggestions for improvements? Exercises for the next day are also discussed.

Reporting

Following the workshop and aft er checks and corrections have been made, the ‘raw data’ from each exercise should be shared among facilitators. One or more team members should then be responsible for compiling the data of each workshop exercise. Later, the compiled data may be made available online. One person should be put in charge of making an analytical report (see the PADev website for formatting suggestions and examples of such reports).

A draft analytical report can be discussed with other facilitators and with a selection of workshop participants before releasing it as a ‘fi nal document’. Doing this properly takes time, but it also makes reporting a shared experience. Involving workshop participants as referees of the (Eng-lish- or French-language) reports of course enters a bias: only those who can read and under-stand these reports can directly play a role. However, a facilitator may also organize broader sessions in which the main fi ndings are being shared, using the local language(s), also with those who cannot read.

Triangulate the facts and fi gures from diff erent participant groups. In our experience this typi-cally showed that participants have a shared picture that is quite accurate when objective facts and fi gures are compared with other sources of information (e.g. data from agencies). One of the more diffi cult data is the relations between agencies that are related to projects and their donors, especially for multi stakeholder initiatives or value chain development. Th ere may be a need to triangulate this data with other sources of information, especially when analyses are made with regard to the types of agencies. However, it is preferable to make clear in reports which data from the workshop are corrected as a result of this triangulation. Such triangulation should be restricted to the facts and fi gures aspects.

It is wise to check parts of the participant data with representatives of agencies active in the area. In our experience participants usually have a relatively shared picture of development that is quite accurate when compared to ‘offi cial’ sources of information. However, sometimes partici-pant data is not fully complete, and occasionally totally ‘wrong’ in the view of agencies. Without including the agency’s voice the analytical conclusions of the overall report and the reliability and acceptability of the method may be undermined. However, this can only be done for those agen-cies still active and willing to participate. Facilitators should avoid allowing agenagen-cies to change the fi ndings from the workshops other than ‘facts and fi gures’, but for the credibility of the results it is important to go back to (important) agencies and validate the data with them.

(20)

Time management and workshop duration

In this guidebook we have tried to give an approximate duration of each workshop exercise. However, this typically varies between participant groups. Th e young men are oft en the fastest, while the older women or offi cials might be the slowest (oft en because they want to give a lot of details about projects and programmes). Th ere is a degree of facilitator skill involved in knowing how far participants can be pushed (or restricted) to elaborate their answers while keeping an exercise to a reasonable duration. Th e facilitator needs to sense the energy levels of the group and ensure breaks are taken at appropriate intervals.

Regarding the length of a workshop day, we feel that participants can comfortably work from 10am to 4pm. However, it should be remembered that participants may not always arrive on time for a variety of reasons, so we oft en ask them to arrive at 9am, even if in practice the work really starts at 10am.

Th e workshop coordinator should try to avoid that diff erent workshop groups have very diff erent time schedules. Particularly at the end of the day it can be problematic if some groups are ready and start to leave, while others still have a lot of work to do. For very fast participant groups some sideline exercises may be designed as on-the-spot experiments.

In general participants are tired and want to go home at around 4pm. Many have to travel and want to be back home for supper and before sunset. When a local market is going on in the neighbourhood of the workshop venue, organisers should not be surprised when participants (or even facilitators) try to sneak out for a while, which may jeopardize the fl ow and quality of the workshop.

Th e day for facilitators is typically much longer. Facilitators should be briefed at the start of each day or the evening before about the exercises to be run and have an opportunity to raise any questions. In the evenings facilitators oft en work late, checking data or transcribing written data into digital form. Needless to say, aft er a thoroughly enjoyable two- or three-day workshop, all involved appreciate a period of rest.

Enhancing participation – stick and stone methods

Th ere can be a tendency for one or two participants to dominate a group. Th is can undermine the voice of other participants, who may then ‘switch off ’. Th is can result in group responses that are skewed towards the views of dominant individuals. A solution might be to introduce a ‘talking stick’ (many African chiefs have one). Th e participant with the stick speaks fi rst, before passing it among other members to respond (not necessarily in a predictable circle), so that all have a chance to talk. Th e person with the stick is called the ‘stickman’ or ‘stickwoman’.

It has also been observed that exercises employing the use of stones generated a lot of discussion and engagement among participants because there was an element of ‘fun’ about them. Facilita-tors can think about how exercises can be adapted to use sticks and stones to make value judge-ments, rather than only giving oral responses.

(21)

In using ‘talking sticks’ please be aware of cultural sensitivities. In some areas with ‘strong chiefs’ it can be a cultural taboo for ‘common people’ to use a stick that resembles the chief ’s stick (although chiefs will usually be in an ‘offi cials’ group). Also, well painted sticks can give the per-son holding it a sense of pride to speak. However, we had one instance where decorations painted on a stick were interpreted by a participant as ‘juju’ (black magic) symbols. If this is a concern, painting symbols or colours from the national fl ag might be an alternative.

Workshop configurations

Th is PADev guidebook describes all the exercises that comprise a three-day workshop. However, it is important to restate that the PADev guidebook is not meant to be prescriptive - exercises can be adapted and extended in original ways. Agencies are also encouraged to use PADev as a tool-box of methods and to employ those exercises that are most relevant to their assessment needs – many diff erent confi gurations are possible. If agencies choose not to utilize some exercises then workshops may be shortened to two days, or even to a single day. It should be noted however that some exercises rely on data collected in a prior exercise and so both exercises would likely need to be run. Th e PADev team encourages you to share with us your experiences of diff erent confi gurations.

(22)

(23)

Exercise 1- Events

Objective

To reconstruct the most important historical events in the research area, and assess their most important eff ects on the community. Th is ‘sets the stage’ and helps to establish the development context. Th e exercise also acts as a nice ‘icebreaker’ for participants.

Workshop groups



◆ Offi cials (all people working for the (local/regional) government or for NGOs) ◆ older men (> 40)

◆ older women (>40) 

◆ younger men (40 and below) 

◆ younger women (40 and below)

If groups have a large number of participants it is preferable to split these to enable full participa-tion of all members (e.g. older men I and older men II). We have used the age of ‘40’ as a divid-ing line between ‘old’ and ‘young’, which generally resulted in groups of more or less equal size. If locally other age diff erentiations make more sense, or if groups become very diff erent in size, other age cut-off points may be used.

Position in a three-day, nine-exercise set up:

First exercise in the morning of the fi rst day.

Materials needed



◆ A1 sheets of paper or fl ip chart sheets (optional) 

◆ Pens or markers with diff erent colours (optional)

Duration

Th e exercise generally takes around 1 hour, with the group of offi cials typically taking the lon-gest time. Some participants may choose to use mobile phones to check years with colleagues or friends elsewhere. Th e elderly groups may go back more than thirty years (with some major events mentioned longer ago). Among the younger people the time line generally goes back twenty to thirty years.

(24)

Exercise

In the group each member briefl y presents him/herself, and recalls one major event in the area, including the approximate year (or decade, if it can’t be made more specifi c) and what eff ect on the community that event had.

Participants may be unsure what constitutes an event and may sometimes confuse events with gradual changes or development interventions. While not a big issue, participants can be told that we will talk about gradual changes (exercise 2) and development interventions (exercise 4) later. A useful defi nition of an event may be: ‘Something that happened in a single year which does not usually happen in other years’. Examples of events may be useful, such as natural events, diseases outbreaks, confl icts, political events, sporting events and cultural events. However, care needs to be taken not to be too specifi c as this may infl uence the kinds of events that participants recall. Th e emphasis should be on local-level events, although regional, national or international events that have somehow touched the community are perfectly acceptable. Participants continue to add events, until nobody can add anything substantial anymore. Collectively participants decide that the end result gives a good impression of the area’s history of major events.

Instrument for recording data

Group Decade Year Event description and effects

Nandom - young men 1980s 1983 Severe drought and wild bush fire (Nandom east); Low crop yields. This resulted in severe hunger and famine. Poverty increased. In the fire much vegetation was destroyed and many wild animals died.

Compilation/analysis

Data from all participant groups needs to be compiled, preserving an indication of which group came up with what event. Th e number of groups that mentions an event can be an indicator of how wide its importance was felt. A chronological history of the area can be reported using a timeline format or table format with the most important events. Th e analysis and write up may take a day or so.

Remarks and potential pitfalls:

This exercise is also an ‘icebreaker’, which can set the tone for the exercises to come. Responses from all participants should be encouraged, so that every participant has had the opportunity to speak in the group. If this exercise is dominated by only a few voices from the start (say, because others are shy), there is a risk this becomes a pattern for the rest of the workshop. The ‘stickman’ method may be useful here (see page 18). There are sometimes long discussions about details like the year or years in which an event happened. Facilitators should attempt to balance between giving room for these spontaneous discussions and refocusing on what the intention of the exercise is.

(25)

Exercise 2 - Changes

Objective

To get a detailed list of perceptions about the positive and negative changes in the research area. Th e assessed changes are organised according to six types (‘domains’) of ‘capitals’ and ‘capabili-ties’. Later on in Exercise 6, participants will be asked which interventions contributed to the major positive changes, and which interventions helped to mitigate the major negative changes.

Workshop groups

Same groups as for Exercise 1-Events. 

◆ offi cials (all people working for the government or NGOs, religious leaders) 

◆ older men (> 40 years old) 

◆ older women (>40 years old) 

◆ younger men (40 years old and below) 

◆ younger women (40 years old and below)

Large groups can be split (e.g. older men I and older men II).

Position in a three-day, nine-exercise set up

Second exercise of the fi rst day, probably starting aft er lunch

Materials Needed



◆ Pens or markers with diff erent colours (optional) 

◆ Stones

Duration

Th is exercise may take two to three hours; meaning that the facilitator has 20-30 minutes per domain. Facilitators should ensure that groups have suffi cient time to discuss changes across all domains. Th e cultural domain, in particular, oft en causes much debate and people like to talk about it in a nuanced way (“trend is positive, but”; or “no, we don’t like this trend, however…”).

(26)

Method of preparation

Each of the six domains (see format for reporting) should be discussed with participants to ensure common understanding. Facilitators need to have a common understanding of domains and sub-domains in order to explain these concepts consistently to each subgroup.

Th e research team must select a certain time frame prior to the exercise. Th e same time frame must be used by all participant groups. Up to now, PADev researchers have chosen either 20 or 30 years as a comparative timeframe, which is roughly a generation. Th e research team should be conscious of what the eff ect of choosing a particular timeframe might be (such as if there was a major confl ict, drought, etc 20 years ago). Some participants (especially younger ones) may have diffi culty visualising a 20- or 30-year period, so it may be helpful to reference an event that participants earlier recalled in the Events exercise.

Exercise

In groups, participants are asked to describe any changes that have occurred unprompted. So, participants can choose to start with any of the six domains. Participants are then asked whether they perceive the change to be ‘big positive’, ‘small positive’, ‘big negative’ or ‘small negative’. Aft er some debate, members of a group usually come to a negotiated consensus. Nevertheless, there may be cases where a change has both positive and negative eff ects. Th e facilitator can mediate as to what the dominant eff ect is (and record this), but should reassure participants that both positive and negative statements are always recorded (qualitatively) as well. When unprompted changes have been exhausted, the facilitator may prompt the group with specifi c questions about changes in other domains and sub-domains that have not been mentioned. It is important that the reporter records whether each change was discussed prompted or unprompted.

Immediate follow-up action

It is important to use the data from each group that same evening to make a summary of 4-6 major positive trends and 4-6 major negative trends for each domain. Th ese will be needed dur-ing the second day (for “Exercise 7 – Relationship between changes and projects”). Note this compilation can take several hours.

(27)

Format for reporting

Group Domain

Sub-domain

Change Reason for Change Change

effect

Notes on effect

Wulensi - older men

Natural Forest Then there were plenty of trees; Now there are much fewer trees

Women have been cutting and taking firewood without replanting trees; increased bushfires; some farmers don't know the usefulness of trees and so burn the trees on their farms

- Small negative, because "we still have some trees, and people are slowly learning to look after the forest".

Extending the exercise

Th e exercise can also be extended to ask for the reasons why participants perceive a change to have occurred (this fi eld is included in the data format above). Th is may lead participants to attri-bute a change to certain projects or project types. Doing so may replace the need to do Exercise 6 later. However, the research team should be aware that this will further extend the duration of what is already a long exercise. Alternatively, this exercise can also be done for only one or two domains, such as for part of an education evaluation with little time available.

Th e reporting format given above can be used as a format in the workshop report as well, as long as it is clear which group gave what judgment. It is nice to apply a systematic presentation format: e.g. always start with the old men, followed by the old women, followed by the young men and the young women, and fi nally by the offi cials. In a separate section of the report some compara-tive statements can be made, particularly if there are major diff erences (or similarities) between groups. Th e compilation and analysis of this information may easily take two days of work.

Remarks and pitfalls

Some sub-domains and their wording are culture and area-specific. It is also wise to allow discussions about words/concepts (and translations) as interpretations may be quite ‘language-sensitive’. Any con-notations should be noted down by the facilitator, to be used in the analysis later. Facilitators can choose to use the stoneman/stonewoman method to elicit responses from participants.

(28)

(29)

Exercise 3 – Wealth group categorisation

Objectives

1) To obtain a locally defi ned and accepted categorization of wealth groups. Participants identify attributes that serve as proxies of poverty and wealth in their area. 2) To determine the perceived distribution of wealth group categories within the population. Data from this exercise is also used later in “Exercise 8 – Wealth group benefi ts”.

Workshop groups

Same as in exercises 1 and 2 

◆ offi cials (all people working for the government or for NGOs) 

◆ older men (> 40 years old) 

◆ older women (>40 years old) 

◆ younger men (40 years old and below) 

◆ younger women (40 years old and below)

Large groups can be split (e.g. older men I and older men II).

Th ird exercise of the fi rst day, mostly in the second part of the aft ernoon

Materials Needed/Method of preparation



◆ Pens or markers with diff erent colours (optional) 

◆ Stones

Duration: Th e exercise generally takes about one hour, although in groups with more than one

local language, or in the groups with the older people, it may take longer.

Exercise

(30)

Part 1

Groups discuss the local images of poverty and wealth, and the local words used for each of the fi ve categories. Th e classifi cation of fi ve groups (very rich/rich/average/poor/very poor) pro-vides a more nuanced picture of the poverty and wealth situation as perceived by local people as compared to a classifi cation of only three classes (rich/average/poor). Th e classifi cation of fi ve categories enables participants to describe those few local people who are locally regarded as extremely rich even if there are only a handful of them in the whole research area. Also it allows people to describe those who are regarded as ‘hopelessly poor’ as distinguished from the ‘normal poor’. Oft en people tend to avoid talking about the very poor, as these ‘destitute people’ are oft en despised, avoided, or otherwise excluded. By explicitly talking about a category of very poor (even if there are ‘only few’, and ‘they don’t really belong to us’) as one of fi ve categories it enables a ‘natural’ inclusion of these people in the categorization.

Participants are asked to agree on the local names for the fi ve wealth categories (this may be done in more than one language). Participants are asked to describe the attributes of each wealth group. At fi rst this should be unprompted. Following this, participants may be prompted about specifi c attributes. Th e reporter should record whether participants described each attribute prompted or unprompted.

Attributes can include the following: 

◆ Type of job (or combinations of jobs) 

◆ If people are farmers: acreage or number and types of animals owned/used, use of machinery and manual labour



◆ Household size and number of wives and children 

◆ Type of house (material of walls and roofi ng), furniture, utensils (beds, bedding, chairs, etc.) and other assets.



◆ Ownership of transport; travel behaviour 

◆ Type of clothes and ornaments 

◆ Food consumption: how many meals a day and what quality 

◆ Education levels (also of the children) 

◆ Health and handicaps (physical and mental) 

◆ Funeral rituals 

◆ Other specifi c rituals 

(31)

Format for data recording

Group Category Local Name Description Distribution (10 stones)

Daboya - elderly women

Very rich Asowura Very rich are often big traders of farm produce and animals; farm with tractors; More than 1 house; Children able to attend higher educa-tion; Enough to eat and provide for others as well; Vehicles: car, motorbike, bike; Funerals: in Daboya there should be funeral functions on the 3rd day, 7th day, 40th day and after one year. Chiefs are buried under the big tree in town. People come from many places to attend. Food is provided for all, even tea in the morning. Other food: TZ, fufu, meat, soda drinks. A lamb is offered as part of the ritual. Food is prepared at home. People are dressed in beautiful clothes.

1

Part 2

In Part 2, participants are asked about the perceived distribution of the population across the fi ve groups. Ten (or alternatively twenty) stones are given to participants in each group to distribute across fi ve squares representing the wealth groups. Participants discuss the distribution of stones until consensus is reached within the group. If there is diffi culty reaching consensus the reporter should try to report (in words) the reasons for disagreement. Also, the reporter should note any remarks made during the process. It is important to verify that the distribution of the stones is done with in mind the images of the wealth groups that were developed by the group in part 1 of this exercise.

Format for data recording (Extension of the format used in Part 1)

Th e recording format can be used for the fi nal presentation of results, as long as it is clear which group gave which answer. Suggested tables/fi gures for presentation in the report can include:



◆ Local perceptions of the characteristics of fi ve wealth categories 

◆ Perceived wealth group distribution

(32)

inclusion in the fi nal report).

Immediate follow-up action

(Needed if doing “Exercise 8 – Wealth group benefi ts” later in the workshop): During the eve-ning a facilitator analyses the data from all participant groups and makes a summary that comes closest to an ‘aggregated’ compilation of wealth and poverty in the area (even if there are ‘outlier’ groups with completely diff erent views). In case there is a lot of diff erence in interpretation this average understanding is presented back to participant groups the following day to get agreement on the terminology in use for the rest of the workshop days. Of course, this process can some-what redefi ne those wealth classifi cations that participants have already gone to some trouble to defi ne. Th ere is some risk that these aggregated defi nitions may not be subsequently accepted and/or retained by participant groups and may cause further debate.

Remarks and pitfalls

During Part 2, some participant groups may have an initial tendency to state that ‘everybody is poor’. It is useful for the facilitator to recap what the wealth classes are and the attributes given for each. For ex-ample, a very large percentage of ‘very poor’ or ‘very rich’ is unlikely if the descriptions indicate that these categories include the exceptional cases. This may indicate that there is still some confusion within the group about the different wealth classes, or confusion about the exercise itself.

Participants may have an initial tendency to state that the very rich only live in the capital city or overseas. The intention of the exercise is to capture local definitions of wealth and poverty, so it should be cross checked if the given description indeed refers to those who are locally regarded as very rich, and if this is indeed true, this should be brought back into the discussion by the facilitator.

To check participant’s comprehension of the use of stones, one could first ask the participants to distribute the stones according to the number of Europeans and the number of Africans attending the workshop. In some cases, participants will note that the very rich or the very poor are present, but not even up to one stone, and so decide to give them no stones, while still a few persons from that category are present. Such discussions should be reflected in the reported comments.

It may occur that the group is convinced that there are not five but e.g. four groups of wealth classes. Of course if this is a strong and shared conviction, this should be accepted. However, for the data to be com-parable across all participant groups it is preferable that indeed five classes of wealth are used. Prompted or not, a separate debate about poverty and wealth with regard to funeral arrangements often sparks a really interesting discussion about social differences, about exclusion-even-after-death, about solidarity, care and cultural institutions, and about cultural change.

(33)

Exercise 4 – Project recall

Objective

To get a complete list of all development projects (programmes, interventions, initiatives etc), including the dates, agencies and other relevant details. Th is list of recalled projects will be assessed and used in a number of later exercises.

Workshop groups



◆ offi cials (all people working for the government or for NGOs) 

◆ geographical groups; oft en:

o people from the central place (one or two groups)

o people from villages in the north, the south, the east and the west

When doing the exercise at lower levels of scale, it is not necessary to divide participants accord-ing to geographical groups.

First exercise in the morning of the second day, aft er a plenary session to re-open the workshop (oft en with prayers, see before).

Materials Needed



◆ none (or large sheets of paper)

Duration

1.5 hours

Exercise

Make a list of all development ‘projects’ that people can remember in their sub-area (as far back as possible). Information should be added about the initiator agency, which can then be catego-rised according to initiator type as:



◆ government 

(34)

DfID, DANIDA) 

◆ church-based or Catholic/Protestant NGO 

◆ mosque-based or Islamic NGO 

◆ non-faith based NGO with an international or national/local background (e.g. CARE International, Oxfam)



◆ enterprises 

◆ local community or individuals 

◆ any combination that is relevant

Th e year when the project started is noted as well as the year when the project ended (or indicate whether it is still ongoing). Finally, any other project details that people can remember are added. Participants should not try to assess the project as this comes later. Th e recalled project should be recorded under the relevant sector heading in the data worksheet.

Format for recording

Group Sector Project name Initiator name(s) Initiator type(s) Project Description Date(s) from-to Impact 'Then' Impact 'Now'

Reason for impact assessment (+ additional comments) Nandom-central-men Water Construc-tion and operation of piped-water Ghana Water & Sewage G Laying of piped structures and supply of water in Nandom Central

1970 ₊ ₊₊ The piped water was excellent and convenient when it came but few people could afford it; Now it needs maintenance and there are some leaks. However, more people can now afford the water and more people are connected, so greater impact

Immediate follow up

Aft er gathering all information of all groups a detailed list can be made about all the agencies men-tioned (see above). Th is information can then be checked with one or more local well-informed people, including the workshop organiser. Mistakes and inconsistencies that are detected in the results of the groups should immediately be corrected, to get a screened dataset. However, it is important that the corrections made remain visible in the primary reports, because the per-ceptions on agency type give information about what participants know and don’t know about interventions and agencies. For instance: some NGO initiatives are not known by the name of the agency, but by the name of people involved as NGO facilitators. It is this information that should be used for the exercises that follow.

(35)



◆ Chronological list of interventions/development eff orts 

◆ Number of interventions by agency and decade 

◆ Number of interventions by sector and decade 

◆ Proportion of intervention by sector and decade (%) 

◆ Number of interventions per type of agency (solo and in partnership) 

◆ Agency composition of project partnerships 

◆ Number of development eff orts per sector 

◆ Number of intervening agencies per sector 

◆ Proportion of sector involvement per type of agency 

◆ Proportion of type of intervening agencies per sector 

◆ Proportion of interventions by type of agency and decade 

◆ Proportion of interventions by sector and decade 

◆ Proportion of intervening agencies per sector

Remarks and pitfalls on listing

Our experience is that area-groups of the villages/hamlets often list between 30 and 60 development ef-forts, of the small town between 50 and 90, and of the officials more than 100. A decision has to be made beforehand in which sector certain development efforts are to be positioned and this has to be explained well in advance to facilitators. In practice, only one sector is problematic, infrastructure. Infrastructure is intended to cover roads, bridges, railway lines, airfields, communication devices (e.g. for mobile phones) and the like. Not all buildings will be included and some should appear under other sectors: school build-ings under education, health centre buildbuild-ings under health care, water towers or boreholes under water, irrigation canals under crops, churches and mosques under religion, etc. A ‘project’, ‘development effort’, ‘intervention’, or ‘initiative’ is any effort that is made by an individual, group or agency with the intention to benefit more people than only a single household (so really individual initiatives should not be included). In this way, a private grain mill that others can use for payment is included, but a well on someone’s com-pound, which others are not allowed to use is not. However, there will be ‘grey areas’ here.

It is wise to be as specific as possible, so include each particular school as one project, not just ‘schools’, and each set of boreholes as one project, not just ‘boreholes’. In general we used as a rule of thumb: dif-ferent locations of the same project done by the same agency count as a single intervention (but list the details). If done by different agencies, they count as different projects. E.g., boreholes: not all individual boreholes are counted as interventions (there could be as many as 30), but 1) all boreholes done by agency X, 2) all boreholes done by government arrangement Y, 3) all boreholes done by a different government agency at a later period, 4) that one borehole donated by an individual; etc. If specific information is added by reporters about years, it is useful to be clear about the way they write that down: ‘2002’ means: in that particular year, ‘2002 onwards’ means: from 2002 until today, ‘2002-2004’means the three year period starting in 2002 and ending in 2004, ‘ca 2002-2003’ means: more or less in the period covering the years 2002 and 2003.

(36)

Remarks and pitfalls on agencies

Information about the agency responsible for an intervention/development effort should be checked thor-oughly, during (and even after) the workshop, as there are often mistakes or inconsistencies. We noted with surprise that workshop participants themselves check this type of information during the delibera-tions by using their mobile phones and asking those who could know, if they are not sure themselves. However: this remains a problematic part of the method. Checking the information with all the agencies involved would be preferable, but it is often impossible as some agencies that people remember are no longer active, and their offices closed and officials gone. Mistakes that may easily happen are misspelling the names of organisations, wrong categorization (e.g. ‘NGO’ while it is a bilateral donor), incomplete information in cases of multi-donor involvement, mixing up grass-root agencies with their back-donors, etc. Good data recording is also essential for matching projects during data compilation. In practice many development agencies appear to work together and become multi-actor agencies. Some also share fund-ing from a variety of fundfund-ing agents, sometimes even combinfund-ing supra-national (UN; World Bank), govern-ment, NGO, business and community funding, and hence alliances. Activities supporting ‘value chains’ are particularly difficult to assign to one agent or type of agent.

(37)

Exercise 5 – Project assessment

Objectives

To assess the perceived impact of the recalled projects. Th is is done for ‘Th en’ (the fi rst year a project came) and ‘Now’ (the project as it is perceived today). Th e exercise helps to get a picture of the valuation of projects that have come to the community.

Workshop groups

Same as the ‘Project Recall’ exercise, except that the men and women are separated into diff erent groups. Th e assessments should be done separately because men and women do have diff erent and sometimes contradicting opinions about projects.

Second exercise in the morning of the second day, aft er the ‘Project Recall’ exercise.

Materials

n/a.

Duration

2 hours

Method of preparation

Th e ‘Project Recall’ exercise is a prerequisite for this exercise. Th e men and women are divided into separate groups for this exercise, but assess the same recalled projects. Th is means that a copy of the list of recalled projects needs to be made for facilitators of the men’s group and the women’s group.

Exercise

Facilitators should begin by explaining the exercise. Participants are asked what they perceive the impact to be for each project, for both ‘Th en’ (fi rst year the project actually came and started

(38)

running) and ‘Now’ (the project as it is today). Participants may use the following categories: ++ Big positive impact

+ Small positive impact / No impact

- Negative impact

* Cannot assess – Th e project may be too new, participants do not really know the project, etc.

It is important to ask participants to give the reason for each assessment rating, including the rea-son for any change between ‘Th en’ and ‘Now’. If not mentioned by participants, probe about the coverage of the project – did it impact on many people or just a few? In cases where participants cannot assess the project it is especially important to note the reason why.

Format for recording

(extension of the ‘Project Recall’ recording instrument)

Potential additional module

In some of our workshops we added questions about people’s perceptions about the impact of each and every project on the six types of capabilities (see exercise 2 on changes), hoping to get some clarity about people’s perceptions on the ‘impact domain’ of projects. Doing this for each and every project is very cumbersome, though, and the results very much depend on the atmosphere of the group and on the skills of the facilitator. Particularly the women were oft en all-inclusive: every project seemed to have an impact on each of the six capabilities. To make this exercise more useful it is better to restrict it to a selection of projects, to take the time for the stories behind the choices, and to come to some kind of ranking.

Example: a primary school building project in an area where there was no school before, and where aft er the building was ready most of the 200 children of school-going age joined as pupils (while before only 20 of them went to a school far away); a feeder road was included in the proj-ect; the school also has a successful school garden; there are fi ve new teachers (with teachers’ houses) and one of them is very active in a parent-teacher group and in developing culturally sensitive elements of the school curriculum, integrating the opinions of respected elders and traditional leaders.

So people tend to say: the most important impact is on human capabilities; but there are also important impacts on cultural capabilities and awareness; the road itself is ‘ physical capital’ (the school building itself of course as well; but that was seen as part of ‘ human capability enhance-ment’); and the school was built by a local contractor, trained by a foreign aid agency (so adding

(39)

to local economic capabilities). People say that there are also minor impacts on natural capital (the school garden has some impact on the local agricultural practices) and on social-political capabilities (as the parent-teacher organisation in practice improves local people’s abilities to deal with the local authorities).

Using the stone method the men would give 4 out of ten stones to human capabilities; 2 to eco-nomic capital, and one each to natural, physical, cultural, and socio-political capital/capabilities; the women would give 3 to human capabilities, 2 each to cultural and physical capital and one each to natural, economic and socio-political domains.

Without the stone method these fi ndings become very diffi cult to interpret.

Th e following tables or fi gures can be generated from this data: 

◆ Project impact judgments 

◆ Impact judgment per type of agency 

◆ Proportion of impact judgment per type of agency (%) 

◆ Impact judgments per sector 

◆ Proportion of impact judgments per sector (%) 

◆ Impact on (six) domains, scores per type of agency (where scores can be constructed based on average assessments of all projects by an agency as assessed by all participant groups that mentioned these projects.)



◆ Many of the same tables can also be given for coverage. 

◆ All analyses can be made for then and for now + analyses of the diff erences in percep-tion over time.



◆ Cross tabulation of impact and coverage

If the extension with impacts on the domains is done, the following analyses can also be done: 

◆ Project impact per domain (six domains) 

◆ Impact on (six) domains, scores per sector (frequencies) 

◆ Proportion of impact on domains, scores per type of agency (%) 

◆ Proportion of impact on domains, scores per sector (%) n.b gendered data analysis is possible

Alternative Use

Researchers may choose to leave out the ‘then’ state for various reasons, such as a lack of time.

PADev guidebook: participatory assessment of development - PADev guidebook 2013