stakeholders to the robustness of environmental
policy decisions
RM Bijlsma1,2, HA Wolters2, JL de Kok1 and AY Hoekstra1 1 Department for Water Engineering & Management, University of
Twente, P.O Box 217, 7500 AE, Enschede, the Netherlands (E-mail:
r.m.bijlsma@utwente.nl; j.l.dekok@utwente.nl; a.y.hoekstra@utwente.nl)
2 Institute for Inland Water Management and Waste Water Treatment
(RIZA), P.O.Box 17, 8200 AA, Lelystad, The Netherlands (E-mail:
henk.wolters@rws.nl)
Abstract
A methodology is developed in this paper to test the effectiveness of suc-cessful stakeholder participation in creating substantively more robust pol-icy, and to identify the factors responsible for that. The comparison is made to a more traditional management of government policy, in which experts in administrative agencies perform the policy analyses. The meth-odology compares the substantive robustness of the preferred alternative developed in a stakeholder participation process to the robustness of the preferred alternative developed for the same case in an expert-based proc-ess. It comprises a case study design that enables realistic performance of both processes, ensures their comparability and guards their quality. The developed methodology appears promising and is currently being applied. A systematic investigation of the relation between stakeholder contribution and the robustness of policy analyses helps formulating arguments for
stakeholder involvement. Furthermore it can improve the effectiveness of a participation activity.
Keywords: environmental policy, stakeholder participation, robustness, policy risk, perceptions, uncertainty
Introduction
Stakeholder participation in environmental policy analysis is increasingly recognised to improve the quality of the decision-making process address-ing complex problems (Beierle and Cayford 2002; Korfmacher 2001; Fiorino 1990). The comparison is made to the more traditional process of governmental policy developments, in which experts in administrative agencies perform the policy analyses. Three types of rationales are com-monly cited in literature in favour of stakeholder involvement in policy de-cisions: a democratic, a substantive, and a pragmatic rationale (Fiorino 1990). The substantive rationale is the focus of this paper. This rationale is based on the idea that relevant expertise is not limited to professionals and public officials, and that participation of stakeholders will provide essen-tial information and insights.
The substantive contribution of stakeholders is recognized in the litera-ture for management of uncertainty and risk as well. Recent literalitera-ture shows an increased interest in participatory methods to improve the man-agement of expertise and its related uncertainty (Refsgaard et al. 2007; Newig et al. 2005). In this paper we see expertise as including both facts and values, and dependent on perceptions. Uncertainty in this context is understood in a broad sense. It is related to problem framing (due to differ-ent perceptions on the problem, objective, and the related system to be analysed), to the conceptual model of the system (how does the system be-have?), and to the data and parameters used in the analysis (Walker et al. 2003).
Following the substantive rationale, we hypothesize that successful stakeholder participation in policy analysis leads to the formulation of more robust policy. The precondition to this hypothesis is that the stake-holder participation process is successful, since non-successful processes are not likely to deliver robust results. The stakeholder participation is evaluated based on stakeholders’ assessment of the process and the out-come of the policy analysis (Rowe et al. 2004). Policy is considered robust when it keeps performing well under different possible perceptions of the problem, the system and alternative future developments. Robustness of
policy is closely related to the risk of a policy. A policy that is more robust has a lower risk, since the probability that the policy will not have the in-tended effect, but leads to adverse consequences, is lower (Hoekstra 1998). The robustness of policy can be increased by taking into account more per-spectives during the analysis, or by including mechanisms that enable the adaptation of the implemented policy to unforeseen future developments (Pahl-Wostl 2007).
In a stakeholder participation approach, all parties that either have a stake or a responsibility with respect to the policy alternative are included in the development of policy alternatives. Their participation takes place in a structured way. The term stakeholder participation in this paper refers to a “high” level of participation in which stakeholders contribute to each part of the policy analysis, from problem framing to the final decision (Arn-stein 1969). A negotiation process leads to the preferred alternative. In an expert based approach, the competent authority performs the policy analy-sis or outsources it to experts, and takes the policy decision while account-ing for all stakes. The authority is the party that, for example, frames the problem and decides on a weighting of stakes. It can choose to consult stakeholders, but this process is not arranged for in a structured way.
Two ways are distinguished in which stakeholders can contribute to the substantive quality of a policy analysis: improving robustness and improv-ing the accuracy of the knowledge base (by removal of error or providimprov-ing more precise descriptions of the system). The literature provides multiple examples of stakeholders improving the accuracy of the knowledge base for an analysis (e.g. Maxim and Van der Sluijs 2007; Brody 2003). This improves the quality of the analysis without doubt, since factual accuracy positively influences the outcome of an analysis. The focus of this paper will be on the role of stakeholder involvement in increasing the robustness of preferred policy, not on the more transparent contribution in improving the accuracy of the knowledge base.
In a literature review we found seven arguments in support of the posi-tive relationship between stakeholder involvement and the robustness of policy decisions. These arguments focus on different phases of the policy development process: problem framing, system analysis and policy formu-lation. The first five arguments are based on Refsgaard et al. (2007). The last two arguments have been formulated based on case study descriptions. Stakeholder participation improves the problem framing:
1. Stakeholders improve the quality of the problem formulation and questions addressed.
2. Stakeholders provide personal observations/ insights that lead to new foci for empirical research addressing dimensions of the problem that were overlooked or underexposed by the professionals.
Stakeholder participation adds valuable local knowledge to the system analysis:
3. Stakeholders contribute expertise on local conditions and societal values that adds to the existing expertise.
4. Stakeholders criticise the assumptions made by the professionals, which leads to changes towards assumptions that better match real-life conditions.
5. Stakeholders creatively think of mechanisms through which various scenarios may affect different sectors of society.
Stakeholder participation increases the focus on prevention of adverse con-sequences:
6. Stakeholders improve the assessment of potential adverse consequences of measures and propose additional measures in packages to mitigate potential adverse consequences.
7. In case of perceived uncertainty, stakeholders focus more on flexibility of the developed alternative and on a good monitoring plan, enabling adaptation of the alternative in case of potential adverse consequences.
In contrast, however, we found an argument in the literature relating stakeholder involvement to less robust policy decisions (e.g. Yosie and Herbst 1998; Maxim and Van der Sluijs 2007). It states that stakeholders decrease the legitimacy of a policy analysis by on the one hand including non-scientific knowledge of questionable validity and on the other hand negotiation on/ putting less value to established expertise and available tools that are considered important by professionals.
However, the influence of stakeholder participation on the overall ro-bustness of policy decisions has never been examined systematically. Nei-ther has it been systematically researched which aspects noted in the ar-guments contribute most to the robustness. In the next section, a methodology for such a systematic study is described.
Method
A methodology has been developed to compare the robustness of the pre-ferred policy alternative resulting from a stakeholder participation process to the preferred alternative resulting from an expert-based process. The aim of the methodology is to test the effectiveness of stakeholder
involve-ment in increasing policy robustness and to identify the factors that are re-sponsible for that. A precondition for application of the methodology is that both process are comparable, i.e. applied to the same case within an identical context. The methodology consists of two steps that will be de-scribed below: an assessment of the difference in the robustness of the pol-icy alternatives and a systematic analysis of the differences in the em-ployed expertise for the development of the alternatives, identifying the aspects that have been most influential on the robustness.
To test the robustness of a policy alternative, its performance is evalu-ated by confronting it to another analysis perspective. The expert-based al-ternative and the stakeholder participation alal-ternative are both developed from a different perspective, with different assumptions on the nature of the problem, its relevant aspects, the functioning of the system and future developments. This different analysis perspective is reflected by different criteria on which the policy is evaluated. The robustness of each alterna-tive is tested by confronting the policy both to its own criteria and to the criteria developed in the other approach. On the former it will probably score quite well, on the latter worse. The most robust policy is the one that scores best on average for both sets of criteria. As a second aspect, the standard deviation between the scores on both sets of criteria is important; a policy is not robust when it can both turn out very well and very bad si-multaneously.
To systematically analyse the differences in expertise used for the de-velopment of the preferred alternatives, the expertise has been classified into the analysable variables shown in Table 1. For each alternative, the
Table 1. Classification of expertise into analysable variables for the different
phases of the policy analysis. The variables are used for systematic analysis of the differences in employed expertise due to difference in perspective.
Problem framing +
objective formulation System analysis Policy formulation • Temporal, spatial
and substantial scope of the problems identified
• Objectives formulated • Foci set for the
system analysis
• Conceptual model of the system used • Analysis tools used • System data used • Use of analysis tool
outputs • Criteria used
• Weighing criteria used
• Response options considered
• Preferred alternative and monitoring plan developed
expertise variables and underlying considerations are described and then the differences are identified. Not all differences have an equal level of in-fluence on the robustness of the analysis. The ones leading to difference in either the preferred alternative or the underlying criteria are most influen-tial. After identification of these most influential differences in expertise, we can conclude in which part of the analysis stakeholders have had the largest influence on the policy robustness.
Case study requirements
Application of the methodology requires a stakeholder participation ap-proach and an expert-based apap-proach implemented for the same case. This chapter discusses the design requirements and subsequent design of such a case study, showing the considerations and discussing the practical con-straints and threats encountered. The four main design requirements identi-fied for testing the hypothesis are that:
1. The stakeholder process is successful;
2. Both approaches start from a common knowledge base;
3. The expert-based preferred alternative is not influenced by the stakeholder participation process;
4. The expert-based process is of good quality.
Ad 1. The stakeholder process is to be tested for its success, as a pre-condition for statements about the hypothesis. Several authors suggest sets of criteria to measure the success of stakeholder participation (e.g. Beierle and Koninsky 2000; Halvorsen 2001; and Rowe et al. 2004). They relate the success of stakeholder participation to the process followed. Besides the process followed, the context in which the process takes place influ-ences the activity. However, context appears not to predetermine success (Beierle and Cayford 2002). Sets of criteria to measure success are mostly a combination of criteria related to the process and the outcome of the ac-tivity. We chose to use the criteria and instrument as developed by Rowe et al. (2004), since it is universally applicable, made operational and claimed to be validated. The authors defined nine criteria related to process and outcome: task definition, representativeness, early involvement, inde-pendence, cost-effectiveness, influence, transparency, structured decision making, and resource accessibility. They have developed a questionnaire to be spread among the participants to score the criteria. We assume the process to be successful when none of the criteria scores poorly.
The success of the stakeholder participation process can be negatively influenced by the simultaneous development of an expert- based policy
al-ternative. This can lead to the impression of a ‘hidden agenda’ of the com-petent authority: the development of an alternative to be implemented in case of an unfavorable outcome of the stakeholder participation process. The design of the case study should prevent such an interpretation. To le-gitimate the development of an expert-based alternative next to the stake-holder participation process, we ask both the competent authority and the representatives of each major stake to develop their own preferred alterna-tive. An additional advantage of this approach is that it supports the stake-holder process and its analysis, by making explicit the points of departure of the participants.
Ad 2. Both approaches start from a common knowledge base, to be able to measure only the differences in robustness of both approaches, and not the differences caused by error in the expertise. The start of the stakeholder participation process consists of a phase for collection and joint approval of the data and a model, in order to remove all known errors. The expert-based process starts after approval of the knowledge base by the stake-holders, but before the negotiation phase starts in which a confrontation of different perspectives takes place to come to an alternative that is preferred by consensus.
Ad 3. The expert-based preferred policy alternative should not be influ-enced by the simultaneous performance of a stakeholder participation process. This, however, does not imply an isolation of the development of the expert- based preferred alternative from stakeholders’ input. In a tradi-tional expert- based approach, regular interaction exists between stake-holders and the competent authority. Moreover, the authority consults stakeholders in some cases during the policy analysis to find out their posi-tion, or to obtain data or insight. The interaction during the first phase of the policy analysis, in which a joint knowledge base is developed together with the stakeholders, is therefore assumed to only have a small influence on the realism of the developed expert- based preferred alternative; it can be compared to an extensive consultation. The preferred alternative of the competent authority is likely to be influenced during the negotiation phase, based on discussions with stakeholders, evolving insight and subsequent reframing (Dewulf et al. 2004). Therefore the development of the expert-based preferred alternative should be finalized before the negotiation phase starts. The time path of both approaches and their interaction are visualized in Figure 1.
Ad 4. As a last aspect, the quality of the expert-based approach is to be tested, to allow statements about the hypothesis. The quality of the
expert-based approach depends on the abilities of the experts performing the analysis. The experts are assumed to be sufficiently capable of performing a good quality analysis when: i) at least one of the experts is familiar with the performance of policy alternatives of similar nature; ii) the experts per-forming the analysis are considered experts on the substance and entrusted to complete the policy analysis with good result, both by people internal and external of the competent authority.
Fig. 1. The time paths of the stakeholder participation approach and the expert-
based approach, and the interactions between both processes. Both processes coin-cide in the development of a joint knowledge base; the expert knowledge base is complemented and adjusted based on stakeholder expertise. The development of the expert-based (EB) preferred alternative takes place before the negotiations about the stakeholder participation (SP) preferred alternative start. The competent authority’s perspective, reflected in their preferred alternative, enters the stake-holder participation process as one of the perspectives.
Case study application
The method is being applied to a Dutch case study of developing water management alternatives for the Natura2000 area Vecht Beneden Regge and its surroundings. This area is situated in the East of the Netherlands. The Natura2000 status of the Vecht Beneden Regge area requires the reali-zation of nature conservation goals. For meeting part of these goals a wet-ter environment is required. This adversely affects the farmers, inhabitants and entrepreneurs in the surrounding area. The point of departure of the Natura2000 legislation is that their stakes should not be harmed dispropor-tionally. The responsible authority waterboard Velt en Vecht has set up a stakeholder participation process to develop a water management plan. Af-ter identification of the stakeholders, a sounding board group has been set
up in October 2006, consisting of representatives of all stakes. The sound-ing board is to meet four or five times, aimsound-ing at a negotiated water man-agement plan by the beginning of 2008. In addition, regular meetings are organized with subgroups, consisting of representatives of one or a few stakes or representatives of a sub area. The first meetings were aimed at agreement on the knowledge base: what is the current status of the water management in the area and what is the ideal situation for each stake. The process is currently at the point of formal approval of the knowledge base by the sounding board. With this information as a point of departure, the negotiations start off.
At this point in time, before the negotiations start, the methodology is applied. Separate sessions are being organized with the competent author-ity waterboard Velt en Vecht, to design an expert-based solution. Similar sessions are performed with the representatives of each major stake, in or-der to design their optimal solution while taking into account all stakes. The outcome of these sessions serve as the basis for the negotiations in the sounding board group and for their subsequent analysis.
During the first phase of the process, differences in the problem framing between the waterboard and the different stakeholders already became ap-parent. The waterboard has restricted the problem formulation to a water quantity problem. Some stakeholders suggested including water quality, since the nature goals pertain to this aspect. Water quality can be a bottle-neck in meeting the nature goals; joint water quantity and quality measures could be more effective or could make some water quantity measures re-dundant. The waterboard does not share the perspective of water quality being a bottleneck in the process. Regarding the policy formulation, the re-sponse options considered by the stakeholders are tending more to spatial development, but also contain technical alternatives that are considered unsustainable by the waterboard. Differences such as these are expected to result in different preferred alternatives and difference in robustness. The methodology aims to gain more insight in this process.
Conclusion and discussion
We developed a methodology to compare a stakeholder participation proc-ess to an expert- based procproc-ess in terms of the robustnproc-ess of the preferred alternatives. After testing the difference in robustness of the preferred al-ternatives, it identifies the factors responsible for that. It is hypothesized that successful stakeholder participation in policy analysis leads to more robust policy, as compared to a more traditional management of
govern-ment policy, in which experts in administrative agencies perform the pol-icy analyses. In the scientific literature, arguments have been found that support and question this hypothesis, but a study systematically investigat-ing this aspect has not been found. The developed methodology comprises a case study design that enables realistic performance of both processes, ensures their comparability and guards their quality. The developed meth-odology appears promising and is being applied for a case of developing water management alternatives at this stage. Application to multiple case studies is necessary to enable accurate statements about the influence of stakeholder participation on policy robustness. The insight obtained by ap-plication of the methodology can be used to provide arguments towards decision-makers for stakeholder involvement. Furthermore, knowledge on the stakeholders’ contribution in each phase of the analysis can be used to improve the effectiveness of a participation activity.
Acknowledgement
The research presented in this paper was funded by the EC FP6 project AQUASTRESS, IP grant 511231.
References
Arnstein SR (1969) A ladder of citizen participation. Journal American Institute of Planners 35: 215–224
Beierle TC, Cayford J (2002) Democracy in Practice. Public Participation in Envi-ronmental Decisions. Resources for the future, Washington D.C
Beierle TC, Koninsky DM (2000) Values, conflict, and trust in participatory envi-ronmental planning. Journal of Policy Analysis and Management 19(4):587– 602
Brody SD (2003) Measuring the Effects of Stakeholder Participation on the Qual-ity of Local Plans Based on the Principles of Collaborative Ecosystem Man-agement. Journal of Planning Education and Research 22:407–419
Dewulf A., Craps M, Dercon G (2004) How Issues Get Framed and Reframed When Different Communities Meet: A Multi-level Analysis of a Collaborative Soil Conservation Initiative in the Ecuadorian Andes. Journal of Community & Applied Social Psychology 14:177–192
Fiorino JF (1990) Citizen Participation and Environmental Risk: A Survey of In-stitutional Mechanisms. Science, Technology and Human Values 15(2) 226– 243
Halvorsen KE (2001) Assessing public participation techniques and comfort, con-venience, satisfaction and deliberation. Environmental Management 28(2): 179–186
Hoekstra AY (1998) Perspectives on water: a model-based exploration of the fu-ture. International Books, Utrecht, the Netherlands
Korfmacher KS (2001) The Politics of Participation in Watershed Modeling. Environmental Management 27(2):161–176
Maxim L, Van der Sluijs JP (2007) Uncertainty: Cause or effect of stakeholders’ debates? Analysis of a case study: The risk for honeybees of the insecticide Gaucho. Science of the Total Environment 376:1–17
Newig J, Pahl-Wostl C, Sigel K (2005) The Role of Public Participation in Man-aging Uncertainty in the Implementation of the Water Framework Directive. European Environment 15 DOI: 10.1002/eet.398.
Pahl-Wostl C (2007) Transition towards adaptive management of water facing climate and global change. Water Resources Management 21:49–62
Refsgaard JC, Van der Sluijs JP, Højberger AL, Vanrolleghem PA (2007) Uncer-tainty in the environmental modelling process – A framework and guidance. Environmental Modelling & Software DOI:10.1016/j.envsoft.2007.02.004 pp 1-14
Rowe G, Marsh R, Frewer LJ (2004) Evaluation of a deliberative conference using validated criteria. Science, Technology and Human Values 29(1):88-121 Walker WE, Harremoës P, Rotmans J, van der Sluijs JP, van Asselt MBA, Janssen
P, Krayer von Krauss MP (2003) Defining Uncertainty: A Conceptual Basis for Uncertainty Management in Model-Based Decision Support. Integrated Assessment 4(1):5–17
Yosie TF, Herbst TD (1998) Using Stakeholder Processes in Environmental Deci-sion-making: An Evaluation of Lessons Learned, Key Issues, and Future Challenges. Ruder Finn, Washington DC
