The ISAC Energy

(1)

The ISAC Energy

Test ing t he public private partnership regarding cyber securit y in the Dutch electricit y sector against theory o f public private partnerships in the crit ica l infrastructure

Master Crisis & Security Management

Thom Spitzen

Student Number: S2086263

Word count: 21916 words excluding references and appendices

Thesis supervisor: Dr. Vlad Niculescu-Dincă

Second reader: Dr. E. de Busser

(2)

Preface

Before you lies the result of my master thesis research in which I applied the theory of Dunn-Cavelty and Suter regarding public private partnerships in critical infrastructure protection on the Dutch ISAC Energy. Although the official starting moment of my thesis was three months ago, the preparations started months earlier. Finding a relevant and interesting topic was a process in which I met with professionals from the energy sector, the intelligence services and researchers of the Leiden University. Being part of the inner circle due to my professional job at an energy undertaking was a meaningful starting point to be able to reach out to the relevant contacts. This, together with the conversations I had before the actual thesis period started, resulted in a feasible planning for the interviews. Nevertheless, I faced some bumps in the road. Finding the persons that could tell me useful information was step one. Arranging a meeting for an interview is a second. The professionals I met, had busy schedules and a researcher from a university was not their number one priority. However, the interviewees and other persons I spoke were helpful. They sent out a message to the next person before I contacted them. This resulted in a snowballing process, which was bigger than I imagined before it started. I am thankful for everyone who contributed to this process, especially Anne Spoelstra, who started this process.

Furthermore, I want to thank everyone who participated in my research. Besides the meaningful information for my thesis research, the conversations and interviews learned me a lot about public private partnerships, cyber security, and the Dutch energy sector.

Last but not least, I want to thank the persons who helped me during the process of my thesis research. My friends and family for reading through my thesis and helping me out with the English language, and my thesis supervisor Dr. Niculescu-Dinca for his critical view and his help when I faced a problem.

Thanks to all of you, I managed to write my master thesis on an interesting topic. I hope you enjoy reading my thesis as much as I did researching and writing my master thesis.

Thom Spitzen January 13, 2019.

(3)

List of abbreviations

ACM - Authority Consumer and Market (Autoriteit Consument en Markt) AIVD - General Intelligence and Security Service (Algemene Inlichtingen- en

VeiligheidsDienst) ECH - Energy Cleaning House

EDSN - Energie Data Services Nederland EU - European Union

GovCert - Government Computer Emergency Response Team

Hz - Hertz

ICT - Information and Communicatie Technology IRB - ICT Response Board

ISAC - Information Sharing and Analysis Centre

MIVD - Dutch Military Intelligence and Security Service (Militaire inlichtingen- en Veiligheidsdienst)

NCSC - National Cyber Security Centre

NCTV - National Coordinator for Security and Counterterrorism (Nationaal Coordinator Terrorisme en Veiligheid)

NEDU - Nederlandse Energie Data Uitwisseling NIS-directive - Network and Information System – directive NPM - New Public Management

OES - Operators of Essential Services PPP - Public private partnership TLP - Traffic Light Protocol

TNO - Dutch Organisation for Applied Scientific Research (Nederlandse Organisatie voor Toegepast Natuurwetenschappelijk Onderzoek) Wbni - Wet Beveiling Netwerk- en informatiesystemen

(6)

Abstract

The Dutch government classified the distribution of electricity as a vital process. The distribution of electricity in the Netherlands is in private hands. To ensure the continuity of electricity and protection against cyber threats, the government and private actors cooperate in Public Private Partnerships (PPPs).

This research identifies the relevant PPPs regarding cyber security in the Dutch electricity sector. Secondly, the most relevant PPP is tested against the network approach of governance theory of Dunn-Cavelty and Suter. This theory describes an approach to govern PPPs in critical infrastructure protection that solves several problems when using PPP in critical infrastructure protection.

Within the first section, I identified three PPPs regarding cyber security in the Dutch electricity sector. The NEDU/EDSN, an association that is occupied with facilitating the exchange of information between market participants when a customer wants to switch from an energy supplier. The exchange of this information must be secured. The second partnership is the branch organization for the Dutch gas and electricity distribution system operators: Netbeheer Nederland. This branch organization has a theme group Cybersecurity & Crisismanagement and their goal is to ensure an equal level of security within the chain. The last partnership is the ISAC Energy, a PPP in which the public and private actors involved in the Dutch energy sector share and analyze information regarding cyber threats, incidents or other issues. The ISAC Energy is the most relevant PPP since this partnership matches the scope and focusses completely on the critical part of the Dutch electricity sector.

The ISAC Energy meets most of the criteria of the network approach of governance theory and tackles two of the problems for PPPs in critical infrastructure protection. One of the problems, that due to the involvement of the government, international cooperation is harder to achieve, appeared to be the opposite. The involvement of the government does foster international cooperation. Another problem, which states that without this network approach the PPP can only have a limited amount of members, is not considered as a problem by the members of the ISAC Energy. Even more, the members want to limit the amount of members. The last problem cannot be solved by applying the theory, yet the interviewees recognize the problem.

(7)

These conclusions show us that the main PPP in the electricity sector, the ISAC Energy, mostly meet up with the network approach of governance theory. Therefore, I conclude that the ISAC Energy is organized in a way that solves some of the problems regarding PPPs in critical infrastructure protection. Furthermore, it proves that the governance of the ISAC Energy is organized in a useful manner and can be used as an example for other PPPs in critical infrastructure protection in the Netherlands. For both the theory and the ISAC Energy, recommendations are given.

(8)

1. Introduction

Our society is dependent on electricity. Our day to day life would be disrupted if a power outage would occur. Many vital processes would not work if the electricity grid was down. It is therefore not a surprise that many states have classified the power grid as a critical infrastructure of their state. As shown in books like Black Out (Marc Elsberg) and Countdown to Zero Day (Kim Zetter), a hack on electricity undertakings or the power grid can be devastating. Marc Elsberg describes how the (future) smart grid can be hacked with catastrophic consequences for Europe and shows our dependency on electricity. Kim Zetter describes Stuxnet, the “world’s first digital weapon”, and shows how a digital weapon can result in physical destruction within an electricity plant (Elsberg, 2012; Zetter, 2015).

In response to the dependency on digital tools and the increasing cyber-threat, the European Parliament implemented the European Network and Information System (NIS) directive. The NIS-directive classifies the electricity sector as an essential service for the continuity of network and information systems (European Parliament, 2016). Furthermore, the Dutch National Coordinator for Security and Counterterrorism (NCTV) classified certain processes that can cause severe social disruption. The distribution of electricity is one of these vital processes (NCTV, n.d.).

Besides plans and regulation, the international power grid has faced new threats in the last decades. One significant threat is cyber security threats. The British intelligence services alerted Britain’s key power companies to improve their cyber defense in March 2018 (Shipman, Kerbaj, & Wheeler, 2018). In addition, the example of the Ukraine blackout following a cyber-attack in 2015 shows the vulnerability of power grids to cyber-cyber-attacks (Sulleyman, 2017). The above shows that societies must have solid cyber security to protect their vital processes. Although the stated examples are in an international context, it also applies to the Netherlands. The Netherlands is a well-connected state in both the physical and digital world. The power grid in the Netherlands is connected to the power grids of other European states, as shown in the “Wekkergate”. Due to a conflict between Servia and Kosovo, the frequency of the grid in Europe was a little below 50Hz, resulting in a delay in all the digital clocks in Europe (AD, 2018).

(9)

The Dutch electricity sector is in private hands, but the cyber-security of the Dutch electricity sector is organized in several public private partnerships (PPPs) that involves both public and private actors. Different actors with different motivations that need to cooperate is challenging, especially in such a crucial sector.

This study focuses on the Dutch electricity sector. There is little to no research conducted on how the Dutch electricity sector organizes cyber security, but Baauw researched how and when PPPs contribute to cyber security (Baauw, 2018). He used the Dutch electricity sector as a case study (see 2.3 Identifying the gap in knowledge). By testing the way the PPP is organized, crucial lessons can be learned to improve the governance of the electricity sector. In the light of increasing cyber threats from different actors and the numerous advanced persistent threat incidents, this study contributes in responding to the current environment of cyber security. The research can lead to improvements to govern a PPP regarding cyber security in the Dutch electricity sector.

Furthermore, this study contributes to the scientific field of cyber security and tests the applicable theory of Dunn-Cavelty and Suter. This study questions whether their theory is also applicable to the critical electricity sector of the Netherlands.

The research question is as follows:

To what extent does the PPP in which cyber security is organized for the Dutch electricity sector meet up with the success criteria of the network approach of governance theory of Dunn-Cavelty and Suter?

To help answering the research question, I answered two sub questions.

1. What do theories and/or other researches argue about the concepts of cyber security and public private partnerships (in critical infrastructure)?

2. Which public private partnerships regarding cyber security are active in the Dutch electricity sector and which one is the most relevant?

The research goal is explanatory research by applying the theory of Dunn-Cavelty and Suter

(10)

The theory of Dunn-Cavelty and Suter has several criteria that a PPP in the critical infrastructure should meet to solve the problems of public private cooperation in critical infrastructure protection. These criteria are the guidelines for the analysis.

To answer the research questions, documents containing information regarding the PPPs and research documents have been studied. Following the analysis of these documents, interviews were conducted with relevant experts in the field of cyber security in the Dutch electricity sector. These experts are employees of public and private organizations, but it must be clear that there is no clear line between public and private. Some organizations in the Dutch electricity organizations are classified as private and act like a private company, yet public organizations are the shareholders and therefore the “owner” of the organization.

The Dutch electricity sector is partly classified as critical infrastructure. The distributors are labeled as critical, but the producers are not (yet) critical infrastructure (see 1.1. scope of the research). The Dutch organization that is responsible for identifying the critical organizations is the NCTV. They identified a list of sectors and their tasks. This classification is subjective and there is no unambiguous list of the critical organizations in the electricity sector since it has no legal basis. Therefore, this research does not use the current classification of the NCTV to determine the scope of the research.

1.1 Scope of the research

Since this research focusses on PPPs which involves both the public and private sectors, both should be included in the scope. To properly determine the scope of this research, this research uses the definitions given in the NIS-directive to determine which private organizations are relevant for this research. The NIS-directive came into effect on May 9, 2018, and provides legal measures on how the EU-member states should structure (parts of) their national cybersecurity management. The NIS-directive provides clear definitions of what should be included in the critical infrastructure of an EU-member state and will replace the current classification of the NCTV in 2019. This directive identifies criteria for “Operators of essential services” as “a public or private entity that:

A. Provides a service which is essential for the maintenance of critical societal and/or economic activities;

(11)

C. An incident would have significant disruptive effects on the provision of that service” (European Parliament, 2016).

Furthermore, the NIS-directive identifies certain types of entities that need to meet the criteria stated above, to be classified as an operator of essential services. In the electricity sector, three types of entities are identified:

1) ‘Electricity undertaking’ means any natural or legal person carrying out at least one of the following functions: ‘supply’ meaning the sale, including resale, of electricity to customers (European Parliament, 2016; European Parliament, 2009);

2) ‘Distribution system operators’ means a natural or legal person responsible for operating, ensuring the maintenance of and, if necessary, developing the distribution system in a given area and, where applicable, its interconnections with other systems and for ensuring the long-term ability of the system to meet reasonable demands for the distribution of electricity (European Parliament, 2016; European Parliament, 2009);

3) ‘Transmission system operator’ means a natural or legal person responsible for operating, ensuring the maintenance of and, if necessary, developing the transmission system in a given area and, where applicable, its interconnections with other systems, and for ensuring the long-term ability of the system to meet reasonable demands for the transmission of electricity (European Parliament, 2016; European Parliament, 2009).

Some organizations within the scope cannot be classified as purely private since the government is their shareholder. Therefore, the distinction between public or private organization is not correct; there is a grey area.

To determine which public organizations are included in the scope, a definition from the Dutch Police of the public sector is used “Collective name for all government organizations and

semi-government organizations (PolitieAcademie, n.d.)”.1 Not all these organizations are relevant to

this research, but only those organizations concerned with cybersecurity or the electricity sector. Therefore, the public sector is identified as all the (semi-)government organizations related to

cybersecurity of the electricity sector.

(12)

In conclusion, the scope of this research is the Dutch electricity sector: the electricity

undertakings, distribution system operators and transmission system operators that meet up with the criteria of operators of essential services in the Netherlands accordingly the NIS-directive and, the (semi-)government organizations related to cyber security of the electricity sector.

1.2 Reading Guide

The next chapter discusses and answers sub question one: the relevant theories and researches. Chapter three explains the research method, case selection, and operationalization. The main PPPs regarding cyber security in the Dutch electricity sector are discussed in chapter four; this chapter also describes the liberalization of the Dutch energy market, which must be described to understand the context of PPPs in the energy sector. Chapter five discusses the analysis of interviews and documents related to the ISAC Energy and whether the ISAC Energy meets the criteria of the theory of Dunn Cavelty and Suter. Following the analysis, the conclusion and recommendations are presented.

(13)

2. Theoretical framework

In this chapter, I reviewed relevant existing theoretical frameworks and structured an argument regarding the applied theory. Sub question one is answered in this chapter; what do

other theories and/or researches argue about the concepts of cyber security and PPPs (in critical infrastructure)?

2.1 Defining Cyber Security

The term cyber security is defined in many different ways. Furthermore, the term is used as an all-embracing term with vague boundaries. Concepts like information security or Information and Communication Technology (ICT)-security overlap but are not the same as cyber security. Rossouw van Solms and Johan van Niekerk distinguish the differences and identify the relations between information, ICT and cyber security. They argue that Information

security is the protection of information-based assets, stored or transmitted but not by using

ICT, from harm resulting from threats and vulnerabilities. ICT-security is securing information-based assets stored or transmitted by using ICT. Cyber security is defined as “the protection

of cyberspace itself, the electronic information, the ICTs that support cyberspace, and the users of cyberspace in their personal, societal and national capacity, including any of their interests, either tangible or intangible, that are vulnerable to attacks originating in cyberspace (Von

Solms & van Niekerk, 2013, pp. 101, ).”

The NCTV, responsible for the protection of the critical infrastructure in the Netherlands, defines cyber security as follows: “freedom from danger or damage caused by the disruption,

failure or abuse of ICT systems. This danger or damage may consist of a limitation in the availability or reliability of ICT systems, a breach of the confidentiality of information stored in them, or damage to the integrity of that information (NCTV, n.d.).”

The two given definitions of cyber security correspond largely. Although this research is about the Dutch electricity sector, the definition given by Van Solms and Niekerk is used, since their definition more clearly emphasizes the differences between information, ICT, and cyber-security. These different definitions are often used in a wrong context and people do not know the exact differences between the definitions. To be clear regarding the meaning of cyber security in this research, and for showing what I researched, the definition of Van Solms and

(14)

Niekerk is used since they clearly define what cyber security is in relation or in contrast with information and ICT security.

2.2 Defining Public Private Partnerships

Since the Dutch electricity sector is privately owned and many private actors are involved in this sector, models for PPP or cooperation are useful to measure the effectiveness of the way in which cyber security is organized in the Dutch electricity sector.

PPPs are defined in different ways. Sometimes other concepts are used to describe the cooperation between public and private actors. Klijn and Teisman describe this as a public private cooperation2 and define it as “more or less a sustainable cooperation between public

and private actors in which common products and/or services are developed and in which risks,

costs and output are shared 3_{(Klijn and Teisman, 2000, pp. 157). Dunn-Cavelty and Suter gave}

another definition of a PPP: “a form of cooperation between the state and the private sector

(Dunn-Cavelty & Suter, 2009, pp. 179).” The definition of PPP is arguable; however, the

definition given by Dunn-Cavelty and Suter is general and applicable to this research.

The idea behind a PPP is that this collaboration achieves benefits, which cannot be achieved by solely the public or private actor. The assumption of the added value is debated amongst academics. Some argue that PPP is inspired by New Public Management (NPM). NPM argues that the public sector focuses on policy and the private actor should do the implementation of this policy. This kind of collaboration should increase efficiency and effectiveness since the private sector can work more efficiently than the public sector. The other assumption regarding PPP is that through the bundling of information, knowledge, and resources, better coordination and output can be realized. This kind of collaboration is inspired by governance and networking literature. The public and private actors work together on a horizontal level and are dependent on each other (Klijn & Twist, 2007).

2_{Partnership cannot be freely translated to Dutch. In the Netherlands a PPP is often named a Publiek Private}

Samenwerking, which is freely translated a public private cooperation.

(15)

2.2.1 Forms of Public Private Partnerships

There are three different assumptions regarding PPP that lead to various forms of organizations. In the first form, concession or contract form, the government describes the output criteria and the private party is responsible for the execution. Examples are a design, build, finance, or maintain contract. This can lead to innovative ideas from the private party involved and forces this private party to think ahead for future problems, like during maintenance. The responsibilities are separated and both parties have their own responsibilities. This model is often called an economic cooperation and is generally used when large-scale restructuring or investment is needed (Klijn & Twist, 2007; van Montfort, van den Brink, Schulz, & Maalsté, 2012).

The second form is a partner or alliance partnership. This partnership is an organizational collaboration to handle an incidental project or to collaborate for an undetermined period. The public and private sector work together to address a problem and find a solution. Within this partnership, there is less hierarchy between the public and private sector and more trust amongst the members. The starting point of such a partnership is the partners rather than a concrete plan (Klijn & van Twist, 2007; van Montfort, van den Brink, Schulz & Maalsté, 2012).

The concession form is seldom used in the security sector. The alliance form is the more common form in the security sector (van Montfort, van den Brink, Schulz, & Maalsté, 2012). The last form is the improvisation model. This partnership is an ad hoc form, on a small scale, in which the public and private actors seek and improvise to ensure more security. When the government stops fulfilling a certain task, for example, due to budget cuts or when society does not agree with the current way of performance, this partnership fills the gap (van Montfort, van den Brink, Schulz, & Maalsté, 2012).

2.2.2 Conducted research on Public Private Partnerships in cyber security and critical infrastructure

Since this study researches cyber security and critical infrastructure, models regarding PPPs in cyber security and critical infrastructure are addressed in the following paragraphs. Dunn-Cavelty & Suter, Madeline Carr and Sergei Boeke described or applied models for PPPs in cyber security and critical infrastructure.

(16)

2.2.2.1 The network approach of governance theory

The first model is the model of Dunn-Cavelty and Suter that sketches an approach to PPPs in critical infrastructure protection. They argue that PPPs are essential, yet “generating security for citizens is a core task of the state; therefore it is an extremely delicate matter for the government to pass on its responsibility in this area to the private (Dunn-Cavelty & Suter, 2009, pp. 181).” The privatization and deregulation since the 80s together with globalization resulted in a largely privately owned critical infrastructure (see also 4.1 Liberalization of the energy market). The government has no control over the security of these critical infrastructure enterprises, but remains responsible for the security of the society, as the above quotation states. The cooperation between the private critical infrastructure and the government, in the form of PPPs, is criticized for efficiency. The PPPs in the critical infrastructure should not raise efficiency but need to strengthen their security instead (Dunn-Cavelty & Suter, 2009).

Therefore, the authors argue that applying the network approach of governance theory on PPP can be useful to address the problems that arise in PPPs within the critical infrastructure. The neoliberal approach of governance focuses on precisely defined tasks for the private sector. If they fail the government steps in and takes over the essential services of the private sector. This is not useful since the government is not able to perform such specialized tasks. The aim of this approach is the enhancement of efficiency. However, the goal of critical infrastructure protection should be increasing security and not increasing efficiency (Dunn-Cavelty & Suter, 2009). To address this problem, Dunn-Cavelty and Suter present the network approach of governance theory in which the governance structures are, in contrast to the neoliberal approach, not considered a way to improve efficiency. The governance structures are a consequence of progressive specialization in modern societies. Due to the increased specialization, the government cannot maintain control since they do not have the required specialized knowledge. The network approach of governance argues that the government must shape the framework conditions for private actors to perform their tasks, even without constant oversight. Therefore, public administration must become a team sport in which persuasion, negotiations, and mutual trust are more important than regulation and control. The network monitors itself since its members have the specialized knowledge to check their obligations. The government is represented in this network but does not have a special status (Dunn-Cavelty & Suter, 2009).

(17)

This approach leads to a self-regulating network in which the state coordinates and stimulates the functional networks instead of monitoring the involved actors. The government does check whether the network fulfills the public tasks; if not, the state activates new networks for these tasks. This way of indirect control is called meta-governance. Meta-governance consists of creating framework conditions for the network to organize, and coordination and promotion activities. The most crucial responsibility of the government is choosing the right instruments to promote the networks (Dunn-Cavelty & Suter, 2009).

This approach to PPP in critical infrastructure protection offers a new form of government that is a middle way between the absolute state control on provision of security and the hands off policies regarding the security of critical infrastructure (Dunn-Cavelty & Suter, 2009).

2.2.2.2 Public Private Partnerships in national cyber security strategy

The second model is of Madeline Carr, who elaborates on PPPs within national security strategies of the UK and US in her article “Public private partnerships in national cyber-security strategies”. She developed a comprehensive understanding of the roles of policy makers and the private sector in national cyber security. Firstly, the tensions and competing agendas of the PPP need to be clearly understood and spoken about. The weaknesses of PPPs must be openly acknowledged to address these issues. The public sector must understand that the private sector has an aversion to accept responsibility or liability for national security. The private sector approaches cyber security as a cost/benefit framework. Carr argues that a PPP must be based on shared interest or governed by rules. She stresses that the national cyber security strategies of the US and UK are based on PPPs and it is therefore important to acknowledge the weaknesses of these partnerships (Carr, 2016).

2.2.2.3 National cyber crisis management

Several researches are conducted regarding PPPs. Still, these studies are rarely conducted on themes related to cyber security in the Netherlands. Boeke looked into the national cyber crisis management of four different EU-members, including the Netherlands. He stressed that “PPPs feature as the cornerstone of many national cybersecurity strategies (Carr, 2016).” This corresponds with the authors stated above. Boeke applied the three networks of governance identified by Provan and Kenis on the national cyber crisis management of the EU members (Boeke, 2018).

(18)

The three forms of networks are lead organization-governed networks, participant-governed networks, and network administrative organization. The lead organization-governed networks consist of one single participating member, which coordinates all the major network-level activities and important decisions. This network is centralized and the single participating member has more power than the other members have. The participant-governed network is the simplest form of governance. The members of the network govern the network and there is no separate or unique governance entity. This model can be decentralized, named shared participant governance in which the members interact on equal basis, or highly centralized in which the network is governed by and through a lead organization, which is a member of the network. The last network, the network administrative organization, includes distributed authority and responsibilities among the members. One network broker is appointed and mandated, but the broker does not have the responsibility for the network: his role is monitoring and leading the network (Provan & Kenis, 2008).

Boeke identified the Dutch national cyber crisis management as a participant-governed network. The public and private partners trust each other and are equal. There is a central node that facilitates cooperation, but the expertise is decentralized. This central node cannot enforce regulations. Sectoral inspectorates do this. Furthermore, Boeke outlines the structure and all the relevant actors in the Netherlands. He does not look into specific sectors but approaches the structure on national level. Boeke did not look into the specific critical infrastructure sectors in the Netherlands, like the electricity sector. His research contributes in sketching the broader context of this research; he shows the importance of the public private cooperation in solving cyber crises (Boeke, 2018).

2.2.2.4 Effects of using a PPP on cyber security

Whereas Boeke in the previous section researched cyber security at a (Dutch) national level, Floris Baauw went more into the cyber security of specific Dutch critical infrastructure sectors. He researched two PPPs in the Dutch critical infrastructure, the Information Sharing and Analysis Centre (ISAC) of the Dutch telecom sector and the ISAC of the electricity sector. An ISAC is a PPP in which the participants share information and experiences about cyber security. Baauw focused on the effects of using a PPP in the critical sectors and if the PPP is beneficial towards cyber security. Baauw concluded that the ISAC Energy is the main PPP in the electricity sector that focuses on cyber security. The researched ISACs (Energy and Telecommunication) are characterized as a form of the alliance model (van Montfort, van den

(19)

Brink, Schulz, & Maalsté, 2012). Furthermore, he concluded that the PPPs have a positive effect on the interaction between the partners. The partnerships do not save any costs and have not proven to be improving the information efficiency. The partnerships are beneficial for implementing cyber security policies since the government organizes working groups or consultation rounds via the ISACs. Baauw argues, “The biggest challenge currently playing in the PPP in the energy and telecom sector is fixation (Baauw, 2018, pp. 64).” The field of cyber security is complex, and sticking to the current situation can become problematic. His arguments show that PPPs in critical infrastructure are challenging. Therefore, it is useful to test the PPP in the Dutch electricity sector against a theory of PPP in critical infrastructure (Baauw, 2018).

2.3 Identifying the gap in knowledge

There have been researches conducted on PPPs and how they can be organized. Some of this research was focused on cyber security aspects, like strategy and critical infrastructure. Furthermore, Boeke researched the governance of the Dutch national cyber security organization. Other studies, such as the one of Baauw focused on a PPP in the Dutch electricity sector, but he focused on the effects of PPPs on cyber security. No research is performed on testing the PPP in the Dutch electricity sector against a theory of PPP in critical infrastructure protection. Research on this topic contributes to the body of knowledge of cyber security governance and organization. The results of this study can improve the theory of Dunn-Cavelty and Suter and the knowledge on PPPs in critical infrastructure protection. Even more, this study shows whether the theory is applicable to PPPs in the Netherlands. Additionally, the results can be an example of how to organize cyber security in critical infrastructure protection and how PPPs can be used in this field. In light of the increasing dependency on digital tools and the increasing threats, it would be useful to look into the PPP regarding cyber security in the electricity sector. As shown above, researchers looked into themes that touched upon cyber security in general, PPP in cybersecurity, and Dutch PPP in cyber security, but not testing a theory of PPP in critical infrastructure protection on a cyber security PPP in the electricity sector. Therefore, it is useful to research this specific topic.

2.4 Choice of theory

The model of Provan and Kenis classifies PPP into a certain network model. This model does not argue how to improve a PPP. Even more, the model focusses on PPPs in general. Their

(20)

theory allows classifying a certain PPP in a model and shows what can be beneficial about that certain model. Although this can be useful for “normal” PPPs, PPPs in the critical infrastructure are special. Even more, PPPs in critical infrastructure that focus on the protection of critical infrastructure are more exceptional. The theory of Provan and Kenis does not argue how a PPP for the protection or security of critical infrastructure should ideally be organized. Since this research focusses on PPPs regarding cyber security in critical infrastructure, the model of Provan and Kenis cannot be applied.

Carr’s theory that concerns national cyber security strategies focusses, as the title of her research states, on a national level and on strategies. It does address certain principles regarding the cooperation between public and private actors, yet she does not address how a PPP should be organized and which roles are needed. Her model does focus on cyber security, which is one of the main themes of this research but puts a limited focus on the other main theme of this research: PPP. Her model focuses on the national level, including the total critical infrastructure of a nation. This research focuses on a PPP in one certain sector of this critical infrastructure. Even more, Carr’s theory is about making national cyber security strategies. This research is not about strategy, but how a PPP regarding cyber security in a critical sector should be organized.

The theory of Dunn-Cavelty and Suter focusses more how a PPP should be organized. This theory discusses PPP in critical infrastructure protection, which makes it more applicable to the scope of this research. Furthermore, the theory explains how the given structure of a PPP can help solve PPP related problems. The theory of Dunn-Cavelty and Suter is applicable to one certain PPP in a critical sector. Their theory is also applicable to the theme of cyber security since cyber security is part of protecting critical infrastructure. Furthermore, the network approach of governance theory present indicators that are adequate to measure via interviews and document analysis. The theories of Provan and Kenis and Carr, mentioned before, are not suitable for this research. The theory of Dunn-Cavelty and Suter is applicable to this research. Therefore, I used the theory of Dunn-Cavelty and Suter.

(21)

3. Describing the research method

In the previous chapter, I discussed the theoretical framework and the chosen theory. To apply the theory and conduct the research, I discuss the research design, case selection, operationalization, methods and validity in this paragraph.

3.1 Type of research

The research consists of two parts:

1. Identification and delineation of the PPPs regarding cyber security in the Dutch electricity sector.

2. Testing the PPP for cyber security in the Dutch electricity sector against the network approach of governance theory of Dunn-Cavelty and Suter.

In the first section, I identified which PPPs regarding cyber security are active in the Dutch electricity sector. This identification included clarifying how the different organizations and actors are related. The identification started by talking to professionals in this field. When it became clear which PPPs are relevant, based on matching the research scope and the opinion of the professionals, I chose these organizations to elaborate on and asked the professionals if they know persons that I could contact. This started a snowball effect since, through all these persons, I found the right professionals to interview. These professionals had many experience regarding PPPs and/or were closely involved in the PPP. What helped the most during this snowball process was that the first persons I met sent a message to the new persons before I contacted them. This way, I successfully interviewed one person per PPP to go more in depth and to found out which PPP is the most relevant for my research.

Secondly, I chose one PPP that matched the scope and research goal, and is, therefore, the most relevant for my research; see the argumentation in paragraph 4.5 Choosing the most relevant PPP. For contacts regarding this PPP, I once again used the snowballing process. This snowballing process kept on going during the period of my research, resulting in more information and conversations than previously planned. Conversations with informal contacts resulted in important information, personal opinions, and small favors. Thanks to these informal contacts, I managed to speak to the key persons of the PPP I wanted to research, like the

(22)

chairperson, the vice chairperson, and persons from the public sector that were hard to contact. Persons from the public sector were hard to interview since they often had busy agendas and do not like to forward you to a colleague. However, the snowballing process kept on going and thanks to the informal conversations and the relations that were (being) built, some persons wanted to take an extra step for me, resulting in interviewing the wanted persons and collecting the needed data.

I applied a theory on the chosen PPP. This PPP was tested against the theory of Dunn-Cavelty and Suter, see paragraph 2.2.2.1. The network approach of governance theory. The PPP was not only tested against the theory, but I also went more in depth and discussed the discrepancies and similarities between the interviewees, and challenged the theory against my findings.

Hence, the research started with identifying the PPPs regarding cyber-security in the electricity sector. The input was derived from interviews, public and non-public information. Secondly, I tested this organization against the model of Dunn-Cavelty and Suter.

3.2 Selecting the case

The selected case consists of the organizations involved in cyber security in the Dutch electricity sector, as stated in the scope. This includes both public and private organizations. The case selection includes the producers and distributors of electricity and the relevant authorities regarding cyber security in the electricity sector. To conduct the research, several persons in the public and private sector were interviewed.

Interviewees:

- Jos Weyers. He is an employee of TenneT who is concerned with cyber security and is involved in partnerships regarding cyber security in the Dutch electricity sector. He is the vice-chairman of the ISAC Energy. TenneT is the transmission system operator in the Netherlands and is therefore a key organization in the Dutch electricity sector. - Luisella ten Pierik. An employee of Stedin who is concerned with cyber security and

is involved in partnerships regarding cyber security in the Dutch electricity sector. She is the chairperson of the cyber security group of Netbeheer NL and a member of the ISAC Energy. Stedin is a regional distribution system operator.

(23)

- Anne Spoelstra. He is an employee of Eneco Group who is concerned with cyber security. He is a member of the ISAC Energy. Eneco Group is one of the largest producers of electricity in the Netherlands. Furthermore, they operate two black-start units to restore the electricity in case of a blackout.

- An employee of the National Cyber Security Center (NCSC). The NCSC is the part of the Dutch NCTV (as of 29 October 2018) and focusses on strengthening the Dutch society regarding cyber threats.

- Rene Marchal. He is the former senior manager Safety & Security at the TenneT Group and now temporary seconded to the NCTV as a special advisor for strategic partnerships.

- Hidde Brugmans. He is a policy officer for the Ministry of Economic Affairs and Climate. They are responsible for the continuity of electricity in the Netherlands and promote clean, reliable energy in the Netherlands.

- Gerrit Fokkema. He works for EDSN and NEDU. The NEDU is a platform that facilitates the exchange of information for organizations in the Dutch electricity sector.

The interviewees agreed that I transcribed and analyzed the recorded interviews. They stated clearly that they did not want to be quoted unless it is anonymous and verified with the interviewee. The interviewees wanted to check whether their statements were correctly interpreted, that they did not say incorrect or seemingly wrong statements, and they wanted to check the used information to prevent something sensitive from being published. I verified with the interviewees if they agreed on the research part that will be publicly available and what will not be public. One interviewee did not want to be mentioned by name and is therefore anonymous. Since the interviewees do not want their quotes in the public thesis, the coding of the interviews is not publicly available.

Besides the conducted interviews, several conversations took place in order to find relevant information. These conversations are not recorded since they took place during informal meetings, congress meetings or work appointments. Most of those conversations can be classified as small talks. These conversations were, among others, with the chairperson of the ISAC Energy and a researcher of cyber security.

(24)

3.3 Operationalization

I tested one partnership regarding cyber security against the success criteria of Dunn-Cavelty and Suter. This network approach of governance theory consists of criteria that lead to a successful PPP for critical infrastructure protection. The authors of this theory did not classify the criteria in themes. To make it more clear, I have grouped the criteria.

General criteria:

- The PPP is organized as a small and relatively homogenous network that involves all actors who will and can contribute to the fulfillment of a public service in their own interest;

- Persuasion, negotiations and mutual trust are more important than control and regulation;

- The network itself has the responsibility to control the PPP; - The PPP / network is self-organizing.

- Due to the private actors in the network, the network can reach out more easily to international partners.

Members of the network:

- The members fix rules for common actions and determine the responsibilities and commitment of the members;

- The government authorities are represented by the responsible agencies and they do not have a special status or authority;

- All the members are equal;

- The members in the network know each other well;4

- The contribution of the government should be meaningful. Monitoring:

- The network monitors itself since only the members have sufficient expertise to check each other;

- The members assess whether the cooperation is sufficient.

4_{This criteria is coded together with “The members assess whether the cooperation is sufficient”, since one is a}

(25)

Meta governance:

- The government takes the role of coordinating and stimulating the network. The government mainly implies coordination and promotion activities;

- The government verifies if the tasks of the PPP are carried out, but does not check the member directly;

I used these criteria to draft questions for the interviews. The interview questions can be found in the annexes. Not all the interviews are based on these criteria. The interviews with the NEDU/EDSN, NCTV and Ministry of Economic Affairs and Climate are not based on these criteria, since these interviews were used to identify the different PPPs in the electricity sector.

The coding scheme of the interviews that focused on the theory can be found in paragraph 5.1 Coding scheme. When developing the codes, I used both pre-set/hypothesis-guided codes and presentational codes. I created the themes and criteria together with a student of the Master Crisis and Security Management, Tessa Mulders. She uses the same theory but applies it to another Dutch critical sector, the drinking water sector. Since Dunn-Cavelty and Suter did not clearly state the criteria and themes, it was useful to discuss this with a fellow student. The discussion led to a broader view and prevented a possible limited view by one of us. Even more, we went more in depth and challenged each other to explain why we should describe a criterion or theme the way we presented.

3.4 Methods of data collection

The needed data is openly available as well as not publicly available, and can be confidential. Data regarding the general structure of how cyber security is organized is partly openly available. More in depth information is not public, but the involved organizations possess this information through documents or personal knowledge. I collected the openly available data through the internet or requests to the relevant organizations. The not publicly available and confidential information was collected through my network. I work in the field of crisis management in the electricity sector and therefore have connections with other electricity companies, authorities, and other relevant organizations. Since I am part of the “inner circle”, I successfully got access to the required information.

(26)

I collected the empirical data through desk research, interviews, and document reviews. I identified the different partnerships through desk research and interviews with the members of these organizations. The interviews were semi-structured, since identifying the real organizational structure requires in depth conversations. These kinds of interviews enabled me to ask more and deeper questions regarding a certain topic.

The sampling of people and documents is based on relevancy. Some people have more knowledge regarding this organization due to experience, their role or their organization. Some documents contain more useful information than others and therefore these documents are preferable. However, since the field of research is sensitive and this research has limitations in time, the sampling faces practical issues. However, I succeeded in finding the persons that could tell me the needed information.

3.5 Methods of data analysis

To identify the structure of how cyber security is organized, I collected, reviewed and analyzed the data. These data consist of interviews and documents. The documents can be evaluations of incidents or exercises, governmental reports and documents, and documents of the private parties. I used the documents to understand the basics of cooperation regarding cyber security in the electricity sector. For more in depth information, I used interviews.

Regarding testing the organization structure against the model, I looked for characteristics of the applied model. I found the characteristics in the documents or sections of these documents and interpreted this to find out whether it corresponds with the model. During the interviews, I asked specifically or indirectly for these characteristics and if needed, I asked more questions related to one of the characteristics.

To analyze the interviews and document, I used a coding scheme. This coding scheme includes indicators of the criteria of the theory. These indicators are both hypothesis-guided indicators, derived directly from the text in which the theory is described, and presentational indicators, derived from the collected data. The coding scheme can be found in chapter five.

(27)

3.6 Validity and reliability issues

This research was performed in a period of ten weeks. Furthermore, the partnerships regarding cyber security in the electricity sector are extensive and sometimes sensitive. Although I have access to this inner circle, it was not possible to address every person involved in the way cyber security is organized. I was aware of this and to improve the validity of the research, I tried to address the relevant positions in the PPPs. I based this relevancy on the scope and goal of the research and asked the persons I spoke to which persons could tell me more about my research topic.

The used research methods, interviews, and document reviews have, cons. However, since I apply two research methods to collect data, I strengthened the internal validity. Triangulation of different methods decreases the validity issues of the methods.

Since I applied content analysis and conducted interviews, of which interpretation can be part, the analysis can be subjective. Sometimes the text or statement does not explicitly correspond with the criteria, but by interpreting the sentences, it does. I have described when a certain section is categorized in a certain category and when I interpreted a section, I gave an argumentation. I tried to be as objective as possible; however, I am aware of a possible subjective judgement. I discussed the results of the analysis of (possible) confidential information that I analyzed, was discussed with the information provider and I asked for their approval.

(28)

4. Cyber security partnerships in the Dutch electricity sector

This chapter describes the identified partnerships regarding cyber security in the Dutch electricity sector and the position of the Ministry of Economic Affairs and Climate Policy. Before describing the partnerships, this chapter shortly describes the liberalization of the Dutch energy market. It is important to understand the liberalization of the Dutch energy market to understand how the Dutch electricity sector is organized. The liberalization led to private actors in the market and therefore it opened the possibility to establish PPPs.

4.1 Liberalization of the energy market

Before the liberalization, the Dutch electricity sector used to be rather simple. There were energy companies per region and the customer had no choice for a preferred supplier. The local government owned the energy companies, and the sector was not complex and was well organized. Customers could not switch between energy suppliers and the balance on the net was organized per energy undertaking. When one energy company would have a period of maintenance at their electricity plant, the neighboring energy company was contacted and stepped in to compensate to maintain the balance on the electricity net (Energieleveranciers, n.d. and Interview with Fokkema, 2018).

In the mid ’90s, the European Union decided that the energy market should be fully liberalized and privatized by 2008. The process of liberalization in the Netherlands was completed in 2004 and led to a free choice for the customer in choosing a supplier. The liberalization forced a split in roles over various organizations, which were previously incorporated in one organization. These organizations are privately owned, quasi government organizations, government owned organizations or non-profit organizations. The different roles include energy producers, suppliers, distribution system operators, transmission system operators, metering responsible, and program responsible. Furthermore, the Dutch government remains responsible for the continuity of electricity, while private organizations are fulfilling these tasks. The various involved organizations with different roles led to an increase in communication regarding all kind of topics, varying from switches between customers to balancing the net to cyber security (Interview with Fokkema, 2018).

(29)

Since 2004, several initiatives to cooperate have arisen; initiated by both the government and the private organizations. Several of these initiatives exist until today and have evolved to handle more tasks than initially was intended. Besides the evolving initiatives, new collaborations started to face new threats and handle upcoming trends. The main partnerships regarding cyber security in the electricity sector, as of 2018, are the NEDU/EDSN, Netbeheer Nederland and the ISAC Energy (Interview with Spoelstra, 2018).

4.2 Describing the first PPP: NEDU/EDSN

The association Nederlandse Energie Data Uitwisseling (NEDU), freely translated Dutch Energy Data Exchange, and the Energie Data Services Nederland (EDSN), freely translated Energy Data Services Netherlands, are strongly related organizations. Until a few years back, the NEDU office did not exist on its own but was part of the EDSN. To understand the roles of the EDSN and NEDU, we need to go back to the liberalization of the energy sector in the Netherlands (EDSN, n.d. and NEDU, n.d.).

Due to the liberalization of the energy sectors, customers were able to choose the energy supplier they wanted. The free market resulted in customers switching from energy supplier. These switches yielded many mutual communications between the energy suppliers, distribution system operators and other involved market parties. The communications consisted of information on the home connection, the estimated (annual) energy usage, the actual energy consumption, and personal information. To keep all these communications organized, an institute was established to function as a central facilitator for all these communications. The three main energy companies of the Netherlands back then, Eneco, Nuon, and Essent, established this institute called the Energy Cleaning House (ECH). Participation was on voluntary basis, but the sector acknowledged in 2006 that this should be mandatory. From that moment, ECH was named EDSN (Interview with Fokkema, 2018).

At that moment, ECH was occupied with operational tasks and the members of ECH determined what tasks to develop and how these operational tasks should be carried out. At the time of the migration to EDSN, some market-facilitating functions of the distribution system operators were centralized and operated from EDSN. The distribution system operators wanted more to say in the activities of the EDSN, therefore the determination of these tasks was disconnected from EDSN and the NEDU was established. Thus, EDSN is the facilitator for market

(30)

participants to exchange information on energy customers when they switch from energy supplier. The NEDU is an association in which all the energy market parties are represented. Within the NEDU, regulations on the market processes are designed. The regulator of the energy sector, Autoriteit Consument & Markt (ACM), mandated NEDU to set up binding regulations on market processes, which are included in the Informatiecode5_{. The ACM can}

enforce this Informatiecode and can impose a fine on violation (Interview with Fokkema, 2018).

Besides the binding regulations, the NEDU also creates criteria for associated activities, like cyber security of the exchange of information. The security commission of NEDU deals with all the aspects of security. However, this commission only has a mandate for setting up security criteria for the exchange of the information up until the “front door” of the involved companies. The security demands are not mandatory and not included in the Informatiecode but work as an agreement in the market. As a rule, EDSN will implement these demands and therefore pushes them into the market. A few participants prepare the demands beforehand, and the demands are approved in the assembly meeting of NEDU. When a majority agrees, the proposal is accepted and the NEDU members will have to implement the demands (Interview with Fokkema, 2018).

The members of the NEDU consist of all the relevant market parties in the energy sector. The government does not participate, but the NEDU can set up requirements for the Informatiecode. The members of NEDU can bring security information from the government to the table when they retrieve this information in other collaborations with the government, like the ISAC Energy. Governmental security services are not directly involved in the partnership of the NEDU (Interview with Fokkema, 2018).

4.3 Describing the second PPP: Netbeheer Nederland

The branch organization for the Dutch gas and electricity distribution system operators is Netbeheer Nederland. Netbeheer Nederland promotes cooperation between the operators and represents the interests of the operators at the government, politicians, regulators and other stakeholders. Besides representing the interest of the operators, the branch organization organizes meetings between the operators to discuss modifications, trends and other relevant themes for the operators. On behalf of the operators, Netbeheer Nederland proposes changes for the law or legally anchored codes (Netbeheer Nederland, 2018.).

(31)

Netbeheer Nederland organizes events to bring the operators together. In November, Netbeheer Nederland organized a so-called “vital networking day”. During this day, operators, safety regions and other partners came together to discuss themes like cyber security and crisis management. The goal of this event was networking, providing the opportunity for partners to talk to and learn from each other so that they can react effectively if a crisis occurs (Netbeheer Nederland, 2018).

Netbeheer Nederland is subdivided into two main divisions: Klant & Markt (Customer and Market) and Systeem & Infra (System and Infrastructure). The Systeem & Infra division is the most concerned with cyber security. There is a so-called theme group Cybersecurity & Crisis management within this division. This theme group consists of security or crisis management specialists from all distribution system operators, including Gasunie. This group works together on projects to ensure that the whole sector is at the same level regarding these topics. Their goal is to ensure an equal level of security within the chain and to maximize the efficiency because of the scarcity of resources. The equal level of security within the chain is important since the electricity nets of the distribution system operators are interlinked. If one distribution system operator was hacked, resulting in an imbalance on their electricity net; the other distribution system operator would face an imbalance in their electricity net, resulting in a power outage throughout the whole country (Interview with Ten Pierik, 2018).

The theme group addresses topics like the new Dutch law, derived from the NIS-directive, Wet Beveiliging Netwerk- en Informatiesystemen (Wbni). The requirements of this law describe at an abstract level that you should know your risks and take appropriate organizational and technical measures. To make this law more concrete, the theme group discusses what actions need to be taken to meet up with these requirements. Due to this sector-wide discussion, the individual companies can show their legal authority that the whole sector uses these standards and that they all agree that the taken actions are sufficient. To ensure that the sector meets up with the requirements of the law, the theme group invites relevant governmental authorities to join the discussions (Interview with Ten Pierik, 2018).

Netbeheer Nederland does not have the authority or the role to ensure that the members implement the same standards; this is the responsibility of the companies themselves. The theme group reports to the so-called domain council in which all the asset management directors

(32)

of the members are represented. This council reports to the all CEO’s of the involved companies (Interview with Ten Pierik, 2018).

4.4 Describing the third PPP: ISAC Energy

To deal with the threats and vulnerabilities within the cyber domain, the Dutch government initiated to set up Information Sharing and Analysis Centres (ISACs). Public and private actors participate in an ISAC and exchange relevant information and experiences with each other. The participants of the ISAC consult on a regular basis with each other. There are fourteen active ISACs6, each focusing on a certain sector: Ports, Airports, Financial Institutions, Water Management, Multinationals, Telecom, Nuclear, Healthcare, Energy, Drinking Water, Managed Service Provider, Insurance, the National Government, and Pensions (NCSC, n.d. and NCSC, n.d.).

The Netherlands Organisation for Applied Scientific Research (TNO) established the ISACs on request of the Ministry of Economical Affairs. The institute Centre for Protection of the National Infrastructure (CPNI.nl), part of TNO, designed the Dutch ISACs according the US example. Their goal was getting the sector together to exchange and discuss information regarding cyber threats, vulnerabilities, and solutions. The problem with placing the ISACs at TNO was that TNO is also a commercial actor. Several participants of the ISACs had commercial contracts with TNO to deliver services. In the meantime, the DigiNotar hack occurred. Due to this hack, several websites, including governmental, were not safe to use. It was possible to fake an official website and when someone tried to log in on this fake website, his or her data was sent to the hacker. Following the DigiNotar incident, the GovCert (Government Computer Emergency Response Team), which used to be the governmental actor to handle cyber security incidents, was included in a new governmental organization named the NCSC (Leyden, 2011; Interview with Marchal, 2018 and CPNI.nl, 2012).

With the establishment of the NCSC, the ISACs were moved away from TNO and were linked to the NCSC. Linking the ISACs to the NCSC had another reason. The NCSC is a central center in which all the relevant information should come together. It should be noted that the ISACs are not owned by the NCSC; the NCSC participates in the ISACs and provides a secretarial function (Interview with Marchal, 2018 and CPNI.nl, 2012).

(33)

Within the ISAC Energy, the distribution system operators, transmission system operators, and some electricity producers that formerly were distribution system operators (before the liberalization of the energy market) represent the private sector. The NCSC and, depending on the subjects, other governmental agencies with knowledge or information regarding cyber security, like the High Tech Crime Unit7_{or the General Intelligence and Security Service}

(AIVD), represent the public sector. The authority for the electricity market, ACM, and for security, the Agentschap Telecom, do not participate in the ISAC Energy since the ISAC is intended to be a place in which the participants can speak freely. Even more, ISACs are not “owned” by the NCSC. This means that the government cannot impose any rules on an ISAC. The ISACs do use rules or protocols of the government, like the Traffic Light Protocol (TLP) to classify the confidentiality of information, but nobody can legally enforce these rules. An ISAC has so called membership guidelines, which every member of the ISAC must sign. There is a publicly available example of these guidelines. According to Mauriche Kroos, these example guidelines largely correspond with the membership guidelines of the ISAC Energy. These example guidelines are further analyzed in the following chapter (Conversation with Kroos, 2018; Interview with Marchal, 2018 and interview with Brugmans, 2018).

All the chairpersons and vice-chairpersons of the ISACs meet up several times a year to discuss cross-sectoral issues, trends, and other relevant themes. These meetings promote cross-sectoral information exchange and increases the network capabilities of the ISACs. Besides these general meetings, the ISAC Energy meets two times a year with the ISAC Water. The reason for this bilateral meeting is that the field operations of the water and energy sector are interlinked. For example, the sectors use the same digging slots for their cables and pipelines (Conversation with Kroos, 2018).

When the NIS-directive came into effect, via the Wbni, the position of the NCSC changed. The Wbni legally obliges certain organizations, identified as Operators of Essential Services (OES), to report (near-) incidents to the NCSC. This duty to report is designed as a call of assistance. However, several participants of the ISACs are classified as OES and must, therefore, report their (near-) incidents to one of their partners of the ISAC (Interview with Marchal, 2018).

7_{The High Tech Crime Unit is a Dutch Police unit that is occupied with the most advanced forms of cybercrime}

The ISAC Energy

The ISAC Energy

Preface

Table of contents

List of abbreviations

Abstract

1. Introduction

1.1 Scope of the research

1.2 Reading Guide

2. Theoretical framework

2.1 Defining Cyber Security

2.2 Defining Public Private Partnerships

2.3 Identifying the gap in knowledge

2.4 Choice of theory

3. Describing the research method

3.1 Type of research

3.2 Selecting the case

3.3 Operationalization

3.4 Methods of data collection

3.5 Methods of data analysis

3.6 Validity and reliability issues

4. Cyber security partnerships in the Dutch electricity sector

4.1 Liberalization of the energy market

4.2 Describing the first PPP: NEDU/EDSN

4.3 Describing the second PPP: Netbeheer Nederland

4.4 Describing the third PPP: ISAC Energy