Public and Private Cyber Defense Responsibilities

(1)

MASTER THESIS

Public and Private Cyber

Defense Responsibilities

Diverging interest in the energy and

telecommunications industries

Luuk Schrandt

S1019562

Leiden University

Master Crisis and Security Management

Supervisors:

Dr. J. Reijling

Dr. J. Matthys

March 2017

(2)

(3)

Details of the author

Luuk Schrandt S1019562

schrandt.luuk@gmail.com

Title of the research

Public and Private Cyber Defense Responsibilities

Diverging interest in the energy and telecommunications industries Key words

Cyber security, national security, responsibility, cyber defense, public private partnership Subject

(4)

(5)

i

Acknowledgements

Before I continue I want to express my gratitude to those who supported me in writing this thesis. I would like to thank my mentor Dr. Jaap Reijling, and the second reader Dr. Joery Matthys for their guidance in completing this thesis. Their fresh perspectives, suggestions, and comments have greatly contributed to the quality of this thesis.

I also want to thank those who were able to make time in their busy schedules. Without their support this research would not have been possible. Besides the added value to this thesis I greatly enjoyed the conversations we were able to have and want to thank all of you for sharing your experiences with me.

Finally I want to thank my family, friends, and colleagues who had to endure some mental and physical absence from my side. Your support throughout the process of writing this thesis was very much appreciated.

Luuk Schrandt March 2017

(6)

ii

Abstract

National security has always been a responsibility of governments. However, in a digitalized world adversaries no longer need to physically pass borders and enter the states’ territory in order to pose a threat. A government was always able to protect the nation beyond the stretch of (private) self-protection. Today, an adversary can simply bypass any national defense forces and directly attack the infrastructures that underpin our national economic strength.” (PCCIP, 1997, p.22). These changing circumstances cause a bifurcation when it comes to the provision of security for the state itself as most critical infrastructures are owned and operated by private entities and therefore no longer directly managed by the government.

This thesis focuses on these changing circumstances and assesses the changing responsibilities regarding cyber defense in the Netherlands. The research has been limited to the energy industry and the telecommunication industry in order to ensure feasibility. In addition, it is expected that these industries prove to be most interesting due to their role within society (Luiijf, 2003). The following research question has been drafted:

The study has shown that cyber defense responsibilities are as dispersed as the internet itself. Every party, whether public or private, is responsible for its own IT infrastructure. There is no single authority in control, no single authority that has a complete picture regarding the risks we face, and no single authority that sets the risk appetite levels for the Netherlands.

Discrepancies between both industries can be found in the details and nuanced statements of professionals in the field. Both industries have a pivotal role in society, but the telecommunication industry clearly takes the responsibility and questions whether there even is a gap between their risk appetite levels and those that correspond with national security. The energy industry on the other hand acknowledges a gap and argues for a governmental role here. The explanation regarding discrepancies, and therefore also the answer to the main research question is twofold. The responsibilities with regard to national cyber defense in the telecom

How are national cyber defense responsibilities in the telecom and energy sector of the Netherlands distributed between the government and involved private parties as part of corresponding public-private networks and how can possible limitations regarding the effectiveness of those networks be explained?

(7)

iii and energy sector and corresponding discrepancies are heavily connected to (1) the nature of the industry and (2) the nature of the corresponding cyber threat(s).

Practical recommendations derived from correlations between CI characteristics, cyber security characteristics and PPP-arrangements as observed in the Netherlands are twofold. First, knowledge regarding thresholds in risk appetite levels regarding cyber security should be created through an open discussion. Preliminary discussions can be organized on an industry level (e.g. ISAC), but eventually these need to be aggregated on a national level. This should allow public and private parties to have a mature discussion regarding responsibilities as gaps can easily be identified. These discussions currently have a very evasive nature.

Secondly, knowledge regarding cascading effects need to be developed. The complexities of grasping second order dependencies originate from the lack of investing resources. Public and private parties should combine their efforts towards unraveling the supply chains, starting with the most important industries.

In doing so, the Dutch government would substantially improve in allocating resources where they are needed most. It will also deliver knowledge to private parties, possibly creating additional incentives to incorporate the knowledge regarding their dependencies, making the digital infrastructure of the Netherlands a more resilient one.

(8)

iv

List of Contents

Acknowledgements ... i

Abstract ... ii

List of Contents ... iv

List of Figures and Tables... vi

List of Abbreviations ... vii

1. Introduction ... 1

1.1 Research Problem and Objective ... 2

1.2 Scope and Methodology ... 4

1.3 Research Outline ... 4

2. Conceptual Framework ... 6

2.1 Public-Private Partnership Networks ... 6

2.1.1 Public-Private Partnership Network: Governance Forms ... 8

2.1.2 Contingency: Trust... 10

2.1.3 Contingency: Number of Participants ... 11

2.1.4 Contingency: Goal Consensus ... 14

2.1.5 Contingency: Need for network-level competencies ... 18

2.2 Analytical framework ... 18

3. Research Methodology ... 21

3.1 Research Design ... 21

3.1.1 Case Introduction ... 21

3.1.2 Qualitative Research Design ... 22

3.1.3 Operationalization of Concepts... 23

3.2 Data Collection ... 26

3.2.1 Desk Research ... 26

3.2.2 Document Analysis ... 26

3.2.3 Semi Structured Interviews ... 26

3.3 Data Analysis ... 28

3.4 Reliability & Validity ... 28

4. Analysis... 29

4.1 Expounding on the Dutch Approach Towards Cyber Defense ... 29

4.1.1 Governmental Institutions ... 29

(9)

v

4.2 A Detailed Profile: The Energy Industry ... 36

4.2.1 Network Governance Forms ... 37

4.2.5 Contingency: Need for Network-level Competencies ... 47

4.3 A Detailed Profile: The Telecommunications Industry ... 48

4.3.1 Network Governance Forms ... 48

4.3.5 Contingency: Need for Network-level Competencies ... 57

4.4 Concluding: Similarities, Discrepancies, Complexities and Challenges ... 58

5. Discussion ... 63

References ... 67

Appendix I: CI in the Netherlands ... 77

Appendix II: Outline Interview ... 78

(10)

vi

List of Figures and Tables

Figure 1: First and second order dependencies between CI Figure 2: Conceptual representation of complexities in CI

Figure 3: The relationship between the concepts as used in cyber security Figure 4: An abstract representation of Cyber security

Figure 5: Forms of network governance

Figure 6: Key Predictors of Effectiveness of Network Governance Forms Figure 7: Conceptual Representation of Research Design

Figure 8: Case selection

Figure 9: The direct vitality versus indirect vitality Figure 10: Operationalization scheme

Figure 11: Respondents and their affiliation Figure 12: The 14 ISAC's in the Netherlands Figure 13: Labeling CI in the Netherlands Figure 14: Dutch Energy industry labeled as CI

Figure 15: Dutch Telecommunications and ICT industry labeled as CI Figure 16: Schematic overview of most important findings

(11)

vii

List of Abbreviations

ACM Autoriteit Consumenten Markt (industry supervisor)

AIVD Algemene Inlichtingen- en Veiligheidsdienst (civil intelligence service) APT Advanced Persistent Threat

CERT Cyber Emergency Response Team CI Critical Infrastructure

CIP Critical Infrastructure Protection

CSR Cyber Security Raad (Cyber Security Council)

CTIVD Commissie van Toezicht op de Inlichtingen- en Veiligheidsdiensten DDoS Distributed Denial of Service

DHS Department of Homeland Security

GCHQ United Kingdom Government Communications Headquarters ICS Industrial Control System

ICT Information Communication Technology IOC Indicator of Compromise

IRB Incident Response Board

ISAC Information Sharing and Analysis Center JSCU Joint Sigint Cyber Unit

IT Information Technology

MIVD Militaire Inlichtingen- en Veiligheidsdienst (military intelligence service) NATO North Atlantic Treaty Organization

NCSC National Cyber Security Center NCSS National Cyber Security Strategy

NCTV National Coordinator Terrorism and Security NDN National Detection Net

NRN National Response Net

PII Personal Identifiable Information PPP Public Private Partnership

SCADA Supervisory Control and Data Acquisition SME Small and Medium Sized Enterprises THTC Team High Tech Crime

WCIII Wet Computercriminaliteit III

(12)

1

1. Introduction

The failure of one or multiple critical infrastructures (CI) can have a significant impact on society as we know it (Kelly, 2015). An attack on the power grid could potentially impair other CI such as transportation, emergency services, or banking. Luiff (2003) described the first attempts of states reconsidering their vulnerable position when looking at CI and possible cascading effects due to interdependencies. During the cold war these considerations received a lot of attention because the threat was real and identifiable. With the diminishing threat of the cold war a decrease in the attention of risks regarding CI was apparent. Nowadays the cold war threat on CI seems to be replaced by a new emerging threat – a cyber security threat.

In 2009 the President of the United States of America addressed the paradox of the Information Age we live in nowadays. Obama argued, “The technologies that empower us to create and build also empower those who would disrupt and destroy” (2009). In the years since, a myriad of cyber incidents have struck both public and private entities. In 2016, a combination of malware in the Bangladesh Bank’s system and stolen credentials resulted in fraudulent money transfers, potentially endangering trust in the financial systems while the Stuxnet worm back in the autumn of 2010 actually managed to cause physical destruction to an enrichment plant in Natanz (Zetter, 2016; see also Zetter, 2014). The potential devastating impact of only several lines of code gains more traction within the international community, driving the agendas of both politicians and high level executives.

In his speech, Obama deliberately frames the risks in the context of CI because the industries in it are considered to be vital to our society. As the stuxnet case already showed, these industries are not immune to cyber-attacks. Only recently a power station in Ukraine was hit by malware called BlackEnergy3, affecting almost 100.000 residents for several hours. Variants of the BlackEnergy3 malware, which exploits an old well-known vulnerability in Microsoft Word, have also been found in systems within the rest of the EU and US (Zetter, 2016). These incidents show that large parts of our CI can be exploited by actors with malicious intent who have the sophistication and determination to do so.

An additional complicating factor involves the privatization, liberalization and deregulation during the eighties and nineties of the public sector which resulted in an increasing amount of CIs being operated by private entities (Dunn Cavelty & Sutter, 2009). Governments deregulated and privatized markets for efficiency goals and in a pursuit of a more liberal world (Schneider & Jäger, 2001). The situation, where the nowadays privately owned CI converges

(13)

2 with the emerging risks in the cyber domain, forces both public and private parties into a partnership which is multifaceted. Governments are adopting strategies to address these complexities through public private partnerships. This study will focus on the complications and challenges that arise from this ‘forced relationship’ within the cyber defense domain as CI is unequivocally and intrinsically linked to national security (Carr, 2016, p. 45).

1.1 Research Problem and Objective

Traditionally, companies and organizations would protect their assets up to a certain extend to enable them to withstand attacks stemming from low to medium sophisticated adversaries. These companies and organizations will not be able to protect themselves against high sophisticated attackers which are most often state-sponsored. A company operating CI would be able to fend of local criminals (with access control, fences, guards, etc.), but when CI operators face military grade equipment they count on their state to protect them. The state or government acknowledged this responsibility and protected its own physical territory.

However, in a digitalized world these high sophisticated attackers no longer need to physically pass borders and enter the states’ territory in order to pose a threat. “The government has always focused on protecting the nation from threats beyond the capabilities of private self-protection. Today, an adversary can bypass our national defense forces to attack directly the infrastructures that underpin our national economic strength.” (PCCIP, 1997, p.22). These changing circumstances cause a bifurcation when it comes to the provision of security for the state itself. Most CI’s are owned and operated by private entities and therefore no longer directly managed by the government. One could argue that this indicates that our national security increasingly relies upon private party investments in cyber security while generating security for citizens is a core task of the state; therefore it is an extremely delicate matter for the government to pass on its responsibility in this area to the private sector (Dunn Cavelty & Sutter, 2009, p. 181).

The main purpose of this study is to address the blurring border between the vectors involved in national cyber defense – public and private parties. Many states attempt to formalize this relationship in their national cyber security strategies and acknowledge the pivotal role of the public private partnership (PPP) networks. Neither one of these vectors has the capacities to address this problem individually so by definition there needs to be some sort of cooperation. However, the deviating interests between public and private parties lie at the

(14)

3 core of the complex puzzle (Carr, 2016, p.57). It’s worth citing the views of some nation states regarding this joint effort (Dutch, German, U.K., and U.S. respectively):

“Increasing the Netherlands’ digital resilience cannot be achieved by the government alone, as the ICT infrastructure itself and knowledge about this infrastructure is largely in the hands of national and international private parties. Therefore, cyber security is the sum of joint efforts of government bodies, the business community, organisations and citizens, both on a national and international level.” (NCSS, 2013, p.3)

“The protection of critical information infrastructures is the main priority of cyber security. They are a central component of nearly all critical infrastructures and become increasingly important. The public and the private sector must create an enhanced strategic and organizational basis for closer coordination based on intensified information sharing.” (Federal Ministry of the Interior, 2011, p.8)

“Though the scale of the challenge requires strong national leadership, Government cannot act alone. It must recognise the limits of its competence in cyberspace. Much of the infrastructure we need to protect is owned and operated by the private sector. The expertise and innovation required to keep pace with the threat will be business-driven.” (Cabinet Office, 2011, p.22)

“To succeed in its missions the Defense Department must operate in partnership with other Departments and Agencies, international allies and partners, state and local governments, and, most importantly, the private sector” (Department of Defence, 2015, p.3)

Within the academic world, as well as in practice, there is a knowledge gap on how to address this complex puzzle. The body of knowledge within the security discipline, especially cyber security, is still relatively limited compared to other disciplines and this study will contribute towards achieving a more mature status (Griever, 2007). Several studies have been conducted in the field of national cyber defense responsibilities, however most are relatively limited in scope. Studies are often limited to either assessing the effectiveness of a PPP network, discussing the regulatory perspective, considering the public perspective, or assessing the private interests. This study will focus on integrating all these perspectives within two industries, enabling the ability to compare alongside all relevant variables. In practice, cybersecurity has become a matter of global interest and importance (Von Solms & Niekerk,

(15)

4 2013, p. 97). Improving our understanding of cybersecurity and how it changes risks for society would be beneficial to both public and private entities.

1.2 Scope and Methodology

This research will investigate the blurring borders between public and private parties of cyber defense within the Netherlands. The research is limited to the Netherlands, because it represents a modern state with subsequent high cyber security risks. Besides, the Netherlands has a high degree of connectedness and both privatization and digitalization variables are present. The research will be limited to the telecommunication and energy sector to ensure feasibility. In the Netherlands both the telecommunication and energy sector are considered to be CI. These cases in particular are interesting in a comparative perspective due to the expected motivations of the attackers. Both industries face high sophisticated adversaries, however there might be some discrepancies regarding their motivations. This scope and the research objectives translate into the following research question:

This explorative research is designed as a comparative case study. The methodologies used in this study are literature study, document analysis, and semi-structured interviews. The Netherlands has published several documents regarding its cyber security strategy, however there is significantly less information available regarding how things work in practice. Insights into these practices have to be achieved by conducting interviews.

1.3 Research Outline

The outline of this report will look as follows. Chapter 2 will describe the conceptual background. A consistent theoretical framework will be used to answer the research question. The terms used in the research will be conceptualized using relevant academic literature. Chapter 3 provides an overview of the research design. The research design is aligned with the theoretical framework and will expound on the decisions made regarding the operationalization. Subsequently, chapter 4 dives into the analysis of the gathered data. This chapter will also provide an answer to the research question. Finally, chapter 5 discusses the

(16)

5 limitations of the research. It will also provide recommendations and pointers for government policies and possible future research.

(17)

6

2. Conceptual Framework

This chapter will elucidate on the concepts used to answer the research question. Due to the absence of a theory or model that perfectly encapsulates the integration of (1) national security responsibilities that specifically address (2) cyber security issues related to (3) critical infrastructure, existing literature will be combined to suit this particular research. The combination of the academic sources will substantiate an analytical framework that is the basis of the empirical design of the study in chapter 3 and the analysis in chapter 4. The researcher has chosen PPP networks as central idea due to the pivotal role these have in current cyber security strategies (Federal Ministry of the Interior, 2011; Cabinet Office, 2011; NCSS, 2013; Department of Defence, 2015).

2.1 Public-Private Partnership Networks

PPP networks gained interest and popularity due to liberalization, deregulation and privatization. Neoliberalists encouraged the public agencies to entrust the private industry with more of their tasks, or at a minimum, provide it together in the form of a cooperation. This was believed to be the only way to make the public administration more effective and efficient (Savas, 1982). PPP within this context is defined as follows:

The goal of a public private partnership is to exploit synergies in the joint innovative use of resources and in the application of management knowledge, with optimal attainment of the goals of all parties involved, where these goals could not be attained to the same extent without the other parties (Linder & Vaillancourt Rosenau, 2000, p.5) Essential in this definition is the process of acknowledging common goals and some form of mutual interdependency. The interdependent relation is essential towards establishing a PPP network, but not considered to be sufficient. Kouwenhoven nuances this statement by arguing that the objectives that originate from both public and private parties should not be incompatible – there should be a convergence (1993, p.125). The goals and objectives don’t need to be identical, but there needs to be a common ground, goal, or interest. Waddock states that subsequent ‘linking mechanisms’ need to be present to start a PPP network (1986, p. 279). These linking mechanisms can be either (1) the presence of an existing network or (2) the presence of a broker. An already existing informal network can transform into a PPP network, but when this is not present an outsider (independent) should act as broker to establish the PPP network. Within the confines of national security and cyber security issues, Dunn Cavelty and

(18)

7 Suter argue that “The question is not whether public private collaboration is necessary here, but how it should be organized.” (2009, p.2).

Wettenhall (2003) distinguished two rough categories of PPP networks. The first category includes partnerships which are ‘flat’ and have no hierarchical order. The second category can be described as vertical and has a hierarchical order where often one entity is in control. According to Wettenhall, a ‘real partnership’ relies on equality and imposes limited to no hierarchical order. Provan & Kenis (2009) continued the effort towards distinguishing between different ‘categories’ of PPP networks and identified three governance forms that vary between vertical and horizontal relationships: (1) lead organization, (2) network administrative organization, (3) shared governance network (also see figure 1).

In addition, Kenis & Provan (2008) identified four contingencies (see figure 1) which affect the successful adaptation and effectiveness of any of these PPP network governance forms, respectively: trust, number of participants, goal consensus, need for network level competencies (Drazin and Van de Ven, 1985). These governance forms and contingencies should not be assessed as separate silos, instead, they should be assessed as a whole. The greater the inconsistency between the governance form at hand and the contingencies, the higher the chances are that the network will be ineffective and unsuccessful.

Figure 1: Key Predictors of Effectiveness of Network Governance Forms (Adapted from: Provan & Kenis, 2007)

Certain forms of network governance fit specific situations and other forms of network governance won’t produce the same output. This does not imply that certain governance forms are bound to fail, but each form is designed with a specific goal in mind. Thus, not every governance form will produce the desired outcome, and not any one form is superior to another. It all depends on the circumstances in which the network operates; within this research: (1) national security that specifically addresses (2) cyber security issues related to (3) CI. Hence, the different governance forms of Kenis and Provan provide a framework to assess the effective

(19)

8 distribution of responsibilities, however it is important to embed complementary academic sources regarding these circumstances in the identified governance forms and corresponding contingencies as several scholars argue that there are some serious limitations to the solution of PPP networks within the domain of CI protection and national cyber security (Boeke, 2016; Broeders, 2014; Carr, 2016; Dunn Cavelty & Suter, 2009; Germano, 2014).

2.1.1 Public-Private Partnership Network: Governance Forms

The first governance form as identified by Provan and Kenis is labeled as the ‘shared governance network’ (2008, p. 234). A ‘shared governance network’ is governed by the members themselves. Only by uniting all network members can the network reach its full potential. This form of network governance is also dubbed as the simplest form of network governance. Different organizations work together, but do not have a clear entity that governs the network. Figure 2(a) provides a graphical representation of what a shared governance network could look like in practice. The strength of this form of network governance lies in the flexibility, as the network can easily adapt to the changing needs and the organizations have a high level of inclusiveness. A weakness would be the efficiency of a shared governance network due to the absence of a ‘central entity’. The shared governance network reaches its true potential in small and ´concentrated networks’ (Kenis & Provan, 2009, p. 446).

Figure 2: Forms of network governance (Adapted from: Kenis & Provan, 2009)

The second form of network governance is known as the ‘lead organization network’ (Provan & Kenis, 2008). Within this mode of governance one single organization leads the network, and this is often seen in vertical relationships. The lead organization network shows similarities to the distinction made by Wettenhall – horizontal versus vertical relationships. Wettenhall (2003) argued that true relationships have no hierarchical order, but Kenis and

(20)

9 Provan (2009) do see a use, in for example buyer-supplier relationships or regulatory arrangements. Network governance is extremely centralized and the lead organization often acts as a broker. A visual representation can be found in figure 2(b). The strengths of a lead organization network are its efficiency and legitimacy due to the presence of the lead organization and its capacity. It manages to avoid the messy process of consensus building among all stakeholders as there is clearly one entity in the lead. Weaknesses stem from this characteristic as well. The goal of such a network is often identical to the goal of the lead organization and the lead organization might have its own agenda. This can cause resistance among other minor parties in the network.

The final mode of network governance is labeled as ‘network administrative organization’ (Kenis and Provan, 2009). In this case a separate entity governs the network and its corresponding activities – it acts like a hub. The visual outlook can be found in figure 2(c). The network administrative organization acts like a lead organization, but is not another entity with its own interests. The network administrative organization has the sole purpose to govern the network and enable the network to be efficient and effective. Network administrative organizations can be both formal and informal, where the informal would likely be an individual who coordinates the network and the formal would typically be an organization that needs official recognition from internal and external stakeholders to be legitimate (Kenis and Provan, 2008). Strengths of this governance form can be found in its legitimacy, durability, and efficiency. Weaknesses lie mainly in the posture of the network partners, who tend to rely too heavily on the network administrative organization.

When determining a specific network governance form within the domain of national security, public agencies need to develop a strategy with the tradeoff between cooperation and regulation in mind. A shared governance network or network administrative organization positions the government closer to the private parties, but might hinder their ability to impose national security related legislation and vice versa (Broeders, 2014; Germano, 2014). When a government positions itself closer to the private parties it needs to rely and trust on the private operators to implement the appropriate security measures. Self-policing and self-regulating by members of national security (often CI) related PPP networks can establish the norms that are needed to create a baseline for security capabilities (Dunn Cavelty & Suter, 2009). It is argued that previously established networks are better at defining their own norms than networks which were established through a broker (Waddock, 1986; Aviram, 2006).

(21)

10 Favoring a lead organization network can allow a government to restore their ability to monitor and control a cyber security measures within CI more directly. However, regulating markets or acting intrusively within the markets will hinder the ability of private companies to compete. At the same time the government has to address certain issues from a national security perspective. The ideal policy and network governance form within this area must absorb both the unfavorable results of (1) privatization, (2) liberalization, and (3) globalization from a national security perspective, while at the same time maintain the favorable effects from an economic perspective (Dunn Calvelty & Suter, 2009).

2.1.2 Contingency: Trust

Mutual trust in itself is important for a relationship to be effective and successful (Kouwenhoven, 1993; Linder, 1999; Wettenhall, 2003; Provan & Kenis, 2008; Klijn, Edelenbos & Steijn, 2010; Germano, 2014; Whelan, 2015). Trust can be defined as “the willingness to accept vulnerability based on positive expectations about another’s intentions or behaviors” (McEvily, Perrone, and Zaheer 2003, p. 92). Additionally, for a network to be successful it is important to consider the distribution of trust among the members of the network. The distribution of trust should heavily influence the choice regarding a governance form. The density of trust should correspond with the network form at hand. As can be seen in figure 2, different levels of trust densities demand different governance forms.

As discussed earlier, these contingencies should be considered within the circumstances in which the PPP network operates. The contingency of trust is intrinsically connected to the secretive nature of both cyber and national security. As former director of NSA Hayden argued: “This stuff is hideously overclassified, and it gets into the way of a mature public discussion as to what it is we as a democracy want our nation to be doing up here in the cyber domain.” Germano (2014) emphasizes that private parties look for benefits in relationships, if there aren’t any they will refrain from establishing them. From a cyber security perspective it is often unclear for companies what the benefit from a relationship with the government is. This correlates with the conditions of access and secrecy. The former addresses the idea that private parties are afraid that a PPP might give the government access to or possibly even the ability to control their digital infrastructure. A PPP network in this context has a substantial negative notion. The latter, secrecy, is the fear of private parties that in reality the PPP network will be a one sided relationship where the government is there to collect information, but unable to return anything tangible (they fear a lead organization governance form, possibly including

(22)

11 mandatory actions). The government should be a partner they can truly rely on and should encompass more than the “mere quid pro quo argumentation” (Broeders, 2014, p. 34).

Thus, from a private party perspective, trust is all about being in control as “companies and corporations are afraid of negative press, civil litigation, and regulatory scrutiny” (Germano, 2014, p. 4). Companies want to be in control in order to manage press releases, timing, patches, and disclose the breach when they deem fit. Working together with the government might jeopardize their attempts to do just this. One of the main aspects of a PPP is the exchange of information regarding threats or breaches which is heavily subjected to trust (Boeke, 2016). The larger the circle becomes, the less its participants are prepared to share sensitive information (connected to the contingency of number of participants). The private sector is deterred by lack of confidentiality and possible subsequent reputational damages and the government is held back the risk of exposing their intelligence sources. Additionally, most public agencies who participate and host the PPPs are not positioned within the intelligence community and therefore excluded from direct access (Boeke, 2016).

2.1.3 Contingency: Number of Participants

The number of participants is the second contingency that should influence the selection of a specific network governance form. For every additional member in the network the amount of relationships increases exponentially, creating an increasingly complex situation. One can understand that a shared governance network would be highly inefficient with a large group of organizations. Centralization of certain network tasks, through either a lead organization or a network administrative organization, can potentially solve such a problem.

Within the context of national security and cyber it is often challenging to set the right participant scope for a PPP network. States have a natural tendency to consider CI labeled industries as their primary audience, and subsequently exclude non CI labeled industries. (Dunn Cavelty & Suter, 2009). Many complexities in defining the number of participants of a PPP network in the domain of national security originate from the increasing amount of interdependent relations between the individual CI’s (Lauge et al, 2014; Rinaldi et al, 2001; Pedersen et al, 2006). CI’s cannot be assessed as separate silos, each CI is tightly coupled with multiple other CI’s. Thus, creating an increasingly complex system of interconnected parts. The tight coupling between the CI’s allows for a failure in one CI to propagate to one or multiple other CI’s – also known as cascading effects (Menashri & Baram, 2015). When a

(23)

12 product or a service cannot be delivered to the next entity in the value chain it cedes to operate normally.

The potential devastating nature of cascading effects is what makes the complex system of interconnected CI so vulnerable, and subsequently selecting the right participants for a PPP network becomes increasingly challenging. Laugé et al. (2014) argue that many organizations have the ability to grasp direct dependencies of their own infrastructure – first order dependencies. However, these organizations have very limited understanding of the overall complex system of interdependencies (Papa & Shenoi, 2008; Lauge et al, 2014). Figure 3 illustrates the difference between first and second order dependencies. Organization B is dependent on the products or services from organization A. Subsequently, there is a mutual interdependent relationship between organizations B and C (figure 3-I). This results in a cascading effect from organization A to C when A is unable to deliver their products or services (figure 3-II).

Figure 3: First and second order dependencies between CI (Adapted from: Lauge et al., 2014, p. 17)

Organizations often have no insights into these interdependent relations as the visualization in figure 3 is only a simple representation of the empirical reality. The extent of interdependent relations has a more accurate conceptual representation in figure 4, although still a toned down version of reality.

(24)

13 Digitalization and the subsequent emergence of cyberspace does not only make it more complicated due to its intangible nature, but also because of its global footprint. Dependencies and risks are not merely confined within the borders of a nation – dependencies and risks transcend borders. Many argue that cyberspace can no longer be regarded as a discrete sector (Warfield, 2012; Alcaraz & Zeadally, 2014; Clemente, 2013). In reality, cyberspace is embedded so deeply within a myriad of other CIs that it can be regarded as a ‘nervous system’ running through all CI. This new layer that runs through all other CI amplifies the already existing interdependencies among CI, further increasing the challenge to set to right scope for a PPP network.

A study by Luiijf and Klaver (2003) shows the direct (first order) and indirect (second order) vitality of industries, many of which are commonly defined as CI (see figure 5) and can potentially guide states in the process of defining CI. Broeders (2012) argued that this can be a political process as well. Especially the telecommunication and energy industry have a very high direct and indirect impact on society.

Figure 5: The direct vitality versus indirect vitality. The higher up and/or more to the right, the more vital the product or service is to society. (Retrieved from: Luiijf & Klaver, 2003)

As both public and private parties have a hard time grasping these interdependencies – both horizontal (between various CI) and vertical (dependency of CI operators on multiplicity of small and medium sized enterprises) it is commonly a challenge to define the admission

(25)

14 criteria of a PPP network in the domain of national security (Dunn Cavelty & Suter, 2009). The increased amount of interdependent relations due to digitalization is urging states to redesign their CI protection PPP, including more smaller members into their networks. However, due to the relation between the number of participants and the level of trust it remains a challenge to define the right scope for a PPP. Further collaboration is needed to create the trust that is necessary in vertical and horizontal cooperation, but this classic paradox, where both attributes are absent but necessary to establish either one of them, forges a situation where only small networks with pre-established trust or cooperation have a potential to be successful (Dunn Cavelty & Suter, 2009).

2.1.4 Contingency: Goal Consensus

As third contingency, a high level of goal consensus contributes to the effectiveness of a network, but also in the case of lower goal convergence a network can potentially be successful. Again, the focal point is the governance structure of a network. Shared governance networks need relatively high goal consensus in order to be effective, while lead organization networks and network administrative organizations can also operate with moderate levels of goal consensus. An important notion is made by Park (1996) and Kouwenhoven (1993) who argued that goals do not need to be similar, they need to converge. Similar goals might also generate reluctance among network partners when they face competitive interests. Partners might be hesitant to share information or refuse to cooperate.

It is important to expound on the concepts of cyber [security] related to national security as goal are likely to be derived from cyber risk appetite levels within public and private parties. Von Solms and van Niekerk (2013) have examined the differences between the concepts used within this area (see figure 6) as the concepts of (1) cyber security, (2) information security, and (3) ICT security are often used interchangeably. All three concepts aim to preserve the confidentiality, integrity, and availability of their respective assets (ISO 27002, 2005). The confidentiality is maintained by preserving proper access management. The integrity revolves around the authenticity and trustworthiness of the assets – is it unmodified? Finally, the availability factor addresses the ability to access the assets at any given time.

Within this context, information security is the protection of both digital and physical information. The ICT security addresses the protection of the systems which are used to store and transmit information (data). The difference between information and ICT security is that ICT security merely addresses the protection of ICT assets, whereas information security also

(26)

15 encompasses the information itself. Von Solms and van Niekerk (2013) argue that cyber security is more than just the protection of information.

Figure 6: The relationship between the concepts as used in cyber security (Adapted from: von Solms & van Niekerk, 2013, p. 101)

Cyber security is about protecting the assets that use those resources and protect them from risks that find their inception in the ICT vulnerabilities. Thus, cyber security is about protecting humans and their interests. As shown in figure 7, the assets are only vulnerable due to the abuse of various threat actors. CI operators that operate within cyberspace face an increasing amount of threats. Malicious actors are continuously developing their capabilities and increasing their sophistication to exploit vulnerabilities. The asymmetrical relations within cyberspace render no one safe, because the malicious actor only needs to detect one single vulnerability to exploit a system, while the defender needs to plug all the holes at all times, preferably within an acceptable timespan (Clarke, 2012; Karabacak et al., 2016).

Figure 7: An abstract representation of Cyber security (Adapted from: von Solms & van Niekerk, 2013, p. 101)

In the case of traditional warfare it is rather easy to see the build-up towards a conflict and one can subsequently relatively easy attribute the attack. However, within the digital domain this becomes a lot more difficult. The development of cyber weapons is hard to spot, attribution is very difficult, and the defender has almost no time to respond (Phillips, 2012). As

(27)

16 cyber weapons are often in the ‘one-use-only’ category the defender doesn’t know what he is defending against. There is a wide variety of malicious actors, but we can distinguish two broad categories - low sophisticated adversaries (e.g. script kiddies) and high sophisticated adversaries (e.g. states or state affiliated proxies | APT: advanced persistent threat) (Wieren et al, 2016).

One might initially assume that developed states have the upper-hand in cyber related scenarios, but the opposite can easily be true. Developed states have moved up along the digitalization axis al lot faster than those states which did not poses the means to manage such technological advancement (Phillips, 2012). Thus, developed states are much more interconnected within the CI domain and therefore vulnerable for cyber related attacks.

According to the U.S. ICS-CERT the main risks in the cyber domain for CI originate from nation states: “For the next 5 to 10 years, only nation states appear to have the discipline, commitment, and resources to fully develop capabilities to attack critical infrastructures.” (ICS-CERT, 2016). An attack on a system is not an end in itself. Often seen goals vary between sabotage, espionage, financial gain, and disruption (Geers, 2011; ICS-CERT, 2016). The three main goals for nation states who conduct operations in cyberspace are: (1) to sabotage and disrupt when a conflict arises, (2) to realize technological development and (3) to gather intelligence to influence global politics by means of espionage (ICS-CERT, 2016). The Dutch government reports on this trend as well in their annual cyber security report (Cyber Security Beeld Nederland 2015). States and state affiliated parties (proxies) increasingly act in line with the interests from the state and try to influence geopolitics through cyberspace (CSBN, 2015).

Addressing these digital risks from a national security perspective seems natural, but the private ownership of CI complicates things. Dunn Cavelty and Suter (2009) argue that this creates a situation where there is misalignment between public and private goals as the level of security needed for national security is not equal to what is provided under the forces of an open market. The protection of CI becomes heavily reliant on private party investments – it becomes reliant on market mechanisms. As the privatization led to changes in risk ownership – move from public to private ownership – the digitalization mainly leads towards an ambiguous situation regarding risk ownership. Dunn Cavelty and Suter (2009) emphasize the responsibilities of the government as they remain legally accountable for providing national security. Carr argues that the state could potentially be seen as “abdicating not only the authority, but also responsibility for national security” (2016, p.44). Within the outlines of

(28)

17 traditional security it was possible to mitigate the risks to an acceptable level by protecting borders, however border protection becomes partially abundant with the possibilities in the cyber domain and states could subsequently be rendered ill-equipped in mitigating these new risks stemming from the cyber domain on their own. Related to this mismatch is the delicate nature of a PPP in the domain of national security (Dunn Cavely & Suter, 2009). Creating a level of national security is one of the core tasks of the nation state and passing this responsibility on to the private sector through the establishment of a PPP network in the cyber domain is at least somewhat ambiguous. Even more so due to the development of offensive capabilities within the government itself (NATO CCDCOE, 2016). Politicians have been very circumspect in claiming authority or imposing legislation while the relevant private parties seem to be reluctant in accepting any responsibility or liability for national security.

Assaf (2008) identified two perspectives that have a potential to explain public and private motivations [goals] in the national [cyber] security domain: (1) a business continuity perspective, and (2) a national security perspective. In essence, both have similar characteristics such as availability, functioning and continuity of CI systems, however, the underlying drivers seem to be rather polarized. The interest towards investing in cyber security for a corporation (with the absence of regulation by a government) from a business continuity perspective is twofold. A corporation would apply higher levels of cyber security if (a) there is a distinct return on [security] investment, and (b) the relation of mitigation and risk is clear. On the other hand, governments have the inherent tendency to be “exceptionally risk averse to potential threats” (Assaf, 2008, p.11-12). Since 9/11 governments are willing to increase security far beyond the threshold of a business continuity perspective, favoring national security over economic efficiency. Broeders argues that this can be labeled as a fundamental mismatch where private parties are only willing to take responsibilities as long as it remains within “the confines of their business interests” (2014, p.33). This mismatch puts permanent pressure on public agencies to regulate in order to bridge the gap towards national security. Thus, favoring characteristics of a lead organization governance form.

Germano (2014) finally adds the aforementioned transboundary character of cyber. Threats transcend boundaries and so do their targets. Corporations often operate across multiple borders and are subjected to a myriad of policies, regulations, roles, and duties which further complicates the role of a PPP and the underlying goals. The scope of a PPP is often misaligned with the scope of a corporation and causes goals to diverge (Dunn Cavelty & Suter, 2009). A

(29)

18 corporation often only conducts a share of its operations in the respective territory of the PPP and subsequently allocates corresponding resources. Network-level competencies (fourth contingency) are necessary in order to manage to potential abundance and complex nature of PPP networks regarding risks to national security from the cyber domain.

2.1.5 Contingency: Need for network-level competencies

The need for network level competencies is the final contingency that influences the effectiveness of the different network forms. Each governance form places a different burden on the network member. Network members need competencies that correspond to that burden (Provan & Kenis, 2007). When there is a high need for network level competencies, the lead organization network or the network administrative organization are better options since they are often able to handle the burden with their competencies. However, shared governance network members can be less competent in executing certain tasks. Creating legitimacy, dealing with new regulation, handling members, etc. are all tasks that require competencies which influence the efficiency and effectiveness of the network.

Within the circumstances of national security and the corresponding ‘need to share’, Whelan (2012) argues that a centralized entity increases the chances of a PPP network to be successful and create the necessary situational awareness among the network participants. A central entity can process and collate relevant information before relevant [public and private] parties receive the information. Additionally, the centralized actor is “more likely to have the knowledge, capabilities and resources needed to manage different PPP networks simultaneously” (Whelan, 2012, p. 46). The need for network-level competencies relates to the amount of participants in the network, as modifying the scope will significantly influence the required resources. National security related networks are therefore more likely to be brokered to create the flexibility that is needed to manage these PPP networks (Whelan, 2012). Hence, a shared governance network is least expected to be successful in the domain of national security and cyber threats.

2.2 Analytical framework

This thesis focuses on exploring the complexities and allocation of cyber defense responsibilities between public agencies and involved private parties. Questions regarding cyber defense in the CI domain emerged due to the subsequent privatization and digitalization of society. PPP’s seem to provide some solutions, but it remains unclear how these organizational structures are perceived in practice.

(30)

19 Based on the conceptual framework the following research question was drafted:

In order to answer this question two sub-questions questions were drafted. The first question examines the considerations that public and private parties face in the domain of national security and cyber threats towards adapting different governance forms in the Netherlands. The three governance forms as provided by Kenis and Provan (2008; 2009) will be used to identify and assess attempts to manage national cyber security responsibilities. This theoretical framework was chosen due to the pivotal role PPP networks have in current cyber security strategies. Additional academic sources as depicted in chapter 2.1.1 will be used to provide the necessary context regarding national security and cyber threats (Broeders, 2014; Carr, 2016; Dunn Cavelty & Suter, 2009; Germano, 2014)

1. Which different PPP network governance forms related to national cyber defense have been established within the energy and telecommunication industries of the Netherlands?

The second question corresponds with chapters 2.1.2 – 2.1.5 and expounds on the contingencies that influence the outcome of the different PPP networks. It uses the contingencies (trust, number of participants, goal consensus, need for network level competencies) as depicted by Kenis and Provan (2008; 2009) to analyze the current cyber defense PPP’s in the Netherlands. The complementary research of other scholars regarding these contingencies in the domain of national security, CI protection, and cyber security as discussed in chapters 2.1.2 – 2.1.5 will also be included (Assaf, 2008; Boeke, 2016; Broeders, 2014; Dunn Cavelty & Suter, 2009; Germano, 2014, Whelan, 2012). The goal of this question is to assess the current situation in both industries and provide a theoretically underpinned explanation regarding national security PPP networks in the cyber domain of the Netherlands.

2. How do the identified contingencies influence the current national cyber defense posture within the energy and telecommunication industries of the Netherlands?

(31)

20 The analysis that provides the answers to questions 1 and 2 will substantiate an overview of responsibilities, discrepancies and corresponding incentives of public and private parties in the domain of national cyber defense and contributes towards answering the main research question.

Figure 8 shows an overview of how these questions fit within this research. The next chapter will elucidate on the methodological framework used to research these questions. It will provide additional information on case selection, data collection, data analysis, and possible limitations.

(32)

21

3. Research Methodology

This chapter will discuss the methodology used to answer the research question. First the research design will be addressed which introduces the cases and expounds on the research questions. Subsequently the applied data collection and analysis methods will be addressed. Thirdly, the validity and reliability constrains will be discussed.

3.1 Research Design

The following research question has been derived from the theoretical framework:

National security, defense responsibilities, cyber and warfare are all subjects which have received their share of attention in the science domain. However, a combination of these concepts still remains sparse. Especially the question of responsibilities is only occasionally addressed, often as a side note. This research will have a focus on exactly that question. Public private partnerships are repeatedly created in order to reach consensus among parties, but a thorough understanding of the practical implications is still missing. An effort will be made to fully grasp the complexities that arise within this domain. As this field is still developing it is often useful to adapt a comparative method. It can help the researcher to advance from some initial explorative case studies towards more advanced causalities and theories.

3.1.1 Case Introduction

The energy industry and the telecommunications industry within the Netherlands will be studied and compared in order to answer the research question.

Figure 8: Case selection (Author)

(33)

22 These cases – unit of analysis – are selected to ensure that the cases are relevant and not too similar (Neuman, 2014). By focusing on two instead of one sector it is ensured that the issue is not explored through one lens. This will allow a better grasp on the phenomena (Baxter & Jack, 2008). The energy and telecommunication industries were selected due to the interesting combinations they bring forth. The energy industries provides interesting perspectives due to the nature of their business (which is vital to society as we know it) and their related cyber threats. The detailed industry profiles differentiate which could potentially lead to diverging cyber defense approaches. A study by Luiijf and Klaver (2003) symbolizes the importance of both industries towards society (see figure 9). Both industries have a very high direct impact and a very high indirect impact on society.

Figure 9: The direct vitality versus indirect vitality. The higher up and/or more to the right, the more vital the product or service is to society. (Retrieved from: Luiijf & Klaver, 2003)

3.1.2 Qualitative Research Design

This thesis will be conducted by means of qualitative research methods. Qualitative research is more about words than about hard numbers and relies more on interpretive principles. A qualitative research method has been chosen due to the nature of the topic. As the goal is to develop knowledge and provide insight into national cyber defense in the Netherlands it is a better fit to allow adaptations in the midst of research. It also allows the researcher to work with data that is presented in a diffuse form (Neuman, 2014).

(34)

23 3.1.3 Operationalization of Concepts

As the researcher is conducting qualitative research, operationalization provides rudimentary working ideas while the researcher is gathering data. First, the relevant concepts had to be identified. This has been done through the analysis of both academic literature and reports. This is essential towards fully grasping how the different concepts tie into one another. The operationalization scheme below transforms these abstract constructs and concepts into clear conceptual definitions and enable the researcher to empirically observe it.

# Concept Definition Indicator(s) and corresponding data sources

1 Presence of PPP network

governance forms

- The goal of a public private partnership network is to exploit synergies in the joint innovative use of resources and in the application of management knowledge, with optimal attainment of the goals of all parties involved, where these goals could not be attained to the same extent without the other parties (Linder & Vaillancourt Rosenau, 2000, p.5)

- Different governance forms as described by Kenis & Provan (2009):

- Shared Governance Network; - Lead organization Network; - Network Administrative

Organization.

Indicators:

- Inception of PPP networks in the domain of national cyber

security/defense. Is the PPP network brokered or was there a previously established network;

- Existence of horizontal and vertical PPP networks in the domain of national cyber security/defense;

- Documented purpose and goal of PPP networks in the domain of national cyber security/defense;

- Perceived relation between involved stakeholders regarding vertical or horizontal characteristics;

- Parties acknowledge an added value and attainment of goals due to the presence of the PPP network that otherwise would not have been attained.

Data Sources:

- Document Analysis (mainly government policies); - Interviews with public private and independent sources,

(35)

24 2 Existence of trust

between network participants

- Trust is defined as “the willingness to accept vulnerability based on positive expectations about

another’s intentions or behaviors” (McEvily, Perrone, and Zaheer 2003, p. 92);

- The distribution and corresponding density of trust within the network affects the effectives and outcomes of a PPP network.

Indicators:

- Willingness to share information between participants in the PPP network;

- Documented guidelines regarding expected behavior of participants in PPP network;

- Direct access to relevant information of other participants in the PPP network;

- Implemented legal framework to stimulate trust between participants in PPP network;

- Acknowledgement of trust between participants in PPP network by participants themselves.

Data Sources:

- Document Analysis (e.g. code of conduct, or guidelines for PPP participants);

- Interviews with public private and independent sources. 3 Participation in

PPP network

- Parties are labeled as participants when they are admitted in at least 1 PPP network;

- For every additional participant, the complexity is expected to increase exponentially;

- Participation within the context of national cyber security/ defense is commonly linked to CI labels.

Indicators:

- Clear guidelines regarding the admission criteria (including the process of labeling CI);

- Collective view towards presence of participants and corresponding added value;

- Extent to which membership is mandatory or voluntary for participants of PPP network.

Data Sources:

- Document Analysis (e.g. admission criteria, process of defining CI labeled industries);

(36)

25 4 Existence of goal

consensus

between network participants

- High level definition of PPP network values;

- Unambiguous capturing of goals and objectives;

- Unambiguous capturing of strategy; - Existence of short term goals; - Existence of long term goals.

Indicators:

- Existence of documented PPP network goals;

- Existence of documented PPP network responsibility;

- Collective view of PPP network participants on national cyber security/ defense and corresponding responsibilities;

- Collective view of what cyber security entails;

- Collective interpretation of main cyber threats in respective industry; - Experienced goal alignment between PPP network participants. Data Sources:

- Document Analysis (e.g. national cyber security strategy); - Interviews with public private and independent sources.

5 Need for network-level competencies within PPP network

- Burden on the network participants due to their presence in the PPP network;

- Complex PPP network demand more resources, centralized effort potentially mitigate the overall burden for network participants.

Indicators:

- Existence of centralized or decentralized resources within the PPP network;

- Need to centrally process and collate information; - Multiple PPP networks operating simultaneously; - Experienced burden by PPP network participants; Data Sources:

- Document Analysis (e.g. national cyber security strategy); - Interviews with public private and independent sources. Figure 10: Operationalization scheme (Author)

(37)

26 3.2 Data Collection

A triangulation of methods will be used in order to maximize the quality of the research. The thesis will draw upon the following methods: desk research, document analysis, and semi structured interviews. Desk research is a method which assesses the available body of knowledge. Document analysis is the reviewing and assessing of available documents. Semi structured interviews use open questions to allow the researcher and respondents to go more in depth or skip certain topics if they need to. The triangulation adds to the confidence one has in the results (Neuman, 2014).

3.2.1 Desk Research

Desk research will be conducted to ensure familiarity with all the relevant concepts. During the desk research it is important to gain knowledge on all the relevant information (e.g. literature, guidelines, and standards). Desk research will be conducted via Leiden Online Library, Google Scholar and open sources. This process will continue until there is theoretical saturation. The information will be assessed and coded according to the operationalization scheme. Subsequently, both document analysis and semi structured interviews will follow somewhat simultaneously. In practice there will be an iterative character involved. Interviews will lead to new documents and possibly to new interviews. Documents will be obtained via colleagues and other experts within the field.

3.2.2 Document Analysis

The researcher searched for additional documentation using the relevant keywords. Initially single words were used to locate relevant documents, such as public private partnership, and critical infrastructure. Subsequently also a combination of words has been used: critical infrastructure AND cyber security, or public private partnership AND cyber defense. Also documentation that was not publicly available was retrieved via the interview respondents.

Relevant information was administered and coded. Both document analysis and desk research enabled the body of knowledge regarding cyber defense to grow incrementally. This allowed the researcher to draft the right questions and be knowledgeable during the interviews. 3.2.3 Semi Structured Interviews

Several interviews have been conducted to ensure representation of relevant stakeholders. Individuals from both the energy and telecommunications industries have been

(38)

27 included in the interviews to represent the private incentives. Officials from the Dutch National Cyber Security Center have been included to voice public administration incentives. In addition, three neutral experts from neither the public or private side have been interviewed.

As this topic has not matured yet it can be valuable to divert from a very rigid set of questions. Semi structured interviews facilitate in this option, leaving room for the interviewer to develop new ideas based on what the interviewee’s responses are. There is a set of theme’s and corresponding questions which is used as input for these interviews (see appendix II). The challenge of conducting semi-structured interviews lies in the preparation as the researcher needs to be well prepared and knowledgeable on the subject matter (Neuman, 2013).

(39)

28 3.3 Data Analysis

An initial assessment of obtained articles and reports was made based on their abstracts. Some documents were relevant as a whole, like the National Cyber Security Strategy 2, while other documents contained little bits and pieces of relevant data. All obtained data was administered, assessed, coded, and processed (see appendix 3) related towards the theoretical framework as depicted in chapter 2. A second round analysis revealed links in the data. Taken together the data incrementally grew to a body of knowledge allowing the researcher to identify answers to the research question.

The analysis regarding PPP’s in the energy and telecommunications industry has been divided in two subchapters: network governance forms and corresponding contingencies. As the network governance forms and contingencies are unequivocally connected to each other, the analysis concerning a specific contingency or governance form will inevitably touch upon the ‘other’ contingencies as well.

3.4 Reliability & Validity

Consistency is essential in delivering reliable research. In order to maximize reliability of this research the researcher minimized the timespan of the conducted interviews. The topic of cyber is highly politicized and receives a lot of attention in the media. This can potentially influence interviewees in their responses. The responses of the interviewees are ideally under the same circumstances. An example would be the changing regulatory landscape within the cyber domain. New standards are being introduced and could potentially affect responses. Another influence could be a major cyber incident, possibly putting additional pressure on cooperation between public and private parties. None of these things occurred in the timespan of the interviews.

External validity is a problem due to the limited sample size. Generalization would therefore be quite difficult. The research is both limited by the nation state in which it was conducted and limited by the industries in which it was conducted. As both the approaches from nation states and the characteristics of industries vary greatly, one should be hesitant to draw conclusions across these borders too easily.