Assessing the influence of network governance models on cyber crisis management capacity in the Netherlands and Australia

(1)

ASSESSING THE INFLUENCE OF

NETWORK GOVERNANCE MODELS

ON CYBER CRISIS MANAGEMENT

CAPACITY IN THE NETHERLANDS AND

AUSTRALIA

Tim Haasnoot

(2)

FIRST READER AND SUPERVISOR

Dr. Tatiana Tropina SECOND READER

Dr. E. de Busser REFERENCING STYLE

Chicago Author-Date, compiled through Zotero WORD COUNT 23.717 words LANGUAGE Australian English DATE OF SUBMISSION 19th_{of January 2020}

(3)

DEDICATION

I dedicate this study to my supervisor, partner, sister, mom and dad for supporting me to the fullest during the writing process.

I also want to thank each of the respondents for their time, input and willingness to partake in the study. This work could not be realised without

(4)

LI ST OF ORGANI SATI ONS / ABBREVI ATI ONS

ORGANISATIONS – THE NETHERLANDS

Amsterdam Internet Exchange AMS-IX

Cyber Warfare and Training Centre CWTC

Defence Cyber Command DCC

Defence Cyber Expertise Centre DCEC

Defence Cyber Security Centre DCSC

Digital Trust Center DTC

General Intelligence and Security Services AIVD

Information Sharing and Analysis Centres ISACs

International Cyber Security Protection Alliance ICSPA

Joint SIGINT and Cyber Unit JSCU

Military Intelligence and Security Services MIVD

Ministry of Defence MinDEF

Ministry of Security and Justice MinJenV

National Coordinator for Counterterrorism and Security NCTV

National Crisis Centre NCC

National Cyber Security Centre NCSC

National Cyber-Forensics and Training Alliance NCFTA

National Detection Network NDN

National Response Network NRN

Netherlands Organisation for Applied Scientific Research TNO

(5)

ORGANISATIONS - AUSTRALIA

Australian Cyber Security Centre ACSC

Australian Defence Force ADF

Australian Federal Police AFP

Australian Internet Security Initiative AISI

Australian Security Intelligence Organisation ASIO

Australian Signals Directorate ASD

CERT Australia

Defence Intelligence Organisation DIO

Department of Defence DoD

Department of Foreign Affairs and Trade DFAT

Department of the Prime Minister and Cabinet PM&C

Electronic Frontiers Australia EFA

Infrastructure Assurance Advisory Groups IAAGS

IoT Alliance Australia IoTAA

Joint Cyber Security Centres JCSC

Trusted Information Sharing Network for Critical

Infrastructure Protection TISN

INTERNATIONAL ORGANISATIONS

Asia-Pacific Network Information Centre APNIC

European Union Agency for Network and Information Security ENISA

Global Forum on Cyber Expertise GFCE

Nippon Telegraph and Telephone Sydney NTT Sydney

Réseaux IP Européens RIPE

(6)

LI ST OF FI GURES AND T ABLES

Figure I: Crisis Management Capacity: three performative dimensions ... 8

Figure II: Four steps to effective coordinated action ... 10

Figure III: Causal Mechanism ... 13

Figure IV: Going down the ladder of abstraction - network governance models ... 17

Figure V: Going down the ladder of abstraction – coordination capacity ... 19

Figure VI: Circular process of coding ... 23

Figure VII: Research Design ... 26

Figure VIII: Dutch Cyber Security Landscape visualised ... 29

Figure IX: A shared effort with NCSC as central actor ... 32

Figure X: Ministry of Defence - Cyber Capabilities Overview ... 33

Figure XI: Network Validation Process ... 38

Figure XII: Towards effective coordination ... 43

Figure XIII: Australian Government Cyber Security Arrangements ... 44

Figure XIV: The ACSC Network ... 47

Figure XV: Network Validation Process ... 52

Figure XVI: Towards effective coordination ... 55

Table I: Internet Penetration Percentage ... 16

Table II: Operationalisation Overview ... 19

Table III: Interview Questions and Prompts ... 22

Table IV: Codebook ... 24

Table V: Traffic Light Protocol (TLP) ... 40

(7)

Introduction

“Decisions about network governance do not simply emerge out of thin air. Rather, they are determined by decision-makers, like government policy officials, who may want to base their decisions about how the network will be governed on evidence regarding what form seems most likely to be effective under a particular set of conditions (Provan and Kenis 2007, 237).”

Cyber Incidents as wicked problem & institutional design choices

In 2020, cyberspace permeates social and economic relations globally. As computerised systems have become increasingly embedded in modern society, the world has witnessed the rise of ‘wicked’ cyber threats (Rondelez 2018, 309). Whether caused by malevolent actors or accidentally, cyber crises are increasingly being recognised as serious transboundary risks to the IT-dependent societies of today. In one way or another, national governments have developed a sense of responsibility to ‘deal’ with cyber crises. Some note a ‘sense of urgency’ has raised within governments to develop new security guidelines and practices to counter cyber-risks nationally and worldwide (Munk 2015, 156). To improve resilience, most national governments of IT-dependent societies have advanced themselves into a set of distinctive network models that are in charge with national cyber security. The network concept has been heralded as an approach to tackle ‘wicked issues’ more effectively and efficiently (Rittel and Webber 1973, 159). Still, it is being acknowledged that governments feel insecure on how to set up and develop cyber security networks (Rondelez 2018, 309; Smith and Ingram 2017). This insecurity is justified: certain network conditions may lead to different network level outcomes (Provan and Kenis 2007). Ultimately, institutional design choices, i.e. the way we organise, impacts the way we handle an incident and may impact the overall management performance of these cyber incidents. The governance of network models is regarded as an important research object that requires further study: essentially, it determines the success or failure of the cooperative endeavours (Kilduff and Tsai 2003, 129–31). Motivated to address this important topic, this study empirically examines two cases to determine the impact of different network models on the functioning of these networks. In particular, this study examines:

To what extent do network governance models influence cyber incident management performance (coordination performance) in the Netherlands and in Australia?

To deliver an answer to the research question, the study employed two dominant theoretical frameworks: first, the network typology of Provan and Kenis (2007) that was used to determine the type of network for both cases. Second, the three performative dimensions of crisis

(10)

management performance (sense-making, coordination and legitimacy capacities) were used to assess performance, whereby the focus was on the coordination dimension. This study purposely focused on the coordination dimension for reasons of feasibility, operationality and time- and resource constraints. This means that a more in-depth analysis of the coordination dimension could be provided in relation to the research material.

The study carries three primary academic objectives. There is a considerable discrepancy in the security field between the theoretical attention networks receive and the empirical knowledge the field has produced so far on the functioning of these different network models. A first objective was therefore to provide a detailed empirical account on network performance. The second objective of the study was to provide a detailed comparative account of two different political contexts. Careful examination of the literature shows that research into the institutional cyber security arrangements mainly employ European countries as the main foci (Boeke 2018; Kostyuk 2014; Boin, Busuioc, and Groenleer 2014; T. Christensen et al. 2016; Rondelez 2018). I suspect that a comparative exercise between two political contexts might help in identifying certain ‘best practices’ that are generalisable beyond the two cases analysed. Lastly, the third objective of the study was to lay the groundwork for other security students by providing a basic framework for assessing coordination capacity -as part of the wider framework of Boin, Busuioc and Groenleer (2014)-.

Next to the study’s academic objectives, the goal of this study is also to help national governments understand the impact of institutional design choices on their ability to manage a cyber crisis. The results of this study may serve to improve the design of cyber security networks. Ultimately, by explicitly identifying the strong and weak points of the particular network models, national executives will be better equipped with the tools to design a network governance model that is better suited to counter cyber incidents.

To achieve these objectives and to deliver an answer to the research question, this study employed two qualitative approaches. First, a qualitative documental analysis of national cyber security strategies and other relevant policy documents was used to identify the governance model for both cases. Second, using the semi-structured interview method, the study validated the network model findings and tentatively examined the coordination performance of both arrangements.

The study continues in three principal chapters: a theory chapter, a chapter that explains how the study was conducted and lastly an analysis of both case studies. The analysis for both the Netherlands and Australia consists of a network identification exercise, followed by a network validation exercise and a short reflection on the respective ‘score’ in coordination performance. The study concludes with a summary of the main findings and recommendations for future research into network performance.

(11)

Theory

The rise of transboundary -cyber- incidents

The world of crises witnessed a dynamic shift in shape, frequency and consequences in the past decades (Boin 2009, 367). This is quite understandable: in the tightly interconnected systems of modern society, it is perhaps irrational to think that crises would ‘stay’ in the demarcated frontiers of the nation state. On the contrary, it is posited that small incidents can ‘race through’ the closely connected systems of modern society and grow into crises that are costly to recuperate from (Perrow 1999; Boin and McConnell 2007, 2; Eriksson and Rhinard 2009, 248). It is also observed that crises and disasters are increasingly interdependent, having the ability to penetrate and reach across geographical borders and traditional policy boundaries (Boin, Busuioc, and Groenleer 2014, 420). This new type of crises is called the transboundary crisis. Well-documented examples include global terrorism and large-scale migrations.

When speaking about the tightly interconnected system of today’s society, one cannot neglect the role of information technology (IT) (Eriksson and Rhinard 2009, 247). The decreasing cost of computers and other devices along with the increasing worldwide penetration of the Internet facilitated two important developments. First, there is the dependence of social, physical and critical infrastructures on globally interconnected computerised networks (Jayawardane, Larik, and Jackson 2015, 3; Eriksson and Rhinard 2009, 247). Second, actors (in the broadest sense: state actors, individuals, terrorist groups) scattered over the globe now have the ability to form -anonymous- organisations and pursue -malicious- causes online (2009, 247). The two developments are significant in that they form the basis of modern cyber threats. In line with global terrorism and migration, cyber threats have also developed the capacity to cross functional boundaries, for instance through computerised networks. It has also developed the capability to impact a broad array of referent objects that are connected to the internet (Eriksson and Rhinard 2009, 248; Clemente 2011), such as ‘Internet of Things’ (IoT) devices. Hence, cyber threats have developed a similar ‘wickedness’.

Transboundary crises also carry other defining characteristics: they can 1) easily ‘snowball’ to international heights, 2) last much longer than contemporary crises, 3) infect other segments of society and they are bound to change continuously and cause more harm –albeit in a more comprehensive way-. Increasingly so, they pose a serious challenge to the legitimacy of public and private institutions (Boin 2009, 367). It is not clear how these transboundary crises could best be dealt with. In fact, some judge these types of crises as ‘un-manageable’ (Perrow 2011). Nevertheless, there is another side of this debate that provides a more positive outlook. Among those are Bharosa et al, who agree that transboundary crises, or at least the information sharing

(12)

difficulties that originate from them, can indeed be faced though a range of technological and administrative tools (2009, 65).

Managing transboundary crises through networks

From the preceding, managing transboundary crises seems to be an extremely challenging, if not impossible, enterprise. Although academia remain divided over the proposition that these types of crises can be managed, scholars in the crisis and security management field seem to come together on the fact that transboundary crises cannot be dealt with by national governments alone (Boin, Busuioc, and Groenleer 2014, 420–21; Howitt, Leonard, and Giles 2009, 615). This insight goes hand in hand with the increasing relevance of security networks

and the move from government to governance throughout the 90s into the 21st_{century (Dupont}

2004). The more ‘hierarchical’ and state-led provision of security shifted into a more decentralised provision of security, in particular towards ‘polycentric’ or network-orientated shapes. The success and visibility of cooperative networks in Europe ignited a noteworthy interest in scholars to study the network phenomena. Scholars now identify that security provision is increasingly decentralised (Dupont 2004). Succeeding a period of identifying this new development and specifying how different networks are compared to traditional governance structures (O’Toole 1997; Dupont 2004), scholars are now increasingly interested in explaining the differences in network-structure and network-governance models (Provan and Kenis 2007; Human and Provan 2000; Whelan 2016).

Although differences between the networks exist, scholars all point to one thing that networks have in common: decentralised models are widely recognised as the most adequate structure to assess what kind of responses would work best for the crisis at hand (Boeke 2018, 450; Provan and Kenis 2007, 229; ’t Hart, Rosenthal, and Kouzmin 1993). There is indeed power in numbers: participants in networks can leverage more resources and expertise than autonomous organisations by themselves (Stone and Riley 2018). Zooming out more broadly, it is important to keep in mind that the network model does not represent a ‘one fits all’ solution, especially when it comes to cyber security (Rondelez 2018, 301). The implications of cyber security are highly diverse: the concept itself can refer to a great number of risks, ranging from the integrity of our online personal privacy, to critical infrastructure security, to electronic trade, military warfare or to intellectual property protection (Carr 2016, 49–50).

Before we discuss other advantages and disadvantages of the network model, there remains something that ought to be addressed: the definition of a ‘network’. It seems that scholars who deal with networks and network governance struggle in providing a conceptual definition of a network. What is lacking is a precise definition of a network, one that can be employed throughout the further study. In line with this study, Hajer et al. present networks as reciprocal arrangements of interaction and exchange between autonomous yet interdependent organisations which produce a greater number of resources to address ‘wicked’ issues (2004, 303)”. The latter is a fitting definition of what is examined in this study. To demarcate this definition further and render empirical analysis more practical, this study adds an element of the definition by Provan and Kenis. They define networks as a collective of ‘three or more legally autonomous organisations working together to achieve not only their own goals but also a

(13)

collective goal (Provan and Kenis 2007, 231)’. In the institutional cyber security arrangements analysed in this study, for instance, this collective goal could be to collectively manage a cyber incident. Further, it is important to consider that networks can be self-initiated by those participating in the network, or may originate from a mandate, which is typically the case in the public sector (2007, 231).

The benefits of the network approach are noteworthy in the management of transboundary crises. Provan and Kenis point towards a more effective use of resources and augmented learning capabilities as well as improved capacities to tackle complex issues (2007, 229). This is particularly true for tackling cyber issues: network governance, especially when supplemented by private actors, is considered a true ‘cornerstone’ in cyber incident response activities (Silva 2017, 125; Stone and Riley 2018). As such, it is believed that cyber expertise can be leveraged more quickly in a network arrangement (2018). The value of a network also rests on its flexibility: through networks, participants are in a better position to respond to the continuously changing crisis dynamic because resources, skills and decision-making powers are pooled together. In hierarchical bureaucracies on the other hand, administrative processes are often in the way, limiting a quick response (Kapucu and Van Wart 2006).

Although several notable advantages of the network model exist, it is also characterised by its deficiencies. For instance, public-private arrangements are often burdened by a ‘divergence of interests, disparity or disagreement on who will foot the bill’ (Carr 2016, 62). This means that network participants may agree on a certain policy or action that needs to be undertaken, but don’t want to devote their individual resources to the implementation of the task. In addition, the network model is inviting for ‘free-riders’ who benefit from participating in the network, but do not give anything in return. When more participants are involved in a network, it may also happen that decision-making processes become more time-consuming and resource demanding (Provan and Kenis 2007, 242). A last challenge of the network model is that networks are constantly tasked with maintaining legitimacy. Human and Provan argue that legitimacy must be addressed both internally and externally for networks to survive in the long-term (2000, 328–29). Internally, the network must legitimise its coordinated efforts and the potential advantages thereof, to get active commitment by the participants (idem.). Externally, it must maintain viability and preserve ‘a good face’ of the network to outsiders, for instance to secure financial backing or attract new participants that are crucial for network-survival (idem.). In light of this study, some scholars would most probably ask: why study ‘networks’ when it comes to cyber security? For long, scholars thought of cyber security as organised in public-private partnerships (Healey 2017; K. K. Christensen and Petersen 2017). I would posit that these claims no longer hold true: increasingly, we see that other actors join the cyber security governance stage, ranging from international organisations, such as the European Union Agency

for Network and Information Security (ENISA) (Trimintzios et al. 2015), profit and

non-governmental units like Luatix1_{and the International Cyber Security Protection Alliance (ICSPA).}

Meanwhile, groups of individuals are participating in ethical hacking networks on Discord

(14)

(examples are The Hackerspace2_{and Red Ridge}3_{). Again, to address these cooperative}

endeavours as ‘public-private partnerships’ would not be appropriate in the present-day situation. Perhaps it is better suited to address these all-encompassing collaborations as ‘public-private-people partnerships’ or simply as ‘networks’. To quarrel about how cyber security is organised, is however, not in the scope of this study. Nevertheless, it is important to devote attention to the question what is studied and why this study focusses on networks instead of

public-private partnerships.

Networks come in various forms and sizes. In fact, every network may be unique. In the past years, scholars have attempted and succeeded in finding certain network characteristics that exist in a wide variety of cases. Such an attempt was made by Provan and Kenis and it serves as one of the two theoretical backbones of this study. Provan and Kenis provide a typology for different network governance models, which is three-fold and consists of: participant-governed networks, lead organisation-governed networks and network administrative organisations (or: the NAO-model) (2007, 233–34). The framework provides a firm basis to categorise national cyber security networks and is used in this study to compare the two national cyber security arrangements of the Netherlands and Australia. The typology may fall short on one aspect. As Boeke rightfully notes, these models present ‘theoretical ideal types’ (2018, 451). In practice, a network cannot not solely be characterised as a specific network type: networks often display a mixture of characteristics and evade distinct classification (idem.). Besides, it ought to be taken into account that networks can evolve: over time, predetermined goals may be subjected to change, demanding a different network structure (Provan and Kenis 2007, 246). Nevertheless, the typology by Provan and Kenis permits a clear conceptualisation of different network governance models which will make comparison between the different case studies possible. Each of the three network governance models will now be explained in detail.

Participant-governed Networks

In the participant-governed network type, the network is governed by all the actors involved (Provan and Kenis 2007). This means that there is no presence of an ‘overseeing’ governing body: governance is done by the members of the network themselves. Decision-making is symmetrical and generally based on consensus. In addition, the relations between the network actors are equal. Another key characteristic of this highly decentralised form of governance is that participation is generally voluntary. In addition, it is commonly observed that the level of trust between the involved actors is quite high (2007, 237). The participant-governed network is not by default decentralised: individual organisations or a subset of participants may take up certain administrative and coordinative tasks (e.g. in rotation) so that the network participants can fully focus on the collectively determined goals. The characteristics of the participant-governed network and the other networks that are being discussed, are important for the further development of this study: together, they constitute important indicators for determining what type of network model we are dealing with in the Dutch and Australian case.

2_{https://disboard.org/server/618447744469827604} 3_{https://disboard.org/server/609682500473716736}

(15)

Lead Organisation-Governed Networks

Contrary to the participant-governed network, a lead organisation-governed network is characterised by its centralised and hierarchical posture (Provan and Kenis 2007, 235). The lead-organisational model is still a decentralised collaboration between different actors, but it features one single ‘lead’ participant that is responsible for the network governance activities. Generally, the network-goals are closely aligned with the goals of the lead organisation (idem.). Naturally, different relationships can exist between the lead organisation and the other participants. In some cases, the lead actor may be highly involved in the network activities, in other instances it may not really work with the participants at all but only delivers the final say. Similarly, it may solely coordinate the overall response or only act as the provider for financial resources. It may also happen that the lead actor changes its role or accepts other responsibilities. Here again, we see that networks are highly evolutionary and that it is difficult to ‘compartmentalise’ certain networks in discrete categories. Hence, variations within the lead-organisational model exist, but it is important to consider that in this particular network governance model there is always an actor that is leading the network.

Network Administrative Organisation (NAO-model)

Next to the participant-governed network and the lead organisation-governed network, there exists a third network type. This is the network administrative organisation, or the NAO-model. The basic premise of this network type is that the network relies on a separate actor, that works ‘outside’ the network, which is tasked with network-management, e.g. a governmental entity like a ministerial body. Since the governance efforts are managed by one responsible organisation, some would posit that this model is centralised (Provan and Kenis 2007, 236). In terms of governance this may be the case. In practice, however, the administrative unit is solely concerned with the exclusive purpose of network governance, and not so much with other network-related activities, e.g. decision-making or expertise-sharing, that are highly decentralised activities. I would suggest that this network is thus not inherently ‘centralised’ or ‘decentralised’ but can be considered a mixture of both ends.

(16)

Assessing -cyber- crisis management capacity

As Provan and Kenis note, not much is known about the overall functioning of networks (2007, 229). Developing a deep understanding of the functioning of networks requires us to make some form of assessment whereby we look into network performance or capacities. However, to do this without a theoretical support would be a very normative exercise. Performance could be analysed by looking back on how the network performed during certain crisis situations, but doing so would involve what Starbuck and Farjoun famously describe as ‘hindsight bias’ (2005). It makes more sense to examine the functioning of networks by assessing certain capacities. In their contribution, Boin, Busuioc and Groenleer explored what crisis management capacities might look like. They did so by formulating three performative dimensions of crisis management (2014, 420–23), see

Figure I, which allow comparison between the different government models:

Figure I: Crisis Management Capacity: three performative dimensions

Next to capacities on these different levels, crisis management capacity can be further specified (or categorised) in capacities used before crisis and capacities used during crisis (Backman and Rhinard 2018, 264). Pre-crisis capacities concentrate on ‘preparing’ for crises (idem.), e.g. assessing threats, conducting simulation exercises, actively scanning for early signs of security threats through systematic examination. Actual-crisis capacities are ‘activated’ when an actual crisis develops (idem.), e.g. decision-making protocols and communication strategies. The pre/actual divide of crisis management capacity is relevant since each performative dimension has distinct meanings in the two respective situations. Sense-making in a pre-crisis situation for instance, involves developing a general sense on recent security developments and understanding innovative security threats. It differs in an actual crisis when crisis managers need to make sense of real-time information of a particular incident. Regarding coordination capacity, pre-crisis capacities are generally aimed at assembling key parties, exchanging

Coordination

Legitimacy Sense-making

(17)

information on threats and risks, assisting network participants with building capacity and at practicing crisis situations (Backman and Rhinard 2018, 266). During crises, coordination is generally concerned with exchanging information on the situation, joint sense-making, leveraging expertise, communicating responsibilities or actions and operating on the threat coherently, i.e. bringing together a set of differentiated activities into a unified arrangement (Wolbers, Boersma, and Groenewegen 2018, 1522). In short, different capacities are demanded in the two different situations. Like the focus of the study, the three performative dimensions (Boin, Busuioc, and Groenleer 2014) principally centre on actual-crisis capacities. These will now be discussed individually.

S

ENSE

-

MAKING

The first dimension involves the managerial capacity to ‘make sense of a crisis’ that is either emerging or already unfolding (2014, 424). Effective sense-making consists of gathering, analysing and distributing vital information to everyone involved. In one way or another, the sense-making activities need to be coordinated, or there is the risk that different meanings develop. The most famous example of this was the Stockwell shooting, whereby different interpretations led to the shooting of an innocent civilian (Cornelissen, Mantere, and Vaara 2014). Sense-making is even more important in relation to cyber threats. Acting on false interpretations can often do more harm than good, especially when working with -malicious code- that can behave irrationally. There are also administrative challenges. As explained before, cyber incidents can easily transpose geographical and traditional policy barriers. When a cyber incident unfolds, many different actors (public, private, international, national) could be involved, even actors that are not necessarily participating in the network. With or without a clear and joint sense-making mechanism, it is still a major challenge to keep everyone on the same understanding of a rapidly evolving incident. The latter hints to a second important dimension in crisis management: coordination.

C

OORDINATION

The second dimension is theoretically defined as ‘the capacity to coordinate the resources’ of all those involved (2014, 424), but ultimately it is concerned with how ‘different areas of work can be performed in concert (Wolbers, Boersma, and Groenewegen 2018, 1521)’ to address a particular security development. Of the three performative dimensions, the coordination dimension requires extra attention per the scope of this study. Of the three dimensions, coordination is perhaps the most important: failure to coordinate ‘can lead to gaps or overlaps in measures taken (Comfort and Kapucu 2006)’ and ultimately cause the incident response to fail. It may also start a blame game between network participants, jeopardising the relationships between them.

There are various challenges of effective coordination. Boin et al. note that crises generally require distinct combinations of capacities (2014, 424). In crises situations, actors have to bring together diffused sets of knowledge and coordinate together under the pressure of time. The coordination task also entails challenges such as identifying key actors and partners and

(18)

facilitating collaboration between them in relation to the risk, threat, or crisis at hand (Boin et al., 2016:17). Capacities are often scattered across various agencies and organisations: this poses horizontal coordination challenges (Boin, Busuioc, and Groenleer 2014, 426). Regarding cyber crises, specific expertise is often required, that is dispersed over different agencies that not necessarily work together on a regular basis. Say for instance a hacking incident has taken place on critical infrastructure (the water supply, or worse: the electricity generation). Expertise then, has to be mustered from critical infrastructure administrators and specialists in computerised infrastructure networks. Particularly in the participant-governed network, coordinating can be very challenging because of the lack of a ‘central, orchestrating actor’ (2014, 426). On the other side of the spectrum, the lead organisation-governed network has more success with coordinating responses in the network, mainly due to the fact that these networks are more centralised: it is also clearer who is taking the lead (2014, 428). Simply put: centralisation lessens the burden to coordinate. Lastly, effective coordinated action also relies on the willingness of the network’s participants (2014, 428) to work together or share sensitive information with each other.

Four steps to effective coordinated action

As pointed out prior, the coordination dimension is a particular important dimension. The literature suggests that effective coordination consists of several important aspects (Comfort and Kapucu 2006; Faraj and Xiao 2006; Boin, Ekengren, and Rhinard 2013; Bigley and Roberts 2001). Taken together these aspects, I suggest that coordination capacity depends on several steps, see Figure II. The four measurable aspects provide us with a way to assess coordination performance for the network models of the study. As visible in Figure II, the end-goal of the four-step process is effective coordinated action. This requires several steps:

Figure II: Four steps to effective coordinated action

At the base of collective action, I find access to timely and valid information to everyone involved an essential step. Without correct and swift information, the whole collective operation is bound to fail. Naturally for this to occur, pre-crisis information arrangement structures must be in place through which information can be searched, exchanged, absorbed and adapted

(19)

(Comfort and Kapucu 2006, 309). Without information exchange, the coordination process is disrupted and halted. The information acquired is also essential in the second step towards effective coordination action, namely in the joint sense-making activity. This footing relates closely to the first performative dimension of crisis management capacity, but carries an important coordinative element, i.e. gathering everyone involved to make sense of the calamity and making sure everyone is on the same level of comprehension (Faraj and Xiao 2006, 1164–65). When there is an idea of how the situation is unfolding, a next step is to bring experts, know-how and/or resources together to deal with the situation. Bringing together parties across institutions and policy sectors and timely resource management between those involved is also understood by Boin et al. as a key aspect of coordination (2014, 424; Boin, Ekengren, and Rhinard 2013). The acquired insights and interpretation of the threat and distinct expertise would naturally point to actions, which taken together in a unified arrangement, would constitute coordinated action (Wolbers, Boersma, and Groenewegen 2018, 1522). Lastly, effective coordinated action also carries the ability to easily switch roles as part of the response (Bigley and Roberts 2001, 1287). Role-switching is flexible when network participants are relatively easy with transferring responsibilities to another actor, for instance to work on incident response. Willingness to transfer responsibilities becomes particularly interesting when private actors are involved. Private actors are generally not keen on sharing confidentialities or sharing the way how their IT-systems operate. Role-switching capacity is particularly relevant when collective action seems ineffective. It may also happen that a previously undertaken action becomes inappropriate in light of the continuously changing dynamic of the crisis.

L

EGITIMACY

Lastly, there is the dimension of legitimacy in crisis management capacity. Boin et al. posit that in times of crisis, ‘the normal procedural and institutional safeguards are typically relaxed’ (2014, 424). They note that decisions and measures that are made in relation to the crisis management situation, could have deliberate repercussions. For instance, immediate decisions and measures from a cyber hack (e.g. the active monitoring of surrounding networks to determine the perpetrator) could have consequences for the privacy rights of other individuals who happen to be on the same network. The legitimacy dimension is particularly relevant to explore the accountability structures in crisis management, and very well aligns with Boin et al.’s article’s focus on the role of the EU in this process (2014). For the focus of this study however, the dimension is less relevant as this study is mainly concerned with the relation between network types and actual crisis management performance and not so much with the side-effects of the undertaken crisis management decisions.

Crisis Management Capacity in Cyber Incidents

The context of a cyber incident may ask different things of the crisis management process. A cyber incident can involve a wide range of things: ‘cyber security’ is almost as wide-ranging as the term ‘security’ itself. Working with concepts that relate to cyber and particularly cyber security is intrinsically difficult. As noted by Cavelty, nearly every element related to cyber in

(20)

scholarship is subject to contestability: the overuse of the concept has led to a status where ‘cyber could mean everything and, yet, nothing (Cavelty 2007, 14)’. What does crisis management capacity and coordination performance mean in the face of cyber incidents? Is it any different than in ‘normal’ crisis situations? I would suggest that crisis management capacity and in particular coordination performance in the face of cyber crises is not fundamentally different than in ‘normal’ crisis situations. The only difference is the nature of the threat, but the coordination mechanism, i.e. the ‘process of bringing together a set of differentiated activities into a unified arrangement (Wolbers, Boersma, and Groenewegen 2018, 1522)’ along with the different steps on which effective coordination rests, (information sharing, resource/expertise gathering, etc.) remains the same. It makes no difference whether we are talking about a natural hazard, a terrorist attack or a cyber incident, in the end, the process leading to the ultimate response should be similar.

Thus, the capacity of cyber crisis management systems here, is conceptualised as a three sided coming together of sense-making, coordination and legitimacy (2014). Considering the scope of this research and what it tries to achieve, but also due to the previously addressed resource and time-constraints, it is not appropriate to research all dimensions. Instead this study focussed on just one: the coordination dimension. This has several perks: First, one can develop a more comprehensive and detailed analysis of the coordinative aspect of the different network types. Second, it allows for a more easily manageable and better structured empirical study.

(21)

Methodology

Introduction to the research design

The way a study is conducted, for instance what data is selected (and what data should be ignored), how the data is collected and analysed, can affect the study’s outcome. Having addressed the two main theoretical frameworks that this study employs, the aim of this chapter is to describe how this study was conducted. It elaborates further on the methods that were used to provide an answer to the research question: to what extent do network governance models

influence cyber incident management performance (coordination performance) in the Netherlands and in Australia? The causal mechanism, as visualised in Figure III, provides us with an initial and

helpful instruction on how this study was executed. With the network types identified, we can study its relation to the performative dimension of coordination (2014).

Figure III: Causal Mechanism

Important to consider, is the structure of the findings and analysis chapter, which is broadly arranged three-fold: a network identification exercise (to determine the network type), a network validation exercise (using the data from the interviews) and lastly, a reflection on the coordination dimension for the analysed network.

This chapter specifically develops in six parts. It starts with justifying the comparative case study method that was used to carry out this study. Next, the selected cases are accounted for. Then the operationalisations of the study’s main concepts are presented, followed by an account on the different strategies that were used to collect the data on these operationalisations. Lastly, I reflect on the generalisability of the findings and possible limitations of the research design. An overview of the research design is given at the end of the chapter to provide the reader with a simplified view on how the research was conducted.

Network model Coordinative and Sense-making,

legitimate performance

(Cyber) Crisis management

(22)

The Comparative Case Study Method

Different qualitative research strategies exist to deliver an answer to the research question. This study uses the comparative case study method. The case study method is built on the idea of doing research by “obtaining a ‘case’ or a number of ‘cases’ through empirical examination of a real-world phenomenon -within its naturally occurring context-, without directly manipulating either the phenomenon or the context (Kaarbo and Beasley 1999, 372).” Yin defines the case study method as “an empirical inquiry” that investigates a contemporary phenomenon within

a real-life context (Yin 2003)”. The format of this comparative case study analysis is based on

systematic and explorative comparison of two cases.

There are three principal reasons why the comparative case study method applies. First, the case study method is appropriate because it is identified as the best way to study new and explorative phenomena (Yin 2003). Although, the Provan and Kenis’ model has been applied in other studies to classify cyber crisis management structures of -generally- European countries (Boeke 2018; Rondelez 2018). And while different in their attempt, scholars also analysed crisis management capacities for countries or supranational arrangements, such as the European Union (Boin, Busuioc, and Groenleer 2014; Backman and Rhinard 2018). The studied link between crisis management capacity to different network types, however, is experimental: contemporary research does not reveal prior empirical examination of the relationship between the two phenomena, which suggests that a detailed case study analysis is fitting.

A second reason why this method is applicable is because case study research is a good method to examine complex and real-life phenomena (Yin 2003). As it stands, it can be gathered from the theoretical chapter that the relationship between governance structures and network performance is quite complex -perhaps unique for every network analysed-. In addition, the interaction and cooperation between participants in network models that is analysed occur in a real-life context.

At last, a third reason for using the case study method is because I suspect that the comparative exercise between the two networks can lead to interesting results that are perhaps generalisable beyond the two cases analysed. As Yin notes, having multiple cases, improves the study’s external validity, i.e. the extent to which the conclusions of a study can be generalised across other situations (2003, 37). In other words, analysing multiple case studies offers a firm basis to generalise findings (idem.), which is very much in line with the goal of this study: after all, the aim is to theorise a relationship between network governance and network performance. The case study method is best suited to achieve this goal.

Ultimately, the comparative case study method helps us to examine the relation between the network governance model and the coordination performance not for one, but for two cases, thereby producing more compelling evidence for the suggested effect. There are various ways we can reach generalisable conclusions. First, if both cases are identified as participant-governed networks, then it makes the comparative analysis even more compelling since it follows a ‘replication logic’ (Yin 2003, 47), i.e. if the findings in one case hold in the other. To illustrate, the participant-governed network type may positively impact coordination performance in both cases. It thereby strengthens the argument that participant-governed networks, in general,

(23)

complement coordination capabilities. Contrary, this logic also holds: both cases may show a negative impact of the participant-governed network type on the capability to coordinate. A third option exists as well: the Australian case may, for instance, not display the same properties as the Dutch case, thereby ruling out the ability to make cross-case generalisations on the relationship between the particular network type and coordination performance.

All in all, the exploratory intent of the study, in combination with the complex relationship between the two units of analysis and the anticipated benefit from comparing the two cases seems to suggest that the comparative case study method as a research strategy is best suited to address the research question (Yin 2003, 1–2).

Selecting the cases

Now that the comparative case study method is accounted for, a logical follow-up question is what cases would be most fitting to compare. In the case selection process it is important to identify certain characteristics that are similar for both cases, while allowing variations in other features (Resodihardjo 2009, 30). With this method it becomes possible to determine whether a relationship is a recurring phenomenon.

Aware of the advantages of analysing multiple instances, two cases are examined in this study: the Netherlands and Australia. In particular, the emphasis is on the governmental cyber security apparatus for both countries. Variation is introduced in the cases through the selection of two different political contexts, a European context and an Anglo context. The choice to examine both contexts comparatively was made deliberately because it has academic relevance: contemporary scholarship on cyber governance has generally used European countries as their main foci (Boeke 2018; Kostyuk 2014; Boin, Busuioc, and Groenleer 2014; T. Christensen et al. 2016; Rondelez 2018). Of course, multiple governmental arrangements exist within both contexts. The choice to compare the Netherlands and Australia specifically in their respective contexts was made because they are comparable. The similarity of the Australian and Dutch cyber security landscape makes the coordination performance assessment more interesting because of the earlier described ‘replication logic’. It would make the assessment even more interesting if both are identified as the same network model.

Next to the similarity in both cyber security governance landscapes, it is rational to compare both cases for another two reasons.

First, if we zoom out and observe in the general sense, both Australia and the Netherlands are highly IT-dependent market economies where decentralised network types and public private partnerships (or: networks) are deeply embedded in the organisational structures of society to counter a wide range of issues (e.g. in business, local government, etc). Both excel and pioneer in the network approach (Carr 2016, 44): Australia has a long history of public-private partnerships. Regarding cyber security, the 2009 national cyber security strategy illustrates that the emphasis has been on public-private collaboration ever since the beginning of a governmental approach to cyber security (Australian Government 2009). In the Netherlands, cyber security structures have been in place since 2010 that closely align with the

(24)

multi-stakeholder model (Hathaway and Spidalieri 2017, 4). Anno 2020, public-private collectives have become intrinsically important in national efforts in cyber security.

Second, the two cases are similar in a socio-technological sense: both the Netherlands and Australia lead in the Cyber Readiness Index (CRI) (Hathaway 2013, 6), which examines a ‘country’s maturity and commitment to securing its national cyber infrastructure and services (Hathaway 2015, 1–2)’. It is important to select two cases that are on the same level in terms of cyber maturity to make more representative claims. For instance, it would be inappropriate to compare two cases whereby one case is not as ‘cyber mature’ as the other, because this would essentially mean that there is lower cyber crisis management capacity in the first place. Next to the equal score on the cyber maturity index, IT is actively interwoven in society for both cases. Regarding internet usage among the population, both countries have a high internet penetration rate, with Australia having 86.5 % of their population online in 2019 and the Netherlands 95.6 %, see Table I (Internet World Stats 2019).

Table I: Internet Penetration Percentage

Country Population (2019

Est.) Internet Usage (June 2019) % Population (Penetration) Australia 25.099.636 21.711.706 86.5% Netherlands 17.132.908 16.383.879 95.6%

Similarly, both countries roughly have the same -international- outlook on cyber security and value international commitment to secure digital assets (NCSC 2018a, 23; Australian Government 2016, 39). In both the Netherlands and Australia, there is the shared view that ‘domestic’ cyber security also involves building cyber capacity elsewhere (idem.). Therefore, both countries are actively involved abroad in international cyber security arrangements, e.g. the National Cyber-Forensics and Training Alliance (NCFTA) and the Global Forum on Cyber

Expertise (GFCE) and in international information sharing programs (Hathaway 2015, 19).

What to look for: operationalisation of the main concepts

Now that the case selection is accounted for, it ought to be indicated what to look for in both cases in relation to the research question. The conceptual dialogue concerning network models and crisis management performance only takes us so far. What is needed are clearer definitions and specific operationalisations in order to examine the concepts empirically.

Clearer definitions are needed because the perk of the case study method, its richness in data, in combination with the vast information acquired from the semi-structured interview method also provides us with the challenge to make sense of all the relevant data. We have to know

(25)

what to look for to make any valuable claims. In scholarship, theory generally provides a way to make sense of acquired information: it determines what is important to investigate and what is not (Yin 2003, 38). The two theoretical frameworks specified in this study, provide an initial idea on what to look for. To operationalise, I returned to the literature to develop a detailed understanding of what the concepts would mean in practice and how they could be recognised empirically. In essence, the theoretical frameworks offer a set of benchmarks, or indicators, that can be used for analysing the data.

Operationalising network governance models

There are many ways one can arrive at identifying a certain network type, some less valid than others. If we take Provan and Kenis’ framework, several features of the different network types are described, see Figure IV. For instance, the network features of a participant-governed network are:

• shared governance;

• symmetrical decision-making; • high levels of trust;

• voluntary participation; • high-goal consensus.

The study’s network identification exercise weighted the data from the national cyber security strategies and other policy documents with the features of the different network types to conclude on the network model. This analytical process is often described as ‘pattern matching’ (2003, 39) and it helps us making sense of the detailed data. The pattern matching process, that is grounded in theory, also improves the construct validity of the study, i.e. the issue whether we are assessing what we want to assess (Yin 2003, 34): rather than stepping into the unknown and making loose claims, the theory gives us an academically justified direction when something is x or when it is not. Specifically, theory helped guide this empirical work by providing benchmarks, like the ones offered in Figure IV.

(26)

Operationalising crisis management performance (coordination capacity)

Regarding coordination capacity, theory was, again, used to operationalise by the identification of several benchmarks. As demonstrated in the theory chapter, the literature developed a clear idea on the conceptual meaning of crisis management capacity. It found that crisis management capacity can be understood as a three-tier assembly of different performative dimensions (sense-making, coordination & legitimacy). The ‘score’ of the dimensions can ultimately say something about the overall crisis management capacity. The literature developed a short account on the conceptual meaning of the individual dimensions, but now the question is how we can understand

the separate dimensions empirically, and in particular the coordination dimension?

The literature serves as an initial point of departure. In the theory chapter several steps were identified that lead to -effective- coordination, see Figure II. Similarly to the features of the different network types, the footings can be mobilised empirically. To begin, I identified that coordination rests upon timely and valid information and on existing information arrangements that facilitate information search and exchange. This is the first footing, which is grouped under the name: access to timely & valid information through information arrangements, see Figure V.

Second, it was established that effective coordination also encompasses joint sense-making processes (Faraj and Xiao 2006, 1164–65). This is the second footing, which I classified as joint sense-making activities, see Figure V. The findings and analysis part may focus on the question whether or not sense-making activities are facilitated, or to see if sense-making is a standard exercise amongst the participants.

Third, theory identified timely resource management between those involved (2014, 424) and bringing together parties that are scattered across institutions and policy sectors as important in the coordination process (Boin, Busuioc, and Groenleer 2014; Boin, Ekengren, and Rhinard 2013). I group these features as resource & expertise gathering, see Figure V.

Lastly, it was illustrated that effective coordinated action also entails the ability to switch roles. Particularly because of the ‘shared governance aspect’ of the participant-governed network and ‘the sense of equality’ that exists amongst participants, it is interesting to analyse how flexible the network is in changing pre-determined roles (e.g. leadership roles, informative roles). I therefore cluster these under role-switching.

It is important to point out that I do not neglect other important footings for effective coordination, nor do I value one footing over the other, rather this is what the theory and accompanied literature provides and considers as the main aspects of the coordination dimension.

(27)

Figure V: Going down the ladder of abstraction – coordination capacity

O

PERATIONALISATION OVERVIEW

Thus, grounded in the two theoretical frameworks discussed in this study, the following variables and indicators can be identified:

Table II: Operationalisation Overview

Concepts Variables/Dimensions Indicators/Attributes

Modes of network

governance (X) Participant-governed network Shared governance; symmetrical decision-making; high levels of trust; voluntary participation; high-goal consensus.

Lead organisation-governed

network Governance by leading actor; hierarchical; central coordination. Network administrative

organisation Governance by separate body or actor; centralised governance, decentralised network activities.

Crisis management

capacity (Y) Coordination Access to timely & valid information _{through information arrangements;}

joint sense-making activities; resource & expertise gathering; role-switching.

Sense-making ‘Efficient’ gathering, analysis and

distribution of vital information to everyone involved.

Legitimacy Risk-assessments;

Existence of counter-weight actors / check-balances system.

(28)

Data Collection Strategies

Naturally after pointing out what to look for, the next step is to explain where to look for it and which data collection strategies are best suited for the research design. I found two collection strategies fitting. One has been specified before: to determine the network model in both cases, cyber security strategies and official policy documents were studied in relation to the typology of Kenis and Provan (2007). The choice to examine cyber security strategies is based on the fact that they effectively show how national cyber security is ‘managed’, clearly displaying who is

responsible for what, what is deemed important, how this is addressed and by whom. Next to ‘raw’

documental evidence, other academic contributions were taken into account where possible. Albeit limited, contemporary scholarship has worked on governance network type identification before, for instance for Belgium (Rondelez 2018) and for the Czech Republic, Denmark, Estonia and the Netherlands (Boeke 2018). To make the picture as complete as possible, the analysis of the Netherlands (2018) was therefore juxtaposed to the findings of this network type identification exercise. The documental analysis in combination with the network typology, allowed the researcher to identify the network type we are dealing with. The second source of data was data obtained from semi-structured interviews. The semi-structured interview method is most appropriate as a data collection strategy for one principal reason: it allows us to examine how things work in practice. It is no surprise that this study deals with studying complex relations between people. Previously it was pointed out that clues from written documents may not accurately display how network participants work together in everyday settings, after all, network types represent theoretical ideal types (Boeke 2018, 451) . Some of the network features are most likely visible in practice, in the every-day workings of the network. We therefore also need to consider the input of individuals as the main data provider. This is the main reason why the interview method is best suited.

The interview method has two primary functions in this study. First, they served as a way to

validate the identified network type from the documental synopsis for both cases. Naturally, the

second function of the interviews is that it helps us to assess the coordination capacity for both cases.

Interview Type

The choice of the interview method is one thing, still, there is the choice what way the interview is set up. Three main interviewing strategies exist, they are: the structured interview, the

semi-structured interview and the un-semi-structured interview. Each type of interview comes with its own

advantages and disadvantages. The structured interview for instance, makes cross-interview comparison worthwhile but leaves no room for divergence from the topic. The unstructured interview in contrast, makes generalisation difficult per the different questions that are asked and the different answers given. The semi-structured format is often regarded as the perfect middle way (Gillham 2005, 70): it has the advantage that careful generalisations from the interviews can be made whilst at the same time it allows flexibility: the researcher can divert from the pre-set questions, for instance to explore an interesting side-avenue during the conversation.

(29)

The semi-structured interview method rests upon several important pointers (Gillham 2005, 70): • the same questions are asked to all the interviewees;

• equivalent coverage is ensured by prompts, i.e. supplementary questions that the interviewer could address if the topic of these prompts have not yet been addressed; • interview time should be equal for each interviewee.

Who

Ideally, when employing the interview method, one should identify a target population and make a quota sampling on the characteristic that is most relevant for your study for reasons of representativeness. However, due to limited resources (connections) and time constraints, this study resorted to the ‘snowball sampling’ technique, i.e. acquisition of interviewees through referrals. Networks are the holy grail for this technique as each network participant is connected with other participants. The technique is academically supported: Cohen and Arieli identify that snowball sampling is particularly a strong sampling method when access to ‘hidden’ interview populations is difficult, and when the nature of the study is exploratory (2011, 427). This also holds true for the current study: generally, individuals in cyber security network work with confidential information, matters of national security, often in collaboration with national intelligence agencies. Cyber security officials that work in governmental arrangements are generally not easily and publicly approachable. Hence, we are dealing with a hidden population. This in combination with the exploratory nature of the study, makes the snowball sampling technique a sensible option.

Question Type

What types of question are asked, directly influences the answers given. It is important to think about the way questions are formulated. Some individuals that were interviewed were not directly involved with network activities. Recognising the background of some individuals, the interview questions had to be delivered differently: instead of asking directly about the

individuals’ understanding of the network as if he/she was part of it, it was more fitting to ask the

individual’s ‘outsider’ civil society perspective on the contents of the questions.

Naturally, the semi-structured interview method permits that questions are open (Gillham 2005, 70). After all, qualitative research is about discovery and finding new answers, not about demonstrating whether it is x or y. It is also important to develop an interview focus because this allows structured analysis (2005, 72). Ideally, the interview as a whole should focus on one or two sub-topic(s) (idem.), which is exactly what is attempted here: network type validation & coordination capacity. Lastly, by focussing individually on the different attributes that make up the study’s main concepts, questions are relatively distinct from each other, which is considered an important task in developing interview questions (2005, 72).

As described in Table III, the interview setup also contains a scenario question (question 5). The reason behind this particular type of question is because scenario building can proof valuable in the analysis: it displays how the network functions in practice. It is a helpful tool to uncover ‘hidden’ processes. For instance, by asking the interviewee who is called first in a cyber incident

(30)

might uncover a main node or leading actor in the network. Similarly, asking how connections are established in a cyber incident, might bring the underlying coordination processes to light. We also have to be careful about so called ‘interview effects’, i.e. the effects that, for instance, the order of questions or the phrasing of questions may have on the answers given. Asking the individual directly whether there is a leading actor in the network may bring unintended effects. The interviewee might point to an actor that he thinks is ‘leading the network’ for the sake of the question. However, it is not necessarily the case that such an actor exists. Hence, caution is required in the order and phrasing of the interview questions.

On several occasions, the interviewee was more comfortable with speaking in their native language (Dutch). Therefore, the interview questions were translated, and the conversations were held in Dutch. Several parts of the conversations that were used in the analysis were however, translated to English for analysis. They were translated with care so that the meaning of the answer was not lost or altered.

Table III: Interview Questions and Prompts

On the network model (validation)

1. How would you characterise collaboration in your cyber security network?

• Do you work with other organisations (and how many)? Is participation voluntary?

• Would you say that the network features a trusted space for information sharing? / Is it possible for participants to share sensitive information? • Is there an organisation governing the network? Do you share

governance over cyber security issues? 2. How would you

characterise the decision-making process in your network?

• Do private actors have the same decision-making power as public actors? (or: Would you say that every actor is equally involved in the decision-making process?)

• Is there a feeling that a goal / agreement should be met, for instance at the end of a meeting?

• Are concessions easily made (in order to progress to action)?

On coordination performance

3. What kind of information arrangements are in place for sharing cyber threat intelligence?

• Who has access to it?

• How regular is information shared / updated?

4. How flexible is the network in role-switching? (e.g. during incidents)

• Are responsibilities easily, i.e. quick and without hassle, transmitted to other participants? What about leadership roles?

5. What is the code of conduct, if there is any, if a cyber incident arises / is identified?

• Who is called first?

• How do you establish connections between the different organisations in your network?

• How does the network make sense of all the information? Is everyone partaking in the sense-making process?

• How would you characterise the response? Is there an actor who is clearly taking a lead?

(31)

Drawing conclusions

Naturally, what is left is explaining what to do with all the data acquired; how do we make sense

of the data, how should the data be analysed and what conclusions can be drawn?

C

ODEBOOK

D

EVELOPMENT

Analysing interview data is often regarded as ‘a multistep “sense-making” endeavour (DeCuir-Gunby, Marshall, and McCulloch 2011, 137)’. To analyse data from the interviews, it is helpful to develop a codebook. This often, ‘natural’ step is generally not explained in contemporary research (2011, 137), it nevertheless is an important step that requires some elaboration.

A codebook consists of ‘codes, definitions and examples that is used as a guide to help analyse interview data (2011, 138)’. Through the use of codes, i.e. labels or tags assigned to the data acquired from the interview, it can be examined how the interview data answers the research question that is directing the study (2011, 238). The operationalised concepts that were described prior, function as these codes. Developing a codebook is also a way through which research procedures can be documented, which allows other investigators to replicate the analysis. It is therefore an important step which can increase the study’s reliability.

There are two types of codes (Ryan and Bernard 2003, 781): theory-driven codes, which are derived from the literature and data-driven codes, which result from other -raw- data sources, in this case from the interview data. To develop theory-driven codes, codes were developed akin to the operationalised concepts of the two theoretical frameworks, see Table IV. As clearer insights are gained on the interview data, some codes may need to be revised or added, this is natural for the coding process: coding is an iterative process, see Figure VI (DeCuir-Gunby, Marshall, and McCulloch 2011, 138). The consequent alterations and extensions are the data-driven codes.

Figure VI: Circular process of coding4

Practically, a codebook should consist of three components (2011, 138): the code name or label, a full definition that specifies when something can be regarded as the particular code and lastly, a real-life example where appropriate.

Assessing the influence of network governance models on cyber crisis management capacity in the Netherlands and Australia