• No results found

SOS! Our System was hacked. When Public Sector Organizations call in Management Consultants for Cyber Crisis Management.

N/A
N/A
Protected

Academic year: 2021

Share "SOS! Our System was hacked. When Public Sector Organizations call in Management Consultants for Cyber Crisis Management."

Copied!
116
0
0

Bezig met laden.... (Bekijk nu de volledige tekst)

Hele tekst

(1)

Page 1 of 116

Johanna-Emma Kosian | s1471988 | Intake: February 2015

Crisis and Security Management

MSC Dissertation

SOS! Our System was hacked.

When Public Sector Organizations call in Management Consultants for

Cyber Crisis Management.

Thesis Coordinator: Joery Matthys Second Reader: Ruth Prins

(2)

Acknowledgments

As my time as a student is coming to a close, I would like to take a moment to express my sincere gratitude to a few of you, who have done so much to support me in this endeavor. The hardest part of writing your master dissertation is not actually the writing - but the conceptualization of ideas that flourish into research questions, just waiting to be

answered. Especially the very beginning can be difficult. Whilst confusion is taking over and frustration slowly waves goodbye to your last shred of confidence, you struggle to come up with a topic that truly reflects your interests and aspirations. And just as the battle seems lost, before it even began, you are being comforted with the information that you are not alone; in fact, every master student is a bit confused in the beginning and may have some hurdles to overcome throughout the process.

Joery (Matthys), I thank you for your guidance in the past months, your expertise provided valuable angles to my research; your encouragement enabled me to discover the fun that can come with writing your dissertation. Dr. Joery Matthys is not only a lecturer and supervisor at the University of Leiden; he also is an indispensable mentor that sincerely invests in his students’ research and remains supportive along the way. Also, I would like express gratitude to the Faculty members of the ‘Crisis and Security Management’ Masters program, who have made the past year and a half a memorable experience. Ranging from the professors to lecturers, program coordinators and study advisors, the enthusiasm and

encouragement with which you choose to guide us is truly inspiring.

Above all, I thank my parents, who have always heartened me to follow my ambition and always remained optimistic. Your support knows no boundaries; your encouragement cannot be bound. How grateful I am to have you on my team.

In this spirit, I fully acknowledge that I could not have undergone this process without the fantastic support I was granted by all of you. I acknowledge, that this dissertation is a product, in which many of you have taken part in – and I remain grateful to you all.

(3)

Page 3 of 116

Abstract

The purpose of web 2.0 is no longer to only connect people; instead the cloud has become our very own storage unit – sheltering information of the state’s infrastructure, its people, and its financial data. An increasing dependency on the Internet of things, to provide government agencies and public sector organizations with citizen’s personal information to verify identity and assets has created a demand for cyber security – specifically, cyber crisis management. Traditionally, the public sector involves external providers (management consultants, freelancers, contractors) to drive modernization by remodeling the

organization’s IT structures. However, previous literature refers to a public sector that is often hindered or disinclined to hire external providers, identifying four main categories of factors that may influence a public sector organization’s contracting behavior: the

Economic, Political, Organizational, and Institutional Factor. Whilst scholars address governmental agencies general contracting behavior, though mainly focused on a UK/US narrative, it fails to consider the factors’ influence in service-specific instances. Interestingly, based on past literature’s insights, cyber crisis management – as an IT service, could be considered a reasonable service for the public sector to contract out.

Thus, what are the factors that would influence the public sector organization’s contracting behavior? And, how do they influence the public sector organization’s

contracting behavior? The answer is rather surprising. Indeed, public sector organizations are required to follow extensive competitive bidding processes (Economic Factor) and abide to bureaucratic processes (Institutional Factor), though the organizations (case 1, 2, 3) have modernized the process through an informal bidding process. Thus, whilst the organization is still required to submit formal proposals (bureaucratic processes) an increasing initiation of informal grooming can be observed, encouraged by political pressures and a lack of human resources. Although, there is no single universal factor that can be considered to have the most influence on public sector organization’s contracting behavior, the organizations (case 1, 2, 3) stressed the importance of business familiarity, confidentiality and objectivity. These concepts were coded as indicators for the “grey box” variable ‘Level of Professionalism’ which was added to the Organizational Factor – and listed as the variable with the most positive influence on public sector contracting behavior across all cyber crisis management stages.

(4)

Table of Contents

Cover page 1

Acknowledgements 2

Abstract 3

List of figures and tables 6

Part 1: Research Project

Chapter 1: Introduction 8 Topic 8 Research Problem 9 Academic Relevance 11 Social Relevance 12 Overview 13

Chapter 2: Theoretical Framework 14

Introduction 14

Conceptualization of Cyber crisis management 15

Conceptualization of Public Private Collaboration 18

Commodified Market Model

Contextual Factors: 24

Factor 1: Economic Context Factor 2: Political Context Factor 3: Operational Context Factor 4: Institutional Context

Chapter 3: Methodology 29

Research Design 29

Case Study 32

Data Gathering 34

(5)

Page 5 of 116 Part 2: The Analysis

Chapter 4: Case 1 40

Factor 1: Economic Context 40

Factor 2: Political Context 41

Factor 3: Organizational Context 43

Factor 4: Institutional Context 45

Summary 46

Chapter 5: Case 2 48

Factor 1: Economic Context 48

Factor 2: Political Context 49

Factor 3: Organizational Context 49

Factor 4: Institutional Context 51

Summary 52

Chapter 6: Case 3 53

Factor 1: Economic Context 53

Factor 2: Political Context 54

Factor 3: Organizational Context 55

Factor 4: Institutional Context 56

Summary 57

Chapter 7: Comparison 59

Factor 1: Economic Context 59

Factor 2: Political Context 60

Factor 3: Organizational Context 61

Factor 4: Institutional Context 62

Summary 63

Part 3: Concluding Remarks

Chapter 8: Conclusion 65

Conclusion 65

Reflexions

(6)

List of figures and tables

Figures

Figure 2.1 – NIST Cyber Model 16

Figure 2.2 – BSI Crisis Management Lifecycle 16

Figure 2.3 – Public Private Cooperation 20

Figure 2.4 – Amirkhanyan et al. “Stage one: contracting decision” 24 Figure 3.1 – Factors that influence contracting decision 30

Figure 3.2 – Competing case characteristics 33

Tables

Table 2.1 – NIST Cyber Model 17

Table 2.2 – BSI Crisis Management Lifecycle 17

Table 3.1 – Operationalization 36

Table 4.1 – Case 1 Summary 46

Table 5.1 – Case 2 Summary 52

Table 6.1 – Case 3 Summary 57

Table 7.1 – Comparison (positive/negative) factor influence 63 Table 8.1 – ‘Grey-Box’ variable, Operationalization addition 66

(7)

Page 7 of 116

Chapter 1: Introduction

The term “Consultocracy” 1, coined by Hood in 1991, and generally associated with the New Public Management, describes the thriving influence of consultancies across industries2. Despite its newly designated title, ‘New public Management’ was fairly a new concept in academia, as its theories were much discussed towards the late 1980s during which management-consulting spending increased exponentially3. Privatization reforms intended to drive modernization thus gave rise to management consultants in the public sector much earlier4, stimulated through a need for distinctive skill-sets and expertise that were lacking in public sector organizations. The employment of management consultant in the public sector can be traced back to as early as 1968, in which the Bank of England became subject to much controversy when employing McKinsey5. The motivation thus, for which public sector organizations chose to employ management consultant was, to modernize public services in countries such as the UK, started off in the 1970s. However, the rapid technological development brought about a turning point in the perceived value of management consultants/contractors6, shaping its reputation until this day.

The need for governments (and established private sector organizations) to adapt to a new era of Information technology (IT) further fostered the management consultant expenditure7. Bureaucratic structures and organization, accused of generating gradual decision-making processes, in need of new modes of efficiency and performance required an IT revolution. Scholarly contribution labeled management consultants as wizards in business, ‘transforming the public sector’8 as ‘agents of change’9. The notion of management consultants as ‘agents of change’ as such, though often required to execute assignments, which demand specific skill-sets, is a scarce one. Instead, mainstream academia resorts to discussing rather traditional economic indicators, naming cost-effectiveness as the major

1 (Saint Martin 1998)

2 (Lapsley and Oldfield 2001)

3 (Hood, The new public management in the 1980s: variations on a theme 1995) 4 (Lapsley and Oldfield 2001)

5 (Peet 1988)

6 (Lapsley and Oldfield 2001)

7 (Bloomfield and Danieli, The role of management consultants in the development of information technology,

the indissoluble nature of socio-political and technical skills 32)

8 (Lapsley and Oldfield 2001) 9 (Tisdall 1982)

(8)

incentive for a decision to contract out certain responsibilities in a public sector organization10.

The research of Amirkhanyan et al. seems somewhat a break-through, questioning the importance of monetary considerations and categorizing different factors, creating a contextual framework. The aim of the contextual framework is to identify a variety of sub-categories to four, pre-established, factors: the Economic, Political, Organizational, and Institutional context that may influence a government agency’s decision to contract out11. Indeed, the contracting behavior of a government agency may be influenced by an array of elements – and dependent on individual situational circumstances. As countries are constructed and shaped through individual, political, economic and societal circumstance, such a framework may vary from each country. Union reports and documentation acknowledge contrasting country-specific circumstances and thus advises to tailor frameworks in to the individual country’s 12 need.

Research Problem

As for the Netherlands, increasing privatization reform efforts sparked a scholarly debate on the responsibilities of public sector agencies and their need for a partnership with private sector organizations13. Such an approach has been used extensively in Public sector IT departments, to support existing organizational structures in adapting to the rapidly changing character of Cyber14. Consequently, the established theory of New Public Management is agreed to have evolved, as modern day management consulting abides to the needs of a flourishing IT sub culture15 – nurturing the needs of market expenditure. As privatization reforms in the Netherlands have become progressively matured, public-private partnerships have taken on multiple forms16 to drive IT modernization.

The concept of cyber crisis management (CCM) is a fairly new one, relying on many of the traditional crisis management components, whilst integrating IT structures and strategies to combat the modern-day cyber threat landscape. The start of the new millennium

10 (Amirkhanyan, Kim and Lambright 2007) 11 Ibid. 12 (ENISA 2013) 13 (Brenner 2007) 14 (Amoroso 2001) 15 (Peet 1988) 16 (Crawford 2006)

(9)

Page 9 of 116 has brought many economic and political changes upon our society. With web 2.017 we were forced to witness the rapid development of cybercrime and cyber-terrorism. The purpose of web 2.0 is no longer to only connect people, to exchange data; the cloud has become our very own storage unit – stashing information on a state’s infrastructure, its people, companies and financials. Criminal and terrorist organizations have embraced the facets of web 2.0 and quickly learned to exploit its domain18. The cyber domain has developed into a hub in which the most crucial stakeholders are states, conglomerates and large organizations. These Stakeholders often have the responsibility to protect large databases that may obtain vital private information of its citizens. Whilst the cyber domain is not as such a new domain in itself, public awareness skyrocketed due to the extensive media coverage on recent incidents such as SONY, SITA19, JP Morgan Chase, and Anthem20. Extensive media coverage raised serious questions on data protection, privacy and security, pressuring public and private sector to address the challenges that have become increasingly destructive in character in the rapidly growing domain of cyber. The bureaucratic nature of government agencies structure has become its liability, unable to keep up with the constant change in the dynamic cyber environment.

The era of E-government was quickly followed by the shocking realization that our once ‘private’ information is now visible to the world, as we leave large digital footprints, anyone, with the right skillset, could inspect21. The dynamic trans-border features of cybercrimes accentuate the need for Critical Information Infrastructure Protection (CIIP) procedures and strategies22, to safeguard data of citizens and sectors across borders - especially in international unions, such as Europe, in which public (and private) organizations are often interlinked23. In efforts to raise public sector and governmental agencies’ awareness multiple institutions and individual actors have initiated campaigns to promote Cyber Security and CCM24. At last, the Diginotar crisis in the Netherlands served as a wake-up-call for many of the governmental agencies who had resumed business as normal, ignorant to the

17 Web 2.0 – there is disagreement amongst scholars on the origin of the term ‘web 2.0’, it is undeniable though

that it was popularized during the O’Reilly Conference in 2004 and since then has been used as a term to describe the internet (world wide web) and its user-generated content and accessibility

18 (Provos, Rajab and Mavrommatis 2009) 19 (European Comission 2011)

20 (Quirk, Hollowood and Hampson 2016) 21 (Singer and Friedman n.d.)

22 (European Comission 2011) 23 Europe Cyber infrastructure

Note: i.e. Border control

(10)

changes IT modernization had brought about. Public Sector Organizations were made aware that CCM capabilities could have a vital impact on the outcome of an incident/crisis – often enabling the organization to exercise damage control and keep financial repercussions at bay. As public sector organizations have been reported to have little technical resources, it seems practical to use management-consultants and contractors to provide specific CCM capabilities.

Research Questions

Therefore, the purpose of this qualitative Master dissertation is to examine the context and conditions of the Commodified Market Model25 in the Netherlands, aiming to understand the impact a variety of factors may have on public sector organizations contracting behavior of CCM responsibilities.

Therefore the first research question (RQ 1) is as follows: RQ 1

The central research question is thus as follows:

“What are the factors that influence a public sector organization’s decision to contract out CCM responsibilities to external consulting/advisory organizations?”

Essentially, this dissertation aims to identify and understand the factors that influence a public sector organization’s contracting behavior, but also how (positive/negative influence) the factors may influence its contracting behavior in specific CCM stages.

And the second research question (RQ 2): RQ 2

Thus, the second research question is as follows:

“How do the factors influence the public sector organization’s contracting behavior in the specific CCM stages?”

25 (Matthys 2015)

(11)

Page 11 of 116

Academic Relevance

The field of New Public Management (NPM) has received much attention throughout the 1960s to 1990s, even though its theory was coined at a later time26, NPM was a popular topic to explore amongst scholars. As consultancy emerged as a sub-topic linked to the theories of NPM27, an increasing amount of scholars initiated research dedicated to the development, and the advantages and disadvantages of management consultants. Initially, scholars acknowledged the role that management consultants were to play in the commercialization of information technology28, with a primary focus on private sector organizations. Once privatization reforms, established to advance the pace of modernization29, were increasingly encouraging public sector agencies to converse with private industry organizations multiple scholars raised their concerns of the intentions of management consultants. Recent scholars question the “demonization” of management consultants30, suggesting the label of “would-be agents of change” limited only by the complexities and politics of the public sector31. Most importantly, diving into the literature that is discussing the development of management consultants in the public sector, consultants’ contribution to public sector IT modernization32 efforts are mentioned only sporadically. Failing to provide further skill-set or service specific details to provide insight and context.

Whilst previous scholarly contributions grant valid insight into the development and transition of management consultants33, fostered by privatization reforms and modernization attempts, it is missing contemporary components34. Lacking a critical perspective on recent IT trends, as a tool of modernization, that may threaten modern day privacy and security of Critical Information Infrastructure. The scholars Amirkhanyan, Kim, and Lambright, acknowledged their respective research limitations, encouraging further research into how contracting behavior may “vary across different services within the public sector”35. Thus, despite past scholarly contributions, which emphasize the vital role that management

26 (Hood, A public management for all seasons 1991) 27 (Saint Martin 1998)

28 (Peet 1988) 29 (Dunleavy 1986)

30 (Clark and Salaman 1996); (Peet 1988); (O'shea 1997) 31 (Lapsley and Oldfield 2001)

32 (Bloomfield and Danieli, 1995) 33 (Tisdall 1982)

34 (Lapsley and Oldfield 2001)

(12)

consultants have played in transforming the public sector36, academia is visibly lacking a 21st century perspective that may influence the current decision-making process on IT service specific contracting. It seems that due to the formation of economic IT hubs, initial focus on research relating to this field is often limiting research on the evolution of management-consultants to the USA and UK.

Therefore, as previous research has indeed focused vastly on the U.S. and UK public sector relationship to management consultants37, this dissertation aims to examine the Netherlands’ public sector cooperation with management consultants. Particularly, taking a closer look at the, through privatization enforced, partnership model and certain restrictions in the field of CCM, as an attempt to close the existing gap in current literature.

Social Relevance

Due to the fact that cyber security is becoming an increasingly debated topic, with the international community confirming the necessity of Critical Information Infrastructure Protection (CIIP) and privacy regulations to ensure citizens privacy, it is of great social importance. Specifically, recent reports identify the need of government (in this case, the Dutch government) to cooperate with private sector agencies to ensure an improving data sharing process (best practices). In theory, a voluntary Public-private partnership sounds promising and certainly worth supporting. However, in practice, public-private cooperation is often times managed through contracts, with the goal to modernize existing structures whilst maintaining control over information stored. In the era of the internet of things, the need for a rise in awareness on the definition of the prominent terms: big data and CIIP, becomes increasingly apparent as experts and academia still begin to fathom the limitless opportunities, but also the growing threats, the cyber domain has to offer. Hence, not only cyber security as a topic has great social significance, but also, public-private cooperation through contracting is of importance. Especially as scholars have mentioned the existing conflict of the different stakeholders (such as, private and public sector) on Internet governance, fueled by differences in agendas and expectations (see: Multi-stakeholder theory38). Presenting insight into the motivations of public sector organizations decision to contract consultants to support cyber crisis management responsibilities, and what those are,

36 (Ward 1993)

37 (Saint Martin 1998); (Lapsley and Oldfield 2001); (Wood 1996) 38 (De Nardis and Raymond 2013)

(13)

Page 13 of 116 may foster a better understanding of the internal procedures and organizational means of the public sector.

Overview

Essentially, the objective of this master dissertation to identify the key factors that guide public sectors organization’s decision to contract out certain cyber crisis management responsibilities in the Netherlands. Specifically, the aim of this research is to further analyze the distinctive capabilities that public sector organizations require support in to ensure cyber crisis management and resilience.

The structure will thus be as follows; heretofore, the Introduction to this master dissertation provided some insights into the motivation and scope of the research. Chapter 2 will start off with a conceptualization of cyber crisis management and management consulting, providing a basic definition and identifying differences to fields previously mentioned in academia. It will commence with drafting an overview of the partnership model associated with management consulting: the Commodified Market Model, discussed in the context of the IT revolution and the trending cyber security domain. Followed by an in-depth discussion on the factors that influence a public sector organization to contract out responsibilities, theorized by Amirkhanyan et al. (Economic, Political, Organizational, and Institutional Context). Chapter 3 demonstrates the methodology of this research, its research design process, research method, data gathering techniques, and, operationalization. Chapter 4, 5, 6, and 7 will comprise of the analysis, divided into individual chapter for each case.

(14)

Chapter 2: Theoretical Framework

Introduction

As mentioned previously, the practice of consulting, specifically of auditing and management, reached its momentum towards the 80s and early 90s. The legacy of the popularized “Consultocracy39” movement is much debated, with scholars crowning consultants the missionaries of the commercialization of information technology in private and public sector organizations40. The bureaucratic character of many public sector organizations had hindered its technological development, essentially keeping it from modernizing as a sector. Finally, privatization efforts, following the NPM era, enabled public sector organizations to hire management consultants to innovate internal structures – thus “saved” by the advocates of reform41. As consultancies had branched out, extending their expertise from auditing and tax consulting to strategy advisory, government agencies reliance on the IT environment grew exponentially. The public sector’s increasing reliance on IT services, stimulated by the need to modernize existing structures, exposed a lack in available human resources and a vast demand for trained staff.

The commercialization of the Internet consumed users worldwide, announcing a new digital era of information technology, and for many the first step towards a unified “global village”42. The realization that the anarchic character needs to be tamed and governed only dawned upon experts and government officials at a much later time43. But even then, the complexities of the threat landscape were not yet clear to many, sparking a controversial debate on the ownership and governance of the Internet44. The mutual consent was made that, as cyber is a trans-national phenomenon, it should be treated as such. Different stakeholders should be involved in the decision making on both a domestic, and international level45. Essentially, popularized through the urgency of an international coherent strategy to promote Cyber resilience, the Multiple-stakeholder Governance model was coined.

39 (Saint Martin 1998)

40 (Lapsley and Oldfield 2001) 41 Ibid.

42 (McLuhan 1964) 43 (Lobo 2015) 44 (Whitmore 2009)

(15)

Page 15 of 116

Conceptualization Cyber Crisis Management (CCM)

The cyber domain with its trans-national character facilitates content and platforms of expression46 to an exponentially growing audience with diverse interests; its growing realm empowers opportunities and threats alike47. With society’s increasing reliance on the web as a means to communication and the Internet of things as a large virtual information depot, wary citizens and organizations have raised questions on the autonomy of the cyber domain. Previous supporters of the anarchic structure of the Internet, that had praised it as the tool of choice for a global revolution of democracy and freedom, have since then drastically changed their perception48. The Internet is no longer considered the technological messiah of the people, the instrument of a democracy-oriented metamorphosis of our society. Instead, it has been crowned a new realm of rapidly transforming threat landscapes, a wake-up call for many organizations and governments, popularizing cyber security as the topic of the hour49. The demand for Cyber Security strategies and capabilities also stimulated a topic, still under much debate, the topic of cyber resilience.

As cyber attacks have become increasingly advanced - ranging from system penetration, to a (distributed) denial-of-service (DoS) attacks, to a system black-out - public and private sector organizations alike acknowledged the need for a cyber security strategy which addresses both cyber resilience and cyber crisis management50. Scholars seem to question governments’ ability to comprehend the key differences between cyber and physical conflicts, blaming the international community’s lack of understanding the tremendous scope and ambiguity of cyber attacks. Essentially doubting public sector organizations’ competence to respond to cyber incidents that escalate into a cyber crisis, further fueling public debate on the responsibilities of the state to protect its citizens data – considered a right to privacy51. Within the international community efforts have been made to raise awareness to the attributes and characteristics of advanced cyber attacks through noted advocates that contribute to international conferences or congresses52, but also through Unions53 – such as the EU – who publicly urge member states to conform to EU norms on Cyber Security.

46 (O'Brien 2014)

47 (Singer and Friedman n.d.) 48 (Lobo 2015)

49 (O'Brien 2014)

50 (Libicki 2012); Note: cyber crisis management is commonly considered a sub-category of cyber resilience 51 (Singer and Friedman n.d.)

52 (ENISA 2013)

(16)

Essentially pressing member states to invest in Cyber Resilience and Cyber Crisis Management, to ensure a coherent understanding54 of the cyber threat landscape. Current analysis (research, reports, and best-practices) of the cyber threat landscape and advice on strategy and structure is provided by the European Union Agency for Network and Information Security (ENISA), working together with individual member state’s agencies to establish national cyber response teams55. On a national level, former Government Computer Emergency Response Team (GOVCERT.NL) was developed into the National Cyber Security Center (NCSC)56. The NCSC is a constituent component of the Ministry of Security and Justice (Netherlands) with its primary focus being, to serve as a center of information and knowledge to support CIIP57. As neither the NCSC nor the ENISA offer a Cyber Crisis framework, the framework below (Table 2) proposes a visualization of a Crisis Management Framework tying into a Cyber Model.

NIST Cyber Model58 and the BSI Crisis Management Lifecycle59

60

54 (Libicki 2012)

55 (ENISA 2013)

56 (Van der Heuvel and Klein Baltink 2014) 57 (National Cyber Security Centrum 2015)

58 (National Institute of Standards and Technology 2014, p. 9) 59 (BSI Publication 2014, p.9)

60 (National Institute of Standards and Technology 2014, p. 9)

Figure 2.2. BSI Crisis Management Lifecycle Figure 2.1. NIST Cyber Model

(17)

Page 17 of 116 The Cyber Model was provided by the American National Institute of Standards and Technology with the aim to improve Critical Cybersecurity Infrastructure. Its Framework Core is divided into four parts: Functions, Categories, Subcategories, and, Informative References (For the complete Cyber Model, see Appendix). The Cyber Crisis Body adopts mainly the five Functions of the NIST Cyber Model: Identify (1), Protect (2), Detect (3), Respond (4), and, Identify (5)61. The purpose of these five Functions is, to provide an overview of the main Cyber security’s main activities by organizing and incorporating it into the Framework at their highest level. The Crisis Management lifecycle – Guidance and good practice by the BSI Standard Publication - ties into the NIST Response Function, with the intent to acknowledge both Cybersecurity incident elements, as well as specific Crisis Management components. As such the NIST Cybersecurity framework offers an overview of the strategy functions, whilst the BSI Crisis Management model provides a detailed Lifecycle guide which consists of five steps: Anticipate (1.1), Assess (1.2), Prepare (2), Respond (3), Recover (4), and, Review and Learn (5).

NIST Framework62

Function Category

Response Response Planning, Communications, Analysis, Mitigation

BSI Crisis Management Lifecycle63

Stage Capabilities

1.1.Anticipate Identify potential Crisis and other

disruptions over time horizons

1.2.Assess Analyze evidence and make judgments

about potential impact and actions required

2. Prepare Ensure the readiness of the organization to

face specific risks and handle crises that are not foreseen

3. Respond Act quickly in an informed manner and with

the desired effect when affected by a crisis

61 (National Institute of Standards and Technology 2014, p. 8-10) 62 Ibid, p. 19

(18)

4. Recover Sustain a crisis response into a longer term, strategically directed effort to recover reputation and value.

5. Review and Learn Analyze and reflect on the experience of

validations, testing and exercising, the management of crises and the experience of others in managing crises

Conceptualization of Public Private Collaboration

Towards the end of the 19th- and the beginning of the 20th Century, privatization reforms grew increasingly popular. Governments had acknowledged that in order to improve and strengthen certain services, such as security, which formerly had been exclusively operated by the Government, it would need to resort to alternative security providers. As privatization reforms encouraged private sector to take over responsibilities previously held by the state, it also allowed for government organizations to employ contractors to support missions or departments that lack human resources to provide expected security services to its citizens. Within the field of security, the government may therefore employ outside contractors or consultants for a variety of fields and disciplines, many serving the purpose of upholding infrastructure and national security.

Due to the increasing popularity of ‘E-governments’64 the majority of data, stored by the government, consists of the personal data of a citizen (financial, medical), collected through public sector agencies. Much of the services provided in these ‘E-government’ countries thus rely on the storing and sharing of citizen’s information, enabled through the internet of things, which serves as an important component of an ‘E-government’s’ infrastructure. The main tool of an ‘e-government’ is thus the cyber domain, as it enables the government to collect and store massive chunks of data, whilst ensuring that it will reach the correct point of destination when needed. In light of the popularized reliance on digital infrastructure, international Cyber Security organizations and agency increasingly advocated public-private collaboration as a solution to the dynamic and transnational character of the Cyber domain. As such, it would serve the purpose of distributing knowledge and

64 Short for electronic- or digital- government – which allows for the digitalized communication between Citizen

(19)

Page 19 of 116 information across industry and sector, connecting experts and building a community of sorts – both in public and private sector. The ENISA published multiple reports in which it highlights a need for such a collaboration of public and private, but insists it to stay on a voluntary basis – failing to elaborate on alternative privatization methods that could enforce public private coordination in Cyber Security.

Public Private Collaboration Models

So far three models of Public Private Coordination in Security have been identified: Private Junior Partner Model (1), Commodified Market Model (2), and, Responsibilized Network Model (3)65. The Private Junior Model (1) gives way to the privatization of certain bodies or responsibilities through privatization reforms. The Privatized organizations may further provide public organizations with information and support, the responsibilities divided between public and private sector are clear and limited, governed through policies and/or regulation66. The Commodified market model (2) considers security a good, a commodity that can be traded or bought. As such the private sector can be seen as an alternative provider for specific security services, pre-determined responsibilities established in binding

contracts67. The Responsibilized Network Model (3) essentially considers public-private partnership (PPP model) a necessity, encouraging both public and private sector to unite to provide security. Due to the often ad-hoc character of cooperation, the responsibilities (divided and shared) are somewhat muddy and unclear68. Interestingly enough the

Responsibilized Network Model has been gaining a lot of momentum in recent years due its advocates (ENISA, US Department of Homeland Security), who publicly announced the necessity of such a framework to ensure participation of variety of stakeholders69.

Technically, the PPP model is thus a sub-concept of the Multi-Stakeholder Model, which goes beyond encouraging the participation of public and private sector, favoring the involvement of users to represent the public.

65 (Matthys 2015)

66(Crawford 2006)

67(Krahman, Security: Collective Good or Commodity? 2008) 68 (Klijn and Teisman 2000)

(20)

Commodified Market Model:

Although, all three models are significant in the western world’s public private collaboration efforts, this dissertation is specifically interested in management consulting contracting, which is an element of the Public Private Collaboration model. The second model, the Commodified Market Model, has multiple sub-branches, which distinguish the concepts of “outsourcing” and (management) “consulting” from one another, whilst connected by a mutually important concept “contracting”.

Outsourcing vs. Management Consulting

IT consulting developed somewhat as a “third wave”70 in the consulting industry71, as it’s focus is more specialized than general management consulting (GMC). The IT consulting trend first appeared in the 1950s due to the commercialization of computer processing to increase organizational efficiency72. As the times progressed, due to privatization reforms pressured by technological development and modernization, GMC moved from solely working for/with private industry to welcoming public sector organizations as clients. The development of the Internet popularized IT Consultants and accelerated an already growing

70 According to Kipping (2002) IT consulting can be considered the “third wave” of management consulting,

whilst ‘efficiency experts’ can be considered the first, and ‘strategy consultants’ the third.

71 (Kipping and Clark 2012, pp. 293) 72 Ibid.

(21)

Page 21 of 116 line of work73. Additionally, the changing threat landscape – traditional crimes moving into Cyber Space – further highlighted the need of specialists equipped with the technical and industry-related knowledge to successfully advice clients. Thus, the modernization of business and government created the need for both, the maintenance and system operations side of IT, as well as specific strategies and structures derived from expert knowledge74.

The evolution of outsourcing dates as far back as the popular concept “time-sharing”, enabling several organizations to process data simultaneously – sharing the same mainframe computer. Time-sharing revolutionized the modernization process of many organizations as it provided economic advantages, benefiting from technological enhancement without heavy investment. Over the years then a definition of Outsourcing crystalized, highlighting the repetitive and maintenance work which increasingly has been considered the main characteristics of Outsourcing. Outsourcing thus contains the following external services: ‘applications development and maintenance, system operation, network/telecommunications management, end-user computing support, systems planning and management, and purchase of application software, but excludes business consulting services, after-sale vendor services, and the lease of telephone lines’75. The exponential growth of the outsourcing industry came with the commercialization of computer processors and the Internet. The booming outsourcing business gave rise to two emerging sub-categories: onshore and offshore outsourcing. Onshore outsourcing is understood to primarily include the relocation of certain work departments that are responsible for certain general tasks to a cost-efficient business provider in the same country of origin. Offshore outsourcing makes use of the same ideals as onshore outsourcing but aims to further drive operational cost and efficiency through the relocation of part of the business (or department) to foreign countries in which production costs or salaries are lower. Generally speaking outsourcing thus is aiming to the business need of organizations that require a large maintenance of IT services that are somewhat support and service oriented in nature76.

IT consulting on the other hand caters to very specific business needs, responsibilities and capabilities that attend to a very precise issue and thus require extensive knowledge and

73 (Lapsley and Oldfield 2001) 74 (Kipping and Clark 2012)

75 (Kipping and Clark 2012, pp. 209) 76(Whitaker, Mithas and Krishnan 2010)

(22)

experience from which derives an industry-specific solution77. In the 1960s rapid changes in market due to the rise of information technology78 enabled the big Accountancy firms to take ownership of a big part of cyber strategy and security related services79. IT consultancy quickly proved one of the most profitable consulting services, outpacing the growth rate of the GMC sector for multiple years, essentially succeeding over GMC in market size80. It is important to note that an important factor of the exponential growth of IT consultants in both the private and public sector was not only the constantly transforming opportunities of Information Technology, but also a constantly evolving threat landscape. Consultants, particularly IT, were considered pioneers in the field, equipped with the knowledge and vision to transform existing structures and replace them with modernized practices aimed to revolutionize the client’s organization. Management Consultants (Cyber), often celebrated as ‘Agents of change’81, merely provide organizations with a promise to challenge traditional processes, improving internal IT capabilities and thereby empowering modernization82.

Contracting

Privatization reforms empowered the state’s administrative bodies to employ service delivery agencies and to privatize organizations that previously had been owned and operated by the government83. To enforce this change in governmental perception of the benefits of the business service sector the notion of contracting become increasingly popular, becoming the primary tool of privatization84. As such it is explained to derive from two essential streams of thinking, contracting as a: way to decrease size and scope of government (1), tool to increase efficiency and operate with flexibility (2)85. Kelman provides a definition of contracting for government agencies by considering it “a business arrangement between a government agency and a private entity in which the private entity promises, in exchange for money, to deliver certain products or services on to the government agency or to others on the government’s behalf”86. As contracting was becoming an increasingly accepted and appreciated tool for government agencies, the rapid development of IT further boosted the

77 (Bloomfield and Danieli, The role of management consultants in the development of information technology,

the indissoluble nature of socio-political and technical skills 32)

78 (Singer and Friedman n.d.) 79 (Kipping and Clark 2012) 80 Ibid.

81 (Tisdall 1982)

82 (Lapsley and Oldfield 2001)

83 (Amirkhanyan, Kim and Lambright 2007) 84 (Kettl 1993)

85 (Clynch 1999)

(23)

Page 23 of 116 growth of “market-mediated work arrangements”87 in the business service sector creating demand for experts and seasonal employee support with significant IT experience. Essentially, management consulting and outsourcing can be considered sub-categories of contracting, as contracting seems to be the mechanism through which management consulting and outsourcing is enabled88.

Contextual Framework89: Factors that influence a public sector organization’s

decision to contract out

Through a thorough study of existing literature, discussing contracting behavior and decision-making process of public sector organizations, the following factors where established as playing a significant role in the decision of public sectors to employ management consultants for cyber crisis management purposes.

87 (Abraham and Taylor 1993)

88 (Kipping and Clark 2012, chapter 6-7) 89 (Amirkhanyan, Kim and Lambright 2007)

Government agency's decision

to contract out

Economic Context ConsiderationMonetary

Political Context Poltiical Pressures Anti-government Ideology Organizational Context Capacity to deliver in-house services Capacity to manage contracts Characteristics of Service Institutional

Context RegulationsRules and

(24)

Figure 4 visualizes the “Contracting Decision” theorized in the Article “Putting the Pieces Together A Comprehensive Framework for understanding the Decision to Contract Out and Contractor Performance” by Amirkhanyan, Kim and Lambright, whose study aims to understand the trends and analyzing the implications of government’s contracting behavior. The first stage of Amirkhanyan, Kim and Lambright’s framework elaborates on the government’s decision to contract out, identifying four main contexts: the economic context, the political context, the organizational context, and the institutional context. Whilst this dissertation maintains these four factors as the central category labels, extensive literature review suggests modifications and addition to the factor categories.

Factor 1: Economic Context

Generally the economic context is the most mentioned when discussing a public sector’s decision to contract out certain responsibilities. Traditionally, provider competition would essentially make the contracting process cheaper than individual in-house development; a

formal competitive bidding process90 would drive prices down allowing for a healthy market competition91. Sustaining a competitive bidding process has become one of the most integral components of public sector contracting as it aims to not only ensure cost-effectiveness, but also to ensure the government refrains from trading monopoly suppliers92. Essentially, government officials and politicians use the administrative policies to maintain financial oversight, urging public sector organization to choose the most economical proposal of the bidding process.

For private sector agencies contracting out was commonly an effective solution to avoid extra costs when high wage organizations make use of low market rates for certain administrative or low skill work93. Additionally, High rotation in labor needs94, tasks of seasonal, administrative, maintenance or expert knowledge nature, further promote the use of outside contractors as a cost-effective solution for certain responsibilities95. Employers may thus choose to contract out to cut wage and manage fluctuations in demand, reflecting current human resource requirements (fluctuating company needs). As follows, the notion of economies-of-scale, elaborated on by Katherine Abraham and Susan Taylor, implies that

90 (Amirkhanyan, Kim and Lambright 2007) 91 (Clynch 1999)

92 (Kolderie 1986)

93 (Abraham and Taylor 1993, pp. 3) 94 (Abraham and Taylor 1993, pp. 4) 95 (Abraham and Taylor 1993)

(25)

Page 25 of 116 contractors may also play a buffering role96 mirroring the size of the organization (and market size) and its correlation to the volume of contracting activity97.

Essentially, the competitive bidding process, low market rates and constant labor force rotation98 - due to fluctuating company needs, enable organizations that employ contractors to maintain cost effectiveness by relying on flexible work agreements. Summarizing previous research of public sector and private sector organization’s contracting behavior, we can therefore identify three sub-categories to the Economic Factors: competitive

bidding process, low market rates, and fluctuating company needs. Factor 2: Political Context

Along the economic context, the political context is likely to play a part in a public sector organization’s decision to contract out specific responsibilities to the business service industry. In particular, political pressures on a government agency or a public sector organization may influence initial contracting decision, but also the contracting selection process in itself. Public sector organizations may be urged by politicians or government officials to choose a particular provider, not for economical reasons, but rather for personal gain and individual agenda. Therefore, favoritism may have a large influence on contracting behavior, if the public sector organization is subject to such political pressures. The political context may thus also be related to monetary considerations (economic context) as external (or internal) pressure to reduce an organization’s costs may have a substantial impact on the decision-makers conclusion99.

Frederickson further raises the ethical implications that are often at play between government and contractors100, questioning the impartiality of officials that leads to a decision on whether to keep service in-house or contract-out (linked to favoritism). Ethical

Implications, though often closely interlinked with political pressures and past contracting experience (Organizational Factor), essentially is suspected to have a negative influence on

public sector contracting behavior. As, public sector organizations are expected to decide

96 (Amirkhanyan, Kim and Lambright 2007) 97 (Abraham and Taylor 1993, pp.2) 98 (Abraham and Taylor 1993, pp.4) 99 (Amirkhanyan, Kim and Lambright 2007) 100 (Clynch 1999)

(26)

with the best interest of the public at heart, concerns for government officials’ and politicians’ impartiality101 may facilitate ethical dilemmas.

The political pressures and ethical implications seem competing in nature, highlighting the different stances politicians and government officials may have with regards to public sector contracting behavior. The political factor thus resolves around individual agendas, personal gain, and guiding morals – interlinked closely with the competing Economic, Organizational, and Institutional factor. Essentially, the diplomacy of the political realm may be violating the healthy market competition102, as a lack of competition in the bidding process and public expectation may cause redundant contracting agreements103.

Factor 3: Organizational Context

A third factor that may influence a government’s decision to contract out a particular service may be linked to the organizational context. As infrastructures grow and modernization introduced the public sector to (information) technology it has become apparent that some public agencies are not equipped to develop or maintain certain capabilities104. This may be a result of: a lack of - Human Resources (1)105, Knowledge, Resources and Equipment (2)106, an internal service provision infrastructure (3); the organization’s capacity to manage

contracts (4) and its past contracting experiences (5)107.

First, the public sector organization’s lack of Human Resources can be explained by a highly competitive recruitment market and ambiguous job description. The recruiting of employees with expert knowledge, whilst often desired, may however be harder to accomplish as those individuals thrive on experience in multiple sectors and industries108. As the term “cyber security” has become the buzzword across industry, commercialized through private risk agencies, organizations are competing for the markets’ bright and talented experts109 in the field. Public sector organization are thus in direct competition with private agencies – in which private agencies often have an advantage of attracting prospective employees with high bonus systems and salaries. In addition to a limited pool of experts to

101(Amirkhanyan, Kim and Lambright 2007) 102 (Clynch 1999)

103 (Amirkhanyan, Kim and Lambright 2007) 104 (Kipping and Clark 2012)

105 (Abraham and Taylor 1993) 106 Ibid.

107 Ibid.

108 (Amirkhanyan, Kim and Lambright 2007) 109 (Abraham and Taylor 1993)

(27)

Page 27 of 116 meet recent cyber security demands, job description remain ambiguous further complicating hiring efforts.

Second, the development of (expert) Knowledge, Resources and Equipment to support and conducts critical IT-related projects, often would take the organization too long to develop and sustain in-house110. Also, due to financial dependency, public sector organizations may not receive sufficient budget to invest in supplies that do not offer an immediate or a definite return on investment.

Third, large public sector organizations amount to “slow and rigid bureaucracies that lack monitoring ability”111, unable to ensure quality and efficiency of in-house capabilities or contracted services. In fact, contracting serves as a tool to reduce such size and scope of the government112, relieving public sector officials and departments from their responsibility to closely manage strategy and processes. Internal service provision infrastructures are therefore vital not only to manage in-house capabilities, but also to ensure a transfer in knowledge gained through external contracting projects.

Fourth, closely linked to the necessity of an internal service provision infrastructures lies the need for the capacity to manage contracts. Thus, public sector organizations do not only require an internal infrastructure that ensures the successful transfer of external contractor’s knowledge and expertise, but also must be able to successfully select and manage contracting bids and proposals in advance to the project113.

At last, past contracting experiences may leave a lasting impression on the public sector organization and influence the organization’s contracting behavior in a positive or negative way. Also, positive past contracting experiences may further enforce an informal competitive bidding process and favoritism – positive projects in the past may strengthen professional connections.

Nevertheless, literature closely examining contracting and procurement benefits have voiced concerns over the loss of expert knowledge to privatization reforms, in turn affecting the public sector agencies ability to monitor contractor progress114. As such, the organizational context that may influence a public sector organization to contract out is

110 (Amirkhanyan, Kim and Lambright 2007, pp. 706) 111 Ibid.

112 (Clynch 1999)

113 (Amirkhanyan, Kim and Lambright 2007) 114 (Amirkhanyan, Kim and Lambright 2007)

(28)

heavily debated, indicating a lack of oversight and accountability on one side, and an increase in efficiency and resilience on the other.

Factor 4: Institutional Context

Finally, the Institutional context may influence a public sector organization’s contracting behavior through domestic/national laws, institutional regulations or bureaucratic processes. Traditionally, public sector organization’s are required to abide to domestic/national

laws that may restrict public-private collaborations or enforce specific privacy agreements115 that may affect the organization’s contracting behavior. Domestic laws or regulations may range from serving as guidelines to rules, aiming to manage contracting agreements to preserve financial and/or organizational oversight.

Additionally, as previously mentioned, public sector organizations often struggle with the inflexibility that bureaucratic and hierarchical structures contribute to an organization116. Inflexibility can be caused by uptight institutional regulations (could also be

domestic/national laws), which demand certain procedures and processes, imposing very

specific structures on the organization. Such institutional regulations may be explicit or

implicit in character, either openly (explicitly) promoted through code of conduct rulebooks

or implicitly preached at team meetings and guest lectures.

Most importantly, organizations’ bureaucratic processes may hinder contracting agreements, as internal administrative procedures are time-intensive, require adequate planning and expectation management. Formal requests need to be filed prior to competitive bidding process and demand sufficient financial and operational elaboration, as well as in-depth analysis to prove a lack of internal capacity for the required project117.

An agencies institutional setting is therefore often said to play a large role in the decision it makes118 as its rules and regulations are often linked to additional contexts - such as the economic or political119 – often limiting a public sector organization’s ability to make such decisions on their own accord.

115 (Crawford 2006)

116 (Boyne 1998)

117 (Amirkhanyan, Kim and Lambright 2007) 118 (Peters 2002)

(29)

Page 29 of 116

Chapter 3: Methodology

The following chapter will shed light on the research design and method, explaining in detail why test cases were selected accordingly and how the researcher chose to later report and analyze the cases to conclude and recommend her final remarks.

Research Design

An organization’s decision to contract out to management consultants can indeed be explained by the multiple factors elaborated in the previous chapter, the theoretical framework. Specifically, the factors that influence an organization that operates in the public sector – responsible for certain infrastructure capabilities – were highlighted, supported by existing literature and research. Particularly, drawing on the ‘Contracting Framework’ of Amirkhanyan et al. that identifies four main factors to influence public sector’s contracting behavior. Whilst previous literature provided the bedrock of this research, it lacks an analysis of service-specific contracting motivation, failing to consider the important role of Information Technology as part of our infrastructure. The rapid increase in Big Data and Critical Information Infrastructure subsequently sparked the heated debate on cyber security and protection of public sector agencies that hold individual information – ranging from a financial, medical, or infrastructural nature. Thus the question crystalized, how specific factors influence a public sector’s organization to contract out specific CCM responsibilities and why certain factors may influence their decision more/less than others. The first Research Question is thus as follows:

“What are the factors that influence a public sector organization’s decision to contract out responsibilities to respond to cyber security threats (response phase) to external consulting/advisory organizations?”

Therefore, RQ1 aims to analyze existing factors that may influence a public sector organization to contract out responsibilities – in this case, Cyber Crisis Management responsibilities – to external consulting/advisory agencies. An extensive investigation of existing literature provided a relevant framework ‘Framework for Contract Decision and Performance’, which identifies four key elements: Economic context, Political Context, Organizational Context, and Institutional Context.

(30)

Whilst preceding literature acknowledges the need for further research into service-specific contracting behavior, such as the responsibilities of CCM, it also highlights the importance of understanding ‘how’ factors may influence public sector organization’s contracting behavior. Recommending further research to be conducted into the influence (positive/ negative) that factors may have on public sector organization’s contracting behavior of CCM projects. Therefore, the second Research Question is as follows:

“How do the factors influence the public sector organization’s contracting behavior in the specific Cyber Crisis Management stages?”

For the purpose of this particular research and its focus on Cyber Crisis Management, additional variables were added to existing sub-categories of the individual theoretical concepts as elaborated on by Amirkhanyan et al120.

All four factors were tested in detail through pre-determined and at a later time added, indicators which served to identify and propose the presence of specific motivations and objections. The Graph (3.1.) can therefore be explained as follows, the four factors serve as the foundation of the theoretical concepts which categorically list the Independent variables – Economic factor: competitive bidding process, low market rates, fluctuating company needs; Political factor: political pressure, ethical implications; Organizational Factor: Human

120 (Amirkhanyan, Kim and Lambright 2007)

Figure 3.1. The Individual factor’s variables categorically organize the different indicators,

(31)

Page 31 of 116 resources, Knowledge, Resources & Equipment, Internal Infrastructure, Capacity to manage contracts, past contracting experience; Institutional factor: domestic/national laws, institutional regulations, bureaucratic processes – that may influence the dependent variable – a public sector organization’s decision to contract out.

In addition to identifying and exploring the four factors, the theoretical framework conceptualizes the notion of ‘Cyber Crisis Management’, combining the established BSI Crisis Management standard with the NIST Cyber model. Both documentations have received positive appraisal across industry, commonly used by management consultants and experts of the business. The BSI Crisis Management standard offers a clear overview of the different crisis management (CM) stages in theory, whilst providing practical information and best practice for private and public organizations alike121. Similarly, the NIST Cyber model gained much attention throughout the industry, on the one hand, praised for its concise summary of industry-leading cyber security practices122; on the other, criticized for its reliance on mainly American Critical Information Infrastructure Protection (CIIP) guides. Essentially, the BSI CM standard and the NIST cyber model provide comprehensive working definitions of their respective concepts and offer practical insights through theorized frameworks. Merging both standards provides this dissertation with a construct to track the public sector organization’s contracting behavior across the crisis/incident’s123 stages and explores the various cyber security measurements accordingly.

As the goal of the research was to analyze and gather in-depth information on the behavior and motivations of public sector organizations decision to contract out, the research adheres to a qualitative case study. Specifically, by analyzing ‘what factors ’ and ‘how the factors ’ influence public sector contracting behavior, the research is taking a theoretical and explanatory approach (explorative research). Traditionally, a qualitative research allows for an in-depth understanding of the unit of analysis’ – public sector organizations – objectives and motivations. As this Dissertation is testing and adding to an existing framework, applying it the specific field of CCM, it is taking on a deductive and a general inductive approach. Traditionally the deductive approach, informally known as “top-down”124, aims to test and validate existing theories through generalizable observation and therefore tends to be

121 (Cockram, Larkin and Sheves 2015) 122 (PWC 2014)

123 See Appendix, Table A.1 (differentiation Crisis/Incident) 124 (Social Research Methods 2006)

(32)

associated with quantitative research125. However, this research combined both the deductive approach (theory testing) and inductive, informally “bottom-up”126, approach (theory generating) to come to a noteworthy conclusion within the particular field of CCM.

Case study

The previous sub-chapter, the Research Design, briefly touched upon the qualitative nature of the study and its aim to combine testing existing notions of a public sector organization’s contracting behavior/motivation and to generate new insights through in-depth analysis of the findings. Therefore the research took on the form of a multiple case study, which yielded explanatory insights as it seeks to understand the structure and process of the existing phenomenon of contracting (specifically CCM contracting). Due to the research’s goal to both evaluate existing theory and adjoin “black box” variables to the existing contextual factors the nature of research is both empirical and explanatory, and may as such be considered an extended case method.

Case study

“The in-depth examination of a single instance of some social phenomenon” 127

Extended case method

“A technique developed by Michael Burawoy in which case study observations are used to discover flaws in and to improve existing social theories.” 128

The research was conducted as a qualitative multiple case study to empirically gather detailed information and insights on organizations operating in the public sector, which later could be analyzed and compare to draw a compelling conclusion. The multiple case study approach, as a research method thus will be used to analyze the perception of a variety of factors that organizations have observed to be linked to the contracting behavior of their organization in CCM and contribute knowledge to existing social theories129 on contracting behavior. The

125 (Babbie 2008)

126 (Social Research Methods 2006) 127 (Babbie 2008, pp. 326)

128 (Babbie 2008, pp.326) 129 (Yin 1994)

Referenties

GERELATEERDE DOCUMENTEN

Die vrae uit die onderhoudskedule wat in hierdie tema gebruik word, handel oor die volgende sake: Wat die hoofde voel die grootste rol gespeel het in hul voorbereiding as

The data is analyzed and structured by categorizing the control elements mentioned by the interviewees. The possible categories are: result controls, action controls,

The stubbornly high unemployment rates, the increasing international competitive pressure from South East Asia resulting from globalisation, a loss of competitiveness

Voorafgaand aan het onderzoek was de verwachting dat een significante samenhang zou bestaan tussen de mate van modelgetrouw werken en behandelduur enerzijds én

[r]

In the third section a new two-stage ordinary differential equation model that considers the evolution of carbon, sugar, nutrients and algae is presented.. Careful estimates for

Chapter 7 reports on the results of a comparative study on cell seeding and cell culturing using porous PDLLA scaffolds with either a well-defined, open gyroid pore

H1: Daar bestaan statisties beduidende verskille in die tellings behaal op die volledigheid van die komponente wat ingesluit moet word in talentbestuursprogramme in die