Big Data and Administrative Justice: An analysis of social workers’ big data use and understanding and the perceived impact on treating similar cases similarly and dissimilar cases differently

Big Data and Administrative Justice

An analysis of social workers’ big data use and understanding and the perceived impact on treating similar cases similarly and dissimilar cases differently

Jelle van der Wal 1654713 Master Thesis Public Administration Economics & Governance track Faculty of Governance and Global Affairs

Leiden University

Supervisor: Dr. Nadine J. Raaphorst 7-1-2021

2

F

The master’s thesis hereby presented to you is part of the degree Public Administration: Economics and Governance. I would like to take the opportunity here to thank a number of people for their contributions to this work and process.

First of all, I would like to thank my supervisor Dr. Nadine J. Raaphorst for her guidance throughout the entire thinking and writing process. For her sharp insights, accurate and useful feedback as well as close attention, always with inexhaustive energy, enthusiasm, interest and commitment. I have learned much and experienced a very pleasant cooperation. In addition, I would like to thank everyone who collaborated through interviews, the provision of information and sharing of contacts, due to which I have been able to complete the research, receiving interesting new insights and from whom I have learned very much in terms of content and beyond. You made this experience and research unique. Finally, I am grateful to my family and friends who have always supported and encouraged me.

3

I

List of Abbreviations 5

Introduction 6

CHAPTER 1

REVIEW AND JUSTICIATIONeview and Justification 10

1.1 Literature Review 10

1.2 Scientific Relevance 14

1.3 Societal Relevance 15

CHAPTER 2

FROM THE IRON CAGE TO AGENTS ARMED WITH DATA:

THEORETICAL AND CONCEPTUAL APPROACHES 18

2.1 The dilemma street-level bureaucrats face 18

2.1.1 Street-level bureaucrats 18

2.1.1.1 From top-down to bottom-up: discretion 18 2.1.2 Screen-level bureaucrats and digital discretion 19 2.2 A solution or a challenge? A new policy space 21

2.2.1 Big data use 21

2.2.2 Algorithms 23

2.2.3 Evidence-based policymaking 24

2.3 Administrative justice 26

2.3.1 Normative judgments 26

2.3.1.1 Treating similar cases alike 26 2.3.1.2 Treating dissimilar cases differently 28 CHAPTER 3 METHODOLOGY 31 3.1 Research design 31 3.1.1 Inductive research 31 3.1.2 Qualitative research 31 2.1.3 Case study 32 3.2 Research methods 33 3.2.1 Interviews 33 3.2.2 Operationalization 35 3.3 Data analysis 36 3.3.1 Thomas theorem 36

4

3.4 Reliability and validity 40

CHAPTER 4

ANALYSIS 42

4.1 Use of big data: Background 42

4.2 Discretion 45

4.3 Understanding of data 50

4.3.1 Perceived neutrality 50

4.4 Justice 54

4.4.1 Perceptions of just big data use 54

4.4.2 Treating similar cases similarly

and treating dissimilar cases differently 59

Conclusions 64

Limitations and future research 67

5 List of Abbreviations

BRP Basisregistratie Personen

BSN Burgerservicenummer

GDPR General Data Protection Regulation

IB Inlichtingenbureau

6

I

The infamous slogan ‘Big Brother is watching you’ from George Orwell’s dystopian novel Nineteen Eighty-Four predicted a future of totalitarianism and human repression through technological and political control (1949). While surely society was not as dehumanizing in the year 1984 as envisioned by the novel, more recent advancements in data collection and use as well as broader technologies enable the potential formation of such a society more than ever before (Power, 2016). Big data, a term referring to data so voluminous, varied and complex that they go beyond human comprehension, seem to provide great opportunities technically, economically and for Big Brother to watch over you (Klievink et al., 2017; Giest, 2017; Jiang and Fu, 2018). Currently, China is oftentimes mentioned as an example (Jiang and Fu, 2018).

Even when a government is far from totalitarian, a fully technocratic discourse around big data use overlooks important political and ethical challenges (Jiang and Fu, 2018). Therefore, it is necessary to think carefully about the way one would like big data to be used in relation to the type of society one would like to live in and the type of treatment one expects from government. Street-level bureaucrats who, through their discretion, have significant leeway in decision-making processes and accordingly, have significant freedom to use big data in the way they see fit to treat citizen-clients, form a particularly interesting case (Lipsky, 1980). Street-level bureaucrats, including for instance teachers, police officers and social workers, closely encounter with citizen-clients on a day-to-day basis (Ibid.). As such, they constantly make moral judgments on how to treat citizen-clients (Zacka, 2017). More specifically, the renewed context characterized by an increase in the use of big data by street-level bureaucrats requires them to constantly make moral decisions when selecting or filtering data, having

7 important implications for social justice concerns. In this regard, it is unclear what just use of data actually entails with reference to the interaction between street-level bureaucrats and citizen-clients. Two notions of administrative justice are ubiquitously studied regarding street-level bureaucrats; treating similar cases similarly and treating dissimilar cases differently, respectively (Raaphorst, 2020). An example of the former is where two people steal a bread and receive the same criminal conviction for committing the same crime, implying consistency in treatment. While an example of the latter could be that two elderly persons receive help in the household, but one receives more help since this person is blind, less mobile and confused, whereas the other is ‘merely’ deaf and older, implying that treatment is unequal and responsive to particular circumstances or situations.

The notions of treating similar cases alike or similarly and of treating different cases differently, was first outlined by Aristotle, whose justice formula stated that: “like cases must be treated alike, and unlike cases unalike, proportionate to the differences between them” (Pobjoy, 2010: 184). This suggests that equality of treatment is morally just in principle, but treating different cases differently may also be fair (Ibid.). Little is known, however, about the impact of big data use by street-level bureaucrats on these two notions of justice. There is no consensus about, nor solid proof of whether big data have a positive impact or negative impact on either of the two notions of justice. Current accounts often focus on the relationship between big data and discretion, linking the former or latter directly to ethical questions (Buffat, 2013; Busch and Henriksen, 2018; Maynard-Moody and Musheno, 2012; Mittelstadt et al., 2016; Floridi and Taddeo, 2016). Theories regarding the relationship between the use of big data and the notions of administrative justice – treating similar cases similarly and treating different cases differently - are lacking (although a number of studies regarding the impact of

8 digitalization on discrimination do exist), inviting for inductive research to fill this gap and contribute to theory-building. In particular, this work focuses on the use of big data by street-level bureaucrats (social workers) and the impact this may have on their perceived ability to apply either of the two notions of justice in their daily work.

This qualitative research resolves around the central question: How do social workers’ big data use and understanding affect their perceived ability to treat dissimilar cases differently and similar cases similarly? It is attempted to answer this question by conducting open interviews with social workers to learn more about and analyse their experiences with the use of big data in their decision-making processes focusing on the notions of justice - treating similar cases similarly and/or treating dissimilar cases differently – they prioritise when using big data. Open interviews are useful since the variables on which this study is based involve perceptions and normative ideas and because deductive conceptual models are currently unavailable. In other words, data or information regarding street-level bureaucrats' perceptions of big data use in relation to notions of justice are absent and it is attempted to bridge this knowledge gap using an inductive approach to acquire a more thorough understanding of such perceptions.

In the first chapter the research is justified. It does so, by providing a brief literature review describing the scientific puzzle. The scientific relevance flows from this review and puzzle. Finally, the societal relevance of the research is illustrated.

The second chapter presents the concept of street-level bureaucrats and outlines the moral dilemmas they constantly face when treating citizen-clients and how these dilemmas change within the new context of increased use of big data. The chapter proceeds with the delineation of the concepts of big data, algorithms and evidence-based policymaking, which can be considered both a challenge and solution for the moral

9 dilemmas faced by the social workers outlined at the beginning of the chapter. The chapter ends by demarcating the two notions of administrative justice: treating similar cases similarly and treating dissimilar cases differently.

The third chapter expounds the methodology of the research by first presenting the research design and case study: social workers in the Netherlands. Secondly, the research method section outlines the interview strategy and operationalization. Thirdly, the way of analysing the results is explained by describing the inductive thematic analysis method. The chapter ends with an account of the reliability and validity of the research.

Successively, in the fourth chapter, the analysis of the interviews is provided, focusing on social workers’ data use and the impact this has on their notions of justice in their stories and experiences. Finally, limitations of the research are delineated, conclusions are drawn and recommendations for further research are provided.

10

C

1

Review and Justification

Introduction

To get a more complete idea about the relevance of this study a literature review is provided as well as a description of the scientific and societal relevance flowing from this literature review. The literature review focuses on where there is agreement or disagreement in the existing literature surrounding big data and related concepts and on what is already known and written about these themes, particularly concerning the relationship with discretion. In addition, the literature review provides an overview of the street-level bureaucrats’ literature within the context of digitalization to present the background for the dilemmas faced by such bureaucrats regarding the notions of justice: treating similar cases alike and dissimilar cases differently. Altogether, this contributes to the identification of a research puzzle and gap in the existing literature.

1.1 Literature review

Considerable attention has been paid to the use of big data both in the public and private spheres (Klievink et al., 2017). General consensus exists in the literature regarding the great potential data has to improve work processes, for instance in terms of efficiency (Busch and Henriksen, 2018; Head, 2008; Klievink et al., 2017). The development of the use of big data is also seen as inevitable, in the words of Giest: “The big data movement, however, has moved past the question of ‘if’ and is much more about the ‘how’” (2017: 379). With the perseverance of the movement towards seemingly evermore detailed, advanced, extensive and ubiquitous digitalization and data (hence the term ‘big data’), critical voices have been raised regarding issues of data security, responsibility,

11 transparency, privacy and ethics more generally (Mittelstadt et al., 2016; Floridi and Taddeo, 2016; Andrews, 2018). The latter perspective attends broad normative questions surrounding morality and ethics, referring to a combination of commitment and what is ‘right’ or ‘wrong’, ‘good’ or ‘bad’ when it comes to data and their use and analysis (Busch and Henriksen, 2018; Floridi and Taddeo, 2016).

Street-level bureaucrats constantly face conflicting values (notions of justice) as they closely interact with citizen-clients within their discretionary space (Lipsky, 1980). The increased use and in particular the way data is used, thus present both challenges and opportunities when it comes to these conflicting values, since big data present a new context and can be used to focus on certain values or notions of justice, but may come at the cost of others (Mittelstadt et al., 2016; Floridi and Taddeo, 2016). Some argue that street-level bureaucrats’ discretionary space will eventually disappear altogether, at least in some areas (Barth and Arnold, 1999). This would flow from the capacity of big data in combination with more advanced algorithms and artificial intelligence to replace human agency and the need for human judgments through ‘machine learning’, a term referring to the ability of techniques to recognise and adapt to environmental changes and generate knowledge, patterns and models enabling fully autonomous decision-making, free of human interference (Andrews, 2018; Mittelstadt et al., 2016; Floridi and Taddeo, 2016; Tufekci, 2015). Such ideas of replacing street-level bureaucrats’ discretion have contributed to the supposition that (big) data provide impartial, objective means to replace street-level bureaucrats, replacing human judgments hence human discretion (Busch and Henriksen, 2018).

A more nuanced stream of thinking argues that digitalization and the emergence of big data offer seemingly objective ‘evidence-based’ tools to street-level bureaucrats to

12 guide their decision-making processes, rather than fully replacing these processes (Giest, 2017; Busch and Henriksen, 2018; Buffat, 2013). As such, the distance between street-level bureaucrats and citizen-clients seems to increase and, correspondingly, discretion, which some have now termed ‘digital discretion’, would have been limited or ‘curtailed’ as street-level bureaucrats have become screen-level bureaucrats and in certain cases system-level bureaucrats (Bovens and Zouridis, 2002; Busch and Henriksen, 2018; Buffat, 2013). The use of big data is therefore not only considered efficient, but also, and importantly, fair from a social justice point of view as it would lead to equal treatment of citizens in public policy implementation (Cárdenas and Ramírez de la Cruz, 2017). In other words, big data use would contribute to the notion of treating similar cases similarly. A related yet different notion of justice – treating dissimilar cases differently – appears to be more adequately validated through discretion, particularly with respect to responsive treatment (Bargaric, 2000; Raaphorst, 2020).

While the replacement of human judgment is presented as rather utopian by some academics, as it would solve arbitrariness in decision-making, human flaws and the conflicting values bureaucrats face (Barth and Arnold, 1999), others see such developments as dystopian or are more sceptical and point at some ethical problems related to the use of data (Andrews, 2018; Boyd and Crawford, 2012). One of the main concerns is the inscrutability of data, hence the inscrutability of decision-making based on data (Veale et al., 2018). This may be particularly problematic for street-level bureaucrats’ responsiveness (i.e. to take account of specific circumstances and needs of citizen-clients, which are only sensible through human emotions and expressions currently beyond the capacity of data and algorithms). Indeed, the concept of responsiveness is closely linked to the notion of treating different cases differently (Raaphorst, 2020). Another problem concerns biases and human convictions integral to

13 big data and algorithms, since such biases are replicated and reinforced through machine learning (Andrews, 2018). This is particularly worrisome since such data-driven processes and policymaking move beyond human understanding as their complexity increases (Pasquale, 2015). Andrews refers to this problem as algorithmic unknowns (2018). The complexity of information beyond human capacity may go as far as to lead to large-scale misinformation or even Orwellian dystopia characterised by the use of data for undemocratic, totalitarian purposes as outlined in the introduction (Ibid.; Orwell, 1949). Such ethical dilemmas associated with the replacement of human input in decision-making processes by technological means call for regulation and, indeed, discretion to be at least partly upheld, in order to satisfy multiple notions of administrative justice simultaneously (Pasquale, 2015; Mittelstadt et al., 2016; Boyd and Crawford, 2012).

In conclusion, research that focuses on street-level bureaucrats’ big data use is ubiquitously concerned with the impact big data use has on discretion (e.g. Bovens and Zouridis, 2002). One camp of researchers argues that big data would negatively affect the amount of discretion of street-level bureaucrats, although this camp is divided regarding the extent to which discretion is reduced (Buffat, 2013). Buffat refers to this line of thought as the ‘curtailment thesis’ (Ibid.) Others argue that big data may provide street-level bureaucrats with additional resources and that discretion may be diffusing, which Buffat refers to as the ‘enablement thesis’ (Ibid.; Busch and Henriksen, 2018). What is striking, however, is that most authors seem to assume that a reduction of street-level bureaucrats’ discretion leads to more just decision-making, at least in the sense of more equal decision-making (e.g. Bovens and Zouridis, 2002). Perhaps even more surprisingly, little attention in the street-level bureaucracy literature is paid to the micro-level impact big data use has on two notions of justice – treating similar cases similarly and treating

14 different cases differently - despite their eminence in this field of research (Raaphorst, 2020).

1.2 Scientific relevance

Asserting that street-level bureaucrats still have fair levels of discretion in their decision-making processes (Lipsky, 1980; Bartels, 2013), it is argued that the decisions made are affected rather than replaced by big data availability (Busch and Henriksen, 2018). However, no theories or conceptual models are readily available to explain whether such affects are negative or positive in certain contexts. As aforementioned, the impact of big data use by street-level bureaucrats on their perceived ability to treat similar cases similarly and dissimilar cases differently is underexposed. Moreover, this relationship is largely unknown. Therefore, inductive research is required to learn more about this relationship and contribute to the eventual construction of theory. The formulation and construction of a fully grounded theory are beyond the scope of this work. Still, this research can add to the existing research that underexposes the relationship between big data use and notions of justice at the micro-level of street-level decision-making. It does so by acquiring data through open interviews, which, once analysed, may contribute to the broader project of theory-building necessary due to the current lack thereof.

The changes in the use of data are driven by changes in the behaviour of those at the forefront of policymaking; street-level bureaucrats, whose acts emerge “from a dynamic interaction of external circumstances and internal motives or interests” (Markus and Robey, 1988: 585; Busch and Henriksen, 2018). A more holistic understanding of the intentions or motives of street-level bureaucrats or social workers to use their available data tools provides insights into their perceived ability to apply different notions of justice in their decision-making processes. Such insights are currently underexposed in the

15 existing literature (Hawk and Dabney, 2014). In addition, the use of data may have important implications for the extent to which decision-making is fair in practice. In other words, to gain insight into the perceptions of the just use of data by street-level bureaucrats (social workers) is an important first step to more broadly comprehend the execution of policy through the use of data by those street-level bureaucrats, besides the contribution to theory-building. That is to say, by understanding the way street-level bureaucrats conceive of big data use as morally ‘right’ or ‘wrong’, in accordance with the notions of justice applied, depending on the data and the context, one may eventually understand street-level bureaucrats data-driven decisions, their actual use of data when making policy. Since such research or valid theories are currently unavailable an inductive, exploratory approach is appropriate.

1.3 Societal relevance

Street-level bureaucrats have significant leeway to make judgments (Lipsky, 1980). Through their judgments made within the discretionary space they have, street-level bureaucrats have a direct impact on the lives of people in day-to-day situations, such as in the classroom, at doctor’s consultations, when being stopped by the police or being inspected by tax officials, as they exercise their discretion (Kelly, 1994). However, the way they treat citizen-clients in these encounters depends on and are in accordance with their conceptions of justice and their judgments of what justice requires in specific circumstances (Ibid.; Maynard-Moody and Musheno, 2009). Individuals thus make decisions based on the notions of justice they apply and which outweigh others in their perceptions, hence street-level bureaucrats are continuously confronted with conflicting values that impact the way services and public goods are distributed among citizen-clients. One of such conflicting values involves the choice of treating similar cases

16 similarly and/or treating dissimilar cases differently. This dilemma directly links to themes such as equity, discrimination, responsiveness, accountability and distributive justice widely discussed and highly relevant for the type of society one lives in. While theories of justice and corresponding principles are omnipresent in academic debates, empirical findings on the notions of justice held by street-level bureaucrats are less abound, despite the great direct impact this has on citizen-clients’ treatment (Kelly, 1994).

These notions of administrative justice and the different weights attributed to them by street-level bureaucrats influence policymaking practices and therefore people’s lives. The room for street-level bureaucrats to do this, their discretion, can be increased or decreased through regulation (Pasquale, 2015; Mittelstadt et al., 2016; Boyd and Crawford, 2012). Moreover, the use of big data as policy tools change the nature and scope of discretion, thereby directly affecting policymaking practices and the way ethical dilemmas are managed, hence citizen-clients (Floridi and Tadeo, 2016; Mittelstadt et al., 2016; Wesselink et al., 2014). Whether and the extent to which the use of big data replaces human judgments, affects the discretionary room of street-level bureaucrats and will become decisive in ethical dilemmas depends largely on the acceptance (as well as the quality and extensiveness) of big data as morally suitable by society at large, governments but also and significantly, by street-level bureaucrats. They have the discretionary space within which they can determine whether to make judgments based on their own expertise or, and this seems to be increasingly the case, based on and guided by big data and algorithms (Giest, 2017). The decision, then, when and how to use big data depends on the way street-level bureaucrats conceive of the use of data and the extent to which such perceptions match with their conceptions of justice in a given situation. The perceptual frames of the use of data by street-level bureaucrats may, through their discretionary space of their use, significantly impact many people’s lives. This research

17 attempts to contribute to our understanding of how street-level bureaucrats data use and understanding affect their perceived ability to treat dissimilar cases differently and similar cases similarly, which may have great implications for just outcomes.

18

C

2

From the iron cage to agents armed with data: theoretical and conceptual approaches

2.1 The dilemma street-level bureaucrats face

2.1.1 Street-level bureaucrats

Michael Lipsky’s work shed new light on policy-making in practice, focusing on the encounters, often face-to-face, between citizen-agents and so-called street-level bureaucrats (1980). Street-level bureaucrats are public officials such as teachers, police officers, nurses and social workers, sometimes literally working on the street and by definition interacting closely with citizen-clients (Bullock, 2019). Besides the close interaction with citizen-clients, Lipsky identified the capability to employ considerable discretion as a main characteristic of street-level bureaucrats (1980). In other words, street-level bureaucrats work at the ‘front line’, where they have significant leeway to decide on whom to treat, how and when (Maynard-Moody and Musheno, 2009). During these interactions they convert policies into real-life situations (Bartels, 2013).

2.1.1.1 From top-down to bottom-up: discretion

The term discretion encompasses street-level bureaucrats’ freedom to make decisions regarding sanctions and awards that citizen-clients receive or the provision of service (Lipsky, 1980; Busch and Henriksen, 2018). Thereby, Lipsky essentially endorsed a bottom-up perspective, instead of a top-down perspective, of policy implementation (Gilson, 2015). Rather than seeing discretion as a problem, as asserted by more traditional views of bureaucracy, such as that of Weber, discretion became increasingly seen as beneficial and valuable in decision-making and policy implementation processes (Weber et al., 1978; Bartels, 2013). While Weber considered fully constraining the discretionary

19 space of bureaucrats useful and necessary to prevent arbitrariness and abuse of power, hence the iron cage metaphor, Lipsky condemned such restrictions and detachments from reality counter-productive and undesirable (Ibid.). That is not to say that a new ideal-type bureaucracy as envisioned by Lipsky would mean that street-level bureaucrats work and interact without any constraints. Indeed, there are constraints and ought to be constraints, however not to the extent that bureaucrats cannot ‘make policy’ at all (Bartels, 2013). The constraints and pressures street-level bureaucrats typically face include: rules; scarce time, energy and financial resources; limited information; ambiguous and conflicting organizational goals and expectations; growing demands for services; and the complex nature of work (Lipsky, 1980; Bartels, 2013; Busch and Henriksen, 2018; Buffat, 2013). Such constraints and pressures both curtail and enable street-level bureaucrats to make moral judgments and decisions, to construct meaning and structures, hence determining the scope of discretion (Buffat, 2013; Busch and Henriksen, 2018; Schmidt, 2010). To fill the gaps left open by dynamic pressures and constraints, street-level bureaucrats manoeuvre between laws, rules and policies on the one hand, and real-life situations on the other hand, through their professional discretion (Buffat, 2013). The sources of discretion are henceforward diverse (Ibid.).

2.1.2 Screen-level bureaucrats and digital discretion

In his influential work Lipsky stated that “Street-level bureaucrats have discretion because the nature of service provision calls for human judgement that cannot be programmed and for which machines cannot substitute” (Lipsky, 1980: 161). This statement is questioned within the new context or policy space within which street-level bureaucrats operate, characterised by the use of ICT systems, (big) data, algorithms and artificial intelligence. These terms will be further elaborated on promptly. Either way, the new information techniques and systems available through these different novelties have

20 in common that they have the potential to change the discretionary room of street-level bureaucrats.

In this regard Bovens and Zouridis have argued that street-level bureaucrats have frequently become screen-level bureaucrats and (will) eventually turn into system-level bureaucrats (2002). Their working spaces have been replaced from the street to the computer and the computer may eventually take over their work (thus their discretionary room) entirely (Ibid.). The extent to which such unilateral developments have taken place is context dependent, as some tasks are more readily taken over by technology than others (Busch and Henriksen, 2018). Ultimately, the increased digitalization and routinization of human tasks may replace human judgments, thereby replacing human discretion (Ibid.; Bullock, 2019). However, there are also good reasons to believe that big data and technological developments more broadly convey enabling capacities for street-level bureaucrats, as they provide useful tools (Buffat, 2013). Besides, it appears inappropriate to assume that big data have the capacity to fully replace human discretion, not merely because of the plurality of sources and forms of discretion (Ibid.; Maynard-Moody and Musheno, 2012; Busch and Henriksen, 2018).

Not only has the use of big data a potential (most likely negative) impact on the scope of discretion, but also on the nature of discretion. In line with this argumentation some have argued that human discretion is replaced by ‘digital discretion’, for the same reason Bovens and Zouridis have argued for a renewal of terminology, namely the new digitalised context (Busch and Henriksen, 2018; Bullock, 2019). Digital discretion refers to digitalised procedures and examinations which influence or replace judgments made by street-level bureaucrats (Busch and Henriksen, 2018). In accordance with this

21 definition, discretion would shift from counting on human intellect and understanding to judgments that are at least partly based on (big) data (Bovens and Zouridis, 2002).

The changed scope and nature of discretion of street-level bureaucrats has important implications for the ethical aspects of policy implementation (Bullock, 2019). Whether the new context will lead to more morally just implementation is highly debatable and will depend on one’s notion of justice and individual circumstances. In a way then, the debate surrounding notions of justice regarding big data use is similar to the ongoing debate of whether and to what extent discretion is morally justified, but in a new policy space in which level bureaucrats have new tools: big data. When street-level bureaucrats work in increasingly digitised environments, as the social workers interviewed for this research do, they may therefore more appropriately be considered screen-level bureaucrats.

2.2 A solution or a challenge? A new policy space

2.2.1 Big data use

The concept of big data is ambiguous, although there is some common ground in its definitions (Klievink et al., 2017). What differentiates big data from more conventional forms of data are the volume, as the name implies, as well as the vast complexity, variety and velocity (Ibid.; Giest, 2017). Big data in itself is no (new) technology (Klievink et al., 2017). Instead, big data require new technologies and systems, due to their volume, complexity, variety and velocity (Giest, 2017). Big data combined with the proper techniques may contribute to more efficient and informed policymaking and has the potential to (or already does) change the provision of services by street-level bureaucrats (Ibid.). However, checking the veracity of big data is a difficult task and the uncertainty around big data more generally may lead to a loss of their value to enhance policymaking

22 processes (Ibid.). An important characteristic of big data is that their complexity is beyond human comprehension (Pasquale, 2015; Andrews, 2018).

Given the use of big data for many purposes it is useful to define the use of big data for the particular field discussed, hence the public sector. As such, Mergel et al. helpfully define big data in public affairs as: “high-volume data that frequently combines highly structured administrative data actively collected by public sector organizations with continuously and automatically collected structured and unstructured real-time data that are often passively created by public and private entities through their Internet interactions” (2016: 931). This definition accounts for the fact that the collection and structure of big data can be diverse and that the sources of data may also vary (Mergel et al., 2016). In addition, it is useful to conceive of big data in terms of their use, both actively and passively, thereby delineating the concept to the extent that it affects policymaking processes (including decision-making, regulation and tackling policy issues) (Klievink et al., 2017; Giest, 2017). For this purpose the definition by Mergel et al. is less appropriate. In this respect, Klievink et al. provide a useful overview of uses of big data based on a literature review. In many ways the uses overflow with the definition Mergel et al., in the sense that the use of big data would be characterised by the use of combinations of (un)structured and less structured data of vast and many datasets from several sources (2017). In addition, they note the “development and application of advanced analytics and algorithms, distributed computing and/or advanced technology to handle very large and complex computing tasks” as well as “innovative use of existing datasets and/or data sources for new and radically different applications than the data were gathered for or spring from” as two uses of big data (Klievink et al., 2017: 269). These uses of big data are particularly outlined for the public sector and link the concept of big data use to the technologies that make big data practically workable. The main activities of big data can

23 be summarised to involve the collection, combination, analysis and use of big data; the former three are here understood to form the basis of the latter, the use of big data, on which is focused (Klievink et al., 2017). The use of data can be further subdivided in their application, deployment, decision-making, production of insights and evaluation (Brohman et al., 2000; Miller and Mork, 2013; Cumbley and Church, 2013). Throughout this research the tasks of social workers determine the corresponding uses. Therefore, the focus is mainly on decision-making, evaluation and verification, based on information provided by the interviews conducted.

2.2.2 Algorithms

If we understand big data to constitute of data of a particular type, outlined above, rather than a technology, it makes sense to wonder what technologies are required to compute big data and make their use possible. One important type of technology capable of ordering and making sense of big data involves algorithms (Klievink et al., 2017). Since algorithms are mathematical constructs, their definitions are conventionally entrenched in computer science and mathematical discourse (Hill, 2015). However, as algorithms have gained increased attention and popularity, more colloquial definitions are increasingly used which emphasize the implementation of algorithms (Mittelstadt et al., 2016). Hill provides a formal definition, based on the mathematical aspect of algorithms, but still relatively easy to grasp for those not well-acquainted with more complex mathematical terminology, defining an algorithm as “a finite, abstract, effective, compound control structure, imperatively given, accomplishing a given purpose under given provisions” (Hill, 2015: 47).

Decision-making based on algorithms that are difficult to grasp, particularly for those not trained to do this, may lead to ethical problems (Mittelstadt et al., 2016). Algorithms can be used to turn data into evidence (Ibid.). Once presented as evidence,

24 algorithms and big data combined may prompt actions and motivate decisions by street-level bureaucrats (Ibid.). As Mittelstadt et al. rightfully point out, “algorithms are inescapably value-laden” (2016: 1). As a consequence, ethical concerns related to the use of algorithms (combined with big data) arise, adding further worries and complexity to the ethical dilemmas connected with big data aforementioned. Similarly to big data, algorithms may be conceived of as rather complex, possibly leading to further opacity of around such tools and their uses. Much more can be said about algorithms, but what matters here is that they can be combined with big data to influence decisions and that they bring forward ethical problems.

2.2.3 Evidence-based policymaking

In 2016, then President Barack Obama stated that “government will never run the way Silicon Valley runs” (White House, 2016). Hereby he meant to say that the government has to handle problems of a different nature and consequently attributes different values than the businesses Sillicon Valley is famous for, what Andrews refers to as ‘Sillicon values’ (2018). In a similar fashion and perhaps for this very reason, public sector organizations appear more reluctant of and are arguably late with the implementation of the use of big data as tools to deliver service (Klievink et al., 2017). Still the pressure to make use of evidence-based arguments and decisions, often grounded in data and algorithms, is increasing (Busch and Henriksen, 2018). The concept of evidence-based policymaking is tightly connected with big data (Giest, 2017). The fact that policymaking would be based on evidence (big data) implies that there is room for human interpretation and judgment, and that this would ultima ratio be conclusive (Buffat, 2013). However, the term evidence suggests some neutrality or objectivity, giving the concept appeal to more just policymaking (depending on one’s conception of justice) to the extent that it takes away human subjectivity in policymaking. In this way big data used as evidence can yield

25 additional resources for street-level bureaucrats to make objective decisions, which they can justify or legitimize accordingly (Ibid.). The neutrality connotation attached to such evidence-based policymaking rhetoric may steer street-level bureaucrats’ perceived ability of contributing or not to certain notions of justice as a consequence of big data use. Nevertheless, two important reasons arise why evidence-based policymaking is problematic. Firstly, evidence-based policymaking may not account for dynamic individual needs and circumstances (Bullock, 2019). In other words, evidence-based policymaking may be problematic when it comes to the fulfilment of the notion of treating dissimilar cases differently. Secondly, in spite of its objective connotation and rhetoric, evidence-based policymaking is inherently value-laden, just like big data and algorithms (Head, 2008; Wesselink et al., 2014). Moreover, “there is not one evidence-base but several bases” (Head, 2008: 4). Evidence needs to be interpreted and framed before it can lead to action in practice, is multi-dimensional (diverse) and contestable (Head, 2008; Wesselink et al., 2014). As such, evidence is not undisputable and should be negotiated instead (Wesselink et al., 2014). This is not to say that evidence-based policymaking should be discarded, rather it emphasizes caution when making assumptions that are supposedly neutral but are not.

“The epistemological simplicity of the EBP [evidence-based policymaking] rhetoric adds to its appeal, but detracts from its utility. Few would argue that policy should not follow the evidence, but what is policy-relevant evidence is determined by context. EBP’s rhetoric looks for ‘neutral’, context-free and universally applicable ‘evidence’ fit badly with this reality” (Wesselink et al., 2014: 342).

26 2.3 Administrative justice

2.3.1 Normative judgments

2.3.1.1 Treating similar cases alike

Street-level bureaucrats are (or were at least in the ‘Weberian’ prototype of bureaucracy) not merely expected to use their discretionary room professionally and efficiently, but also impartially (Weber et al., 1978; Bartels, 2013). Yet, as outlined above, Lipsky has shown that depersonalised, detached decision-making is far from the reality (Bartels, 2013). Extensive research has shown that unequal treatment by street-level bureaucrats is ubiquitous and that inequalities present are often upheld and even fed by the interactions or encounters and interpretations of bureaucrats (Ibid.). This raises questions regarding the desirable scope of discretion and what just treatment ought to entail. Still, there is general consensus that boundaries to disparities in treatment are morally appropriate and, along these lines, that equality before the law and equality in practice should not deviate too expansively from each other (Lipsky, 1980; Busch and Henriksen, 2018). The opportunity for street-level bureaucrats to treat similar and different cases differently is provided for by and necessarily (through irrationality, subjectivity and cognitive limitation) part of their discretion (Bagaric, 2000; Bullock, 2019). If discretion would be absent and if everyone is equal in the eyes of the law and other rules that may apply, it seems that the notion of treating similar cases alike is met, as the disparity between policy in the books and policy in reality is closed (Hawkins, 1992). Following this logic and notion of justice, then, it becomes clear why some deem the replacement of human discretion with the use of data as morally desirable. However, this argument leans on the conviction of data as neutral, impartial and therefore fair alternative.

27 The notion of treating similar cases alike as well as that of treating different cases differently, was first outlined by Aristotle, whose justice formula stated that: “like cases must be treated alike, and unlike cases unalike, proportionate to the differences between them” (Pobjoy, 2010: 184). Flowing from this, one could argue that in principle like cases must be treated alike, unless there is a reasonable foundation to believe that making a difference is morally legitimate (Ibid.). If this is indeed the starting point, there seems to be quite some threshold to treat different cases differently, hence a certain level of rigidity when it comes to the differentiation of cases. In other words, the question arises of what is to be considered relevantly alike or different (Ibid.). Another issue is the need for comparison, while cases may be unique (Schauer, 2018).

Moreover, the notion of treating similar cases similarly is often associated with consistency in treatment (Schauer, 2018; Marmor, 2005). This also brings a valid point of critique to the table, since consistency would require similar treatment even if this treatment is unenviable (Pobjoy, 2010). If a referee makes a flawed decision based on wrong estimation of the situation rather than bias, he or she would need to make the same flawed decision were the same situation to occur to the other team, according to this logic. A regime of precedent, which may be the result of the aim of consistency, is hence not to be understood in a descriptive but in an ascriptive sense (Schauer, 2018). Kozel, building on the premise that no decision-maker and no point of view is exactly equal, argued that a regime of precedent rather let to treating unlike cases alike (2013). As such, the notion of treating like cases alike is not synonymous with consistency, although the latter seems more realistic as an aim. Still, the notion of treating similar cases alike is valuable to the extent that it avoids unfair, biased decision-making and policy implementation. In addition, uniformity in treatment and believe in equal treatment by citizen-clients

28 certainly add some value to the notion, although the latter two do not seem to provide a very strong argument (Marmor, 2005).

It is noteworthy that there is a distinction between equity and equality, where the former, broadly defined, implies equal opportunity (Pobjoy, 2010). In addition, it should be mentioned that equality as a goal and equal treatment as an aim in itself should not be confused. Equality as a goal would most likely require unequal treatment (redistribution), while the latter is rather based on a moral principle of what ought to be (Frederickson, 1990). In this research the focus is on this normative principle of equality with regard to treatment (i.e. the treatment of citizen-clients by social workers).

2.3.1.2 Treating dissimilar cases differently

Aristotle’s justice formula also set the basis for another notion of justice; treating different or dissimilar cases differently. This raises similar concerns to the notion of treating similar cases alike, but departs from the idea of equality as consistency and leaves more room for responsiveness in treatment. As such, this notion fits better with the principle of treating everyone according to his or her needs or deservingness (Kelly, 1994; Maynard-Moody and Musheno, 2012). This requires fair levels of discretion for street-level bureaucrats to make moral judgments, influenced by the perceptions and cultural norms and values of what is fair and just (Hawk and Dabney, 2014; Zacka, 2017). Treatment, according to this logic, is fair or just if dissimilar cases receive differential treatment. This allows street-level bureaucrats to be more responsive and could lead to more rather than less equitable, democratic and ethical judgments, taking into account individual circumstances and dynamic contexts (Bullock, 2019; Bartels, 2013). The notion of treating dissimilar cases differently can henceforward be used to endorse certain amounts of discretionary space of street-level bureaucrats.

29 There seems to be some moral trade-off in the decisions and judgments by street-level bureaucrats when it comes to treating (similar) cases alike (equally) or according to their professional judgments, regarding individual needs or deservingness for instance (Head, 2008). Whether more equal or more responsive treatment is appropriate depends on the context and the street-level bureaucrats’ conceptions of justice, among other reasons (Kelly, 1994). The discretion provided to street-level bureaucrats may be considered inherently problematic to the extent that their judgments of differentiating in treatment are based on immoral subjectivity rather than neutral (Zacka, 2017). While aiming at both responsiveness and consistency may seem contradictory, there is not necessarily a trade-off between the notions of treating similar cases alike and treating different cases differently (Raaphorst, 2020). Indeed, as proposed by Aristotle in his maxim the two can be applied simultaneously (Pobjoy, 2010). They do conflict, however, in the extent to which they allow for discretionary space of street-level bureaucrats, hence the importance of which notion one gives priority for discussions of discretionary scope (Bagaric, 2000). In a similar vein, Bullock argued that the evolution of digital discretion, accompanied by a decrease in human discretion, “seems likely to improve ethical and democratic values for discretionary administrative tasks but presents challenges for professional and relational values” (Bullock, 2019: 757).

Various factors may impact equal treatment and/or responsiveness. By means of illustration; one way in which the exercise of discretion may result in unequal treatment, but also in equal treatment when unequal treatment may be more appropriate, is through the use of coping mechanisms (Bartels, 2013). Such coping mechanisms involve short-cuts, heuristics or categorizations to make sense of the complexities of cases and keep the work manageable (Lipsky, 1980). Coping mechanisms can lead to stereotyping and biased decision-making, hence unequal and possibly unfair treatment or discrimination (Bartels,

30 2013). When divergent cases fall into similar categories and are consequently treated as if they were similar, that is treated equally, this may also be considered unfair or undesirable. Yet, in accordance with Lipsky’s argument, it appears impossible as well as inefficient for street-level bureaucrats to work without the application of coping mechanisms, due to the constraints and pressures aforementioned, as well as the dynamic contexts within which they work (1980). Big data may provide opportunities here, since it allows street-level bureaucrats to process information more quickly and extensively as well as providing legitimacy (Bartels, 2013). Still, big data are unlikely to fully discard the use of coping mechanisms and may even be considered similar in terms of the simplistic, two-dimensional image they provide, particularly where big data are too complex to fully grasp by street-level bureaucrats (Pasquale, 2015).

31

C

3

Methodology

3.1 Research design

3.1.1 Inductive research

Given the lack of causal models and data surrounding the empirical relationship of interest in this study, this research uses an inductive approach. As such, the research is exploratory as it attempts to contribute to theory-building (fully developing a theory is beyond the scope of this thesis however), using a ‘bottom-up’ approach (Woo et al., 2017). Herein looking for themes or patterns in observations forms the basis (Ibid.). If these patterns or relationships can reasonably be generalised, this may eventually lead to the formation of a theory (Ibid.). More specifically, this research builds on the qualitative method of inductive thematic analysis, in accordance with Braun and Clarke (2006).

3.1.2 Qualitative research

The research conducted is qualitative in nature. It focuses on a nonnumerical empirical relationship, in an attempt to say something useful about this relationship through interpretation (Hernández Sampieri et al., 2006). Qualitative research thus allows for an interpretation of those researched, social workers, to explain what is observed (throughout the interviews). The amount of people interviewed is limited, hence the inability to apply a quantitative approach. Nor would a quantitative approach be appropriate for the research aim. Instead, the aim of the research is to find new information, in line with the inductive type of research, by seeking to acquire rich information through more in-depth interviews. In summary, the research aims to interpret interpretations, requiring a qualitative approach.

32

3.1.3 Case study

The case study resolves around street-level bureaucrats or screen-level bureaucrats; more specifically, around social workers. Social workers can take multiple functions and may be broadly understood as those working in the social domain. This research particularly concerns social workers in the public sector, working for municipalities in the field of social security in the Netherlands. Examples include customer managers, fraud prevention investigators and quality assurance officers. There primary task is generally to prevent fraud with social benefits, to provide services to (such as helping find a new job) and check whether citizen-clients cooperate and stick to their duties once they receive social welfare assistance. Social workers traditionally encounter actively with citizen-clients (unlike some other public officials), while their tasks simultaneously have the potential to be (partly) automated and their decisions to be data-driven (unlike teachers or nurses for instance, who are unlikely to be greatly affected by big data use at least in their current capacities) (Busch and Henriksen, 2018). Bullock argues that fraud prevention can be greatly improved through artificial intelligence, for instance (2019). In this sense, social workers arguably can neither be completely considered street-level bureaucrats, nor system-level bureaucrats, if seen on Bovens and Zouridis scale of classifications (2002). Instead, where they are on the scale from street-level bureaucrats to system-level bureaucrats depends on their levels of discretion. The social workers interviewed worked in a digital environment and dealt with big data (systems) on a regular basis, while simultaneously maintaining significant discretionary space and frequently encountered with citizen-clients. As such, they could be more appropriately classified as screen-level bureaucrats. At the same time, social workers, despite their similarities, do not constitute of a homogenous group, increasing the generalisability of results to a certain (yet limited) extent. Finally, The Netherlands forms a useful setting

33 since it is considered one of the highest ranked countries on the UN e-government index and has a stable political and policy climate (Klievink et al., 2017). The case of social workers or workers in the social domain was therefore chosen on theoretical grounds, especially while they may be considered screen-level bureaucrats, hence occupying a role where big data is present, but not dominant. Section 4.1 elaborates on the meaning of and big data systems used in the context of social work at municipalities, providing some more background for this case study.

3.2 Research methods 3.2.1 Interviews

Open interviews were conducted with nine people (see table 1 for the interviewee list), either social workers or experts in the field of big data use at municipalities (see appendix 1 for the interview guide). All interviews were conducted in the Netherlands in an online environment (Microsoft Teams) or by phone call. The interviews were conducted in Dutch. The quotes provided in the analysis are all own translations. There are multiple reasons to opt for this interview strategy. The interviews focused on the interviewees’ narratives and stories. More concretely, the interviews centred around stories and examples concerning real-life situations of what was considered just or unjust use of big data in day-to-day work tasks involving citizen-clients. These stories may involve the interviewees as well as other social workers, but necessarily should be about big data use to target citizen-clients. In this way, rather than directly asking about perceptions or priorities for certain principles of justice - treating similar cases similarly or treating dissimilar cases differently – those could be revealed indirectly from the stories and examples. Importantly, this avoids the interviewee from providing ‘wishful’ answers for instance because they believe this to be more morally or socially accepted. Asking further

34 questions or examples, particularly about conflicting values faced, helped to keep the interviews focused, not deviating too much from what is of interest for the research. Evidence that stories are useful to gain insights into normative beliefs and conceptualisations of interviewees is abound (Kelly, 1994). In the words of Maynard-Moody and Musheno:

“(…) when examining moral reasoning, especially when it is deeply embedded in the normative structures of institutions and policy regimes, we cannot expect people, whether frontline staff or upper-level managers, to articulate their actual decision norms. Narratives, on the other hand, provide rich evidence of the normative reasoning and context that shape judgments and actions. Through narratives, storytellers reveal more than they consciously know.” (Maynard-Moody & Musheno, 2012: 21)

They base this on experience, being aware that asking directly about judgments or definitions of justice or fairness provides one-dimensional, rather trivial answers and perspectives (Maynard-Moody & Musheno, 2012).

Interviewees1 _{Generic function Size of}

Municipality2 Male/Female Date Miranda Veldwijk Fraud prevention investigator Small Female 1-12-2020 Justin Stuurman Policy officer: Work, Identity and Income

Medium Male 8-12-2020

Hans Kramer Quality assurance officer

Small Male 8-12-2020

1 _{The real names have been changed to guarantee anonymity and the names noted here are made up.} 2 _{Small <50,000 inhabitants, medium <50,000-100,000> inhabitants, large >100,000 inhabitants}

35 Asha Arneja Customer manager

social domain Small Female 10-12-2020 Nienke Bertrand Customer manager social domain Small Female 15-12-2020 Carola Katendrecht Quality assurance officer Large Female 16-12-2020 Ruben De Vries Data analyst Dutch

Foundation

Male

17-12-2020 Jan

Groothuizen

Data analyst Small Male

17-12-2020 Jeroen van Vliet Compliance officer

social domain

Medium Male

18-12-2020 TABLE 1: Interviewee list

3.2.2 Operationalization

Based on the theoretical framework and literature review in previous chapters a number of concepts are defined in table 2, which are relevant and portrayed in a clear overview with the purpose to guide the interviews. The interview questions (appendix 1) are henceforth based on the concepts operationalised forthcoming.

Concepts Definition Indicators

Treating similar cases

similarly (alike) The principle that like cases must be treated alike, unless there is a reasonable foundation to believe that making a difference is morally legitimate

- Principles of equality of treatment

- Consistency of treatment

Treating dissimilar cases

36 or her needs or deservingness - Focus on individual treatment - Customised work

Big data High-volume data of vast

and many datasets from several sources, which complexity goes beyond human comprehension

- volume - complexity - variety - velocity Algorithm A finite, abstract, effective,

compound control structure, imperatively given, accomplishing a given purpose under given provisions

- Finite time and space - Abstract (no space-time

locus) - Requiring no judgment or understanding - control - structure - imperative (how-to) Digital discretion Digitalised procedures

and examinations which influence or replace judgments made by street-level bureaucrats - Perceptions of the replacement of tasks formerly performed by humans by digital instruments Discretion The freedom street-level

bureaucrats have to make decisions regarding sanctions and awards that citizen-clients receive or the provision of service

- Perceptions of

independence/autonomy in decision-making processes

- Room to solve problems using personal expertise - Human agency

- Close interactions with citizen-clients

TABLE 2: Operationalization of theoretical concepts

3.3 Data analysis

3.3.1 Thomas theorem

The Thomas theorem forms the basis of the particular interest in street-level bureaucrats big data use. The Thomas theorem states that "if men define situations as real, they are real in their consequences" (Thomas & Thomas 1928: 572). In other words, if we take men to be street-level bureaucrats (social workers), what matters is their subjective perception of a situation and how big data use is perceived as a possibility in that particular situation. The ‘situation’ here involves the context in which the street-level

37 bureaucrats chose to use big data or not, based on their perceptions of the situation (which is subjective) and the corresponding priorities of justice principles (equally subjective). These subjective interpretations have real, objective consequences and actions, namely the use of data for decision-making and the conforming impact this has on citizen-clients (Mertol, 1995). In methodological terms the Thomas theorem pleas for the qualitative research approach to gain further insights into how situations are defined, that is how they are interpreted, perceived and hence turned into practice, since this goes beyond mere empirical observations to carry explanatory value instead (Ibid.). Therefore, it is evident how the theorem complements with the latent level of analysis, as described previously. Once the ‘how’ of the use of big data is understood (involving the prioritisation of principles of justice), this provides further insight into how this affects the perceived abilityto treat similar cases similarly or dissimilar cases differently.

3.3.2 Inductive thematic analysis

In order to identify common, salient themes and patterns among the interviews and allow for more in-depth analysis of these themes, inductive thematic analysis was used following Braun and Clarke (2006). As such, the analysis and discussion are structured along the lines of the themes identified, nevertheless providing sufficient flexibility to incorporate all important data from the interviews and not lose sight of the context (Braun and Clarke, 2006). Yet, the importance attributed to a theme and/or sub-theme has more to do with its relevance for this particular research than its prevalence among the interviews necessarily, so as to provide a more profound analysis of those themes that contribute to the inductive, exploratory aims of this research. This is not to say that the themes are predetermined and that the data obtained through the interviews is used to either confirm or disconfirm research questions or theory. On the contrary, the themes

38 are data-driven, flowing from the data in a bottom-up manner to identify themes (Ibid.). This inductive type of thematic analysis was applied given the exploratory nature of this work (see chapter 1). Furthermore, the themes or patterns are determined and identified at a latent level. This implies that the data obtained by the interviews are not merely analysed in a descriptive way (at the semantic level), but rather that the data on which basis themes are identified and which are interpreted to discover and analyse “the underlying ideas, assumptions, and conceptualizations / and ideologies / that are theorized as shaping or informing the semantic content of the data” (Braun and Clarke, 2006: 84). In other words, the stories and examples provided by the interviewees are not only described, but also interpreted in order to identify their broader meaning. However, while the data is interpreted and it is attempted to identify patterns of broader meaning beyond what is articulated, the thematic analysis does not use a constructionist approach since its aim is not to identify how experiences as well as meanings are constructed and reconstructed and since this, while interesting, would be beyond the scope of this work (Braun and Clarke, 2006). Instead, an essentialist/realist approach is applied, to theorize meaning in a more straightforward way building on the assumption that meaning and experience are a reflection of the articulation (Ibid.). While identifying themes at the latent level and simultaneously applying an essentialist/realist approach may sound controversial, this is not necessarily the case (Ibid.). The combination is useful to recognise that what is being said has deeper implications, while not going into too much detail regarding the sociocultural context and social construction that forms the meaning in the first place.

Underlying the analysis are the six steps outlined by Braun and Clarke (2006). First of all, all interviews were transcribed and read in order to familiarize oneself with the data (Ibid.). Secondly, initial codes were generated across the transcriptions

39 systematically (Ibid.). These codes were in essence semantic, but occasionally latent when appropriate. More in-depth interpretation of the data and the identification of overarching themes and patterns based on an analysis of the codes only occurs in the third stage, as described by Braun and Clarke (2006). At this stage, initial patterns, themes and sub-themes are recognised without losing sight of the broader articulations in the interviews. The reason for this is that the patterns, themes and sub-themes still need to be refined, so it is too early to discard information (Ibid.). This refinement occurs in the fourth phase through a review of the previous stages. In the fifth phase, themes are ‘defined and named’ (Ibid.). The themes are identified and linked into a coherent story, which involves both thorough analysis of the separate themes and a more general analysis (Ibid.). Finally, the report is produced and analysis is linked with the research question, complemented with examples and extracts from the interviews (Ibid.). These five phases that lead up to writing the report (phase six) are visually portrayed in the model below.

Figure 1: five phases of thematic analysis (Braun and Clarke, 2006)

Transcribing

Initial coding

Searching for potential themes

Reviewing themes

Defining and

naming themes

40 3.4 Reliability and validity

A major concern regarding the reliability is whether the interviewees speak freely about their beliefs and conceptions. As such, they may reveal information, which may go unnoticeable, or change information so as to make it more socially acceptable (Kelly, 1994). More thorough ethnographic research may be needed to overcome this problem, although the more open type of interviews as well as the provision of examples and more story-like answers partially avoids this as well (Ibid.).

The replicability of the research is enhanced through the provision of the interview guide and the operationalization and description of the type of interviews, trying to be as transparent as possible. In addition, the interviews were recorded and verbal transcripts were made, which can be checked in case of doubt or unclarity. Finally, cross-checking formerly obtained information through interviews in further interviews positively affects the reliability of information collected.

To ensure sufficient internal validity the research was designed to gather in-depth information through open interviews, allowing the interviewees to speak freely. The focus on narrative and examples further provide the interviewees the opportunity to speak freely, while it leaves less potential for steering answers in one direction. Yet, the research exclusively makes use of interviews, hence limiting the internal validity.

The external validity of the research, the generalisability of the findings, is limited by a number of factors. First of all, social workers constitute only one type of street-level bureaucrats, although they form a heterogenous group and are not at the extreme end of the street-level bureaucrats-system-level bureaucrats spectrum (Bovens and Zouridis, 2002). Still, the extent to which the results apply to other street-level bureaucrats may be rather small. Secondly, the findings are largely influenced by the specific context within

41 which social workers operate, therefore limiting the external validity, both beyond national borders and beyond the organizational level. Finally, the amount of interviews, hence observations is limited. Consequently, the results are less likely to be generalisable to other cases. This does not necessarily make the research invalid, but rather highlights the fact that more research is needed to be able to construct a fully grounded theory, while the aim of this exploratory research is merely to contribute to theory-building and not to generalise results. Once more and more exhaustive cases are studied and theory-building is more advanced, findings may also be generalised more extensively or the theory may apply to more cases.