Data-Driven Policy Making for Sustainable Energy: The impact of data readiness on the use of dashboards for renewable energy policies

(1)

Data-Driven Policy Making for Sustainable Energy

The impact of data readiness on the use of dashboards for renewable energy policies

Simone van Raaij_{, s1508016}

Leiden University, Faculty Governance and Global Affairs

Master Public Administration, Specialisation: Public Management Thesis coordinator: Dr. Sarah Giest

Second reader: Dr. Alex Ingrams Word count: 19,858

(2)

Foreword

I would like to thank several people who have helped me with my thesis during the past year and without their support it would’ve been much more difficult to finish the thesis. First of all, I would like to thank my supervisor Sarah Giest who helped me to shape my ideas for this thesis and provided valuable feedback throughout the process.

Furthermore, I would like to thank my supervisor at the province of South-Holland who allowed me to work on my thesis during the internship and the different colleagues who gave me valuable ideas for my thesis, and helped me to get into contact with different co-workers within the province and the different municipalities. I am thankful for the help of Tanja Haring, who helped me to spread my survey across the municipalities and gave helpful insights in the dashboards that are developed by the province, and the relation between the province and different municipalities. Furthermore, I want to thank everyone who participated in the interviews and who helped me to gain valuable information in the data readiness of the municipalities and their use of dashboards.

Finally, I want to thank my friends, boyfriend and family for being there for me when I needed them, who always offered to help me with my thesis, showed interest and provided distraction when needed. While writing my thesis could be a difficult process, they always believed in me and encouraged me to try my hardest.

(3)

Abstract

Municipalities are facing an enormous challenges as they have an important role in the energy transition by stimulating the transition were houses move away from heating by gas and transit towards heat from a sustainable energy source. Different organisations provide dashboards to visualise relevant factors such as the potential of different heat sources or the energy demand. Which, can be used by municipalities for this transition. However, the question remains whether these dashboards are actually used by municipalities and what could influence the use or absence of use of the dashboards. In order to analyse this, the data readiness framework is used that tests whether municipalities are ready for the use of big data for policies. This research focuses on the following research question: “_{To what extent has an organisation’s readiness an impact on the use of dashboards for renewable} energy policies?”

A qualitative small-n case study was selected for the analysis. To select the cases for the research, a survey was send out to all the municipalities in the province of South-Holland. This provided an overall overview of the data readiness of these municipalities. Based on these results, a similar case design was used to select six municipalities of similar size but whose data readiness varied. Within these municipalities, semi-structured interviews were held with employees that either work with data or dashboards, or work on energy related policies. The interview questions were based on the different components of the data readiness framework: organisational alignment, organisational maturity, and organisational capabilities. These components were then operationalised and narrowed down to: a supporting infrastructure, collaboration and coordination with partners, and stakeholder and technical capabilities. Furthermore, dashboard design and the quality of the dashboard was looked at as an explanatory variable.

The results show no relation between a supporting infrastructure and the use of dashboards. There are some indications that collaboration with other organisations, and higher stakeholder and technical capabilities seem to have an impact on the use of dashboards although some uncertainties remain. Overall, it is likely that there is some impact of data readiness on the use of dashboards. However, more research remains necessary. As it seems likely that more collaboration and stakeholder and technical capabilities have some influence on the use of dashboards, it could therefore be interesting to further look into the relation between these concepts and the use of dashboards for future research.

(4)

4.3.3 Organisational Capabilities 34 4.3.4 Use of Dashboards 35 4.4 HLTsamen 36 4.4.1 Organisational Maturity 36 4.4.2 Organisational Alignment 37 4.4.3 Organisational Capabilities 38 4.4 Use of Dashboards 38 4.5 Pijnacker-Nootdorp 39 4.5.1 Organisational Maturity 39 5.5.2 Organisational Alignment 39 5.5.3 Organisational Capabilities 40 5.5.4 Use of Dashboards 40 4.6 Vlaardingen 41 4.6.1 Organisational maturity 41 4.6.2 Organisational Alignment 41 4.6.3 Organisational Capabilities 41 4.6.4 Use of Dashboards 42 4.7 Westland 42 4.7.1 Organisational Maturity 42 4.7.2 Organisational Alignment 43 4.7.3 Organisational Capabilities 43 4.7.4 Use of Dashboards 44 5. Analysis 45 5.1 Organisational Alignment 45 5.2 Organisational Maturity 46 5.3 Organisational Capabilities 47

5.4 Dashboard Design and Quality 49

6. Conclusion and Discussion 50

6.1 Concluding Remarks 51

6.2 Contributions and Limitations 52

6.3 Recommendations for Future Research and Policy 53

References 54

Appendix 58

(6)

1. Introduction

Currently, gas is the common energy source to heat houses, buildings, greenhouses and our tap water. However, according to European agreements, all new buildings have to be (almost) energy neutral in 2020, and in the energy agreement it is said that all the buildings have to have an average of energy label A (which is the greenest energy label). This means that big changes have to be made to provide all the buildings with green energy (Ministry of Economic Affairs, 2016).

This requires a major energy transition, and asks for a considerable role of municipalities whose role is to stimulate that houses are no longer heated with gas, but with more sustainable energy sources (VNG, n.d.). To accomplish this, information is needed to answer different questions such as which alternatives to choose from to heat houses, how to tackle the transition, what the capacity is of different energy sources, or where the gas and electricity infrastructure lays (de Ree, 2019). However, municipalities have indicated that they have a lack of knowledge of the energy transition within the organisation, followed by a lack of knowledge sharing with different parties, and that there is not enough communication with municipalities and regions within the regional energy strategy (RES), which is a programme where stakeholders have to develop an energy strategy within their region (de Vries, Vringer, Wentink, & Visser, 2019) (Nationaal Programma Regionale Energie Strategieën, n.d.).

To overcome these problems, different public organisations try to increase the share of information on the energy transition, one example is the province of South-Holland, who installed a team of around twelve energy advisors to aid the municipalities. These are public servants that officially are in service of the province, but work at one or more municipalities to help them with the energy transition and help in the region with regional coordination, collaboration, and the exchange of experiences and knowledge (Provincie Zuid-Holland, 2019, p. 17).

Furthermore, the province has developed different dashboards to guide municipalities in the energy transition, such as the ‘warmtetransitieatlas’, that was developed as a response to the 40 local heat maps of municipalities in the province. By bundling this information into one map, each municipality has the same information and municipalities can start to work on the transition together with the province (Wiggers, 2016). On the dashboard, information and data is visualised such as: the gas demand per houses, the type of houses, when they were build, different housing corporations, existing heat sources, heat networks, etc. This information can be useful to determine what kind of renewable energy source fits in a certain neighbourhood (Provincie Zuid Holland, n.d.).

The province is not the only organisation who developed a dashboard that is related to the energy transition, as there are numerous dashboards that provide information on different factors such as the location of solar panels or the possibilities for alternatives to heat in different areas (Nationale Energie Atlas, 2019). However, the question remains whether these dashboards are also used by

(7)

municipalities and organisations when developing renewable energy policies, and if they are or are not used, what would be the reason for this.

As dashboards visualise data, and allow to combine and analyse different data sets, the data readiness framework could be useful to look at underlying factors for the use of dashboards for policies (Klievink, Romijn, Cunningham, Bruijn, 2016). The data readiness framework test whether organisations working in the public sector are ready to use big data for policy-making. The framework is based on three main aspects, or uncertainties, where the first aspect is organisational capabilities. Here, the question is raised whether the organisation has enough capabilities to use big data. The second is organisational maturity, which looks the maturity of e-government initiatives and whether there is collaboration with other public organisations. Third, there is organisational alignment, which is whether the organisation’s structure and activities align with the use of big data. Based on these aspects, an organisation's readiness for big data can be assessed (Klievink et al., 2016, p. 268-269). This framework is relevant for this research as data visualisation is one of the big data activities that Klievink et al. (2016, p. 270) mention for the application of analytics. Data visualisation helps to show data real time or near real time, for example with the visualisation of traffic jams (Mattheus, Janssen, Masheshwari, 2018). Furthermore, dashboards allow to use and combine various large datasets, and dashboards can be seen as innovative use of existing datasets for new applications as some datasets that are added in for example the ‘warmtetransitie’ were already available and where gathered from public sources (Brouwer, 2019). These were therefore not necessarily intended for dashboards. Dashboards thereby qualify on three of the five the big data characteristics described by Klievink et al. (2016, p. 269-270) who argue that data can be qualified as big data when it meets three or more data characteristics. Therefore, as dashboards can be seen as a form of big data, the expectation is that the readiness framework can also be used for looking at the use of dashboards at that organisations that are more ‘ready’ for the use of big data, are more likely to be able to use dashboards for policies. This leads to the following research question:

“_{To what extent has an organisation’s readiness an impact on the use of dashboards for} renewable energy policies?”

1.1 Relevance

The goal of this research is to provide a deeper understanding of the challenges and opportunities muncipalities face in the energy transition, and what the role of dashboards is in this transition. This is relevant for a number of reasons. First of all, according to Mergel, Gongb and Bertotc (2018), more research is needed to understand how the structure and characteristics of a government have an influence on the government's’ ability to engage in agile methods. Under structure and characteristics,

(8)

aspects such as centralized, innovation efforts and technology maturity are mentioned. These characteristics can be compared with the aspects of data readiness, which measures an organisation's’ maturity, alignment and capabilities. Furthermore, agile methods enable a government to be more adaptive and are based on approaches that make use of new opportunities such as big data. Data dashboards could therefore be seen as an agile method as they can be used to make a municipality more adaptive to the challenges of the energy transition and they are a relative new method. Therefore, researching the impact of data readiness on the use of data dashboards, this research builds further on one of the gaps set by Mergel et al. (2018).

Secondly, more research is needed on the use of data dashboards, how they are used by different organisations, and for which purposes dashboards are used (Kitchin, Maalsen & McArdle, 2016). This research builds further on this notion by looking whether dashboards are used by municipalities for energy policies.

There is also a societal relevance for this research, as data dashboards have different beneficiaries for the public sector. Dashboards can help with decision-making, enable public participation, and help with transparency by reducing information asymmetry (Mattheus, et al., 2018). As dashboards are recently getting more attention in the public sector, more research is beneficial. By uncovering factors that have an influence on the use of data dashboards in the public sector, these factors can be taken into account when municipalities use dashboards. Which, can make the use of dashboards more effective. Furthermore, Klievink et al. (2016, p.279) argue that while organisations might be capable of using big data, they will not gain from the use of big data as it does not fit with their organisations and main statutory tasks. As the use of dashboards have multiple benefits, this research aims to add new insights in whether data can provide benefits for public organisations, and by looking at the readiness framework it can be assessed whether there are also benefits of using data dashboards even if a organisation does not score high on all the aspects of the readiness framework.

1.2 Outline

In the following sections, first the current research on the digitisation of governance will be looked at in chapter 2. There, literature on data readiness and the different readiness components are discussed. Furthermore, literature on dashboards and the opportunities and challenges of using dashboards, and literature on smart cities will be discussed. In chapter 3 the methodology of this research is discussed. For this research, employees six municipalities in the province of South-Holland, together with employees of the province of South-Holland, are interviewed on their use of dashboards for energy policies and their organisational readiness. To select relevant municipalities, a survey is send to all the municipalities in the province of South-Holland to assess their data readiness so that six municipalities can be selected that are of similar size and vary on their readiness as in accordance to a similar case

(9)

design. This methodology will be further elaborated in chapter 3. Then, in chapter 4 the empirical findings of the interviews are discussed. These findings will be discussed and analysed in chapter 5 based on the hypotheses derived from chapter 2. In chapter 6, a conclusion on the research question is given, together with the limitations of this study, and further recommendations for policies and future research.

(10)

2. Theoretical Framework

In this chapter, an overview will be given on the current literature regarding the use of data in the public sector and data-driven policy making. More specifically, concepts such as smart cities, the data readiness framework, joined-up governance, and the use of data dashboards. Literature on smart cities is relevant, as this research will focus on the use of dashboards by local governments, and joined up and governance is discussed as it is related to the organisational maturity component from the data readiness framework.

2.1 Big Data in the Public Sector

The use of data for policy making is rising in the public sector, and according to the digital-era governance concept, this increase in technology and digitalisation in public sectors can be seen as a positive development. However, public organisations are still behind private organisations, as public organisations don’t always have the capacity and skills to deal with new technologies and big data. This in turn leads to high costs, outsourcing or training public servants (Giest, 2017, p. 369). Yet, public organisations experience pressure from citizens and the private sector to adapt their institutional structures and transit towards new forms of data-driven policy making. A question that thereby arises is whether public organisations are actually ‘ready’ to make this transition. This question is researched by Klievink et al. (2016), who look at the readiness of public organisations to use big data, and their uncertainties for big data, based on their data readiness framework. Klievink et al. (2016) define big data as data that meets three or more of the following five characteristics: it can use and combine large datasets from different sources (1); it can use and combine structured and unstructured data for analyses (2); it can use data in real time or near real time (3); it can develop and apply different analytics and algorithms to handle complex computing tasks (4); and it can use existing data sets and sources innovatively for other applications that the data was intended for (5) (Klievink et al., 2016, p. 269).

As mentioned in the introduction, dashboards meet three of these five characteristics as they: allow to use and combine various large datasets from different sources (1); can be seen as innovative use of existing datasets for new applications; and can be used to shows real time or near real time, provided that they are updated frequently (Brouwer, 2019)(Mattheus, et al., 2018). Furthermore, data visualisation is one of the big data activities that Klievink et al. (2016) mention for data analytics which further indicates the relevance of data dashboards within the data readiness framework. In the following sections, first the concept of data dashboards will be further provided. Followed, the data readiness framework and the different components will be further discussed.

(11)

2.2 Data Dashboards

An application that is more often used in governance, and in the digital government transformation, are interactive data dashboards. Which are especially coming up within smart cities _(Kitchin, Lauriault, & McArdle, 2015) _{. Data dashboards visualise data and give a summary of the most} important information and data on a single screen. Here, data is displayed through means such as graphs, bar charts, colour, and can consist of different layers where each layer provides a visualisation of interconnected data. While data dashboards where originally fixed, they are now analytical dashboards as well which are more interactive as users can interact with the data by selecting which data they want displayed on the dashboard. Data dashboards give an easy and simple understanding of the data that can often be understood without the help of a specialist, and can be used to visualise data in documents or can be shared through social media (Kitchin et al., 2015, pp.10 -12). By visualising information, dashboards can give an overview at glance, and provide new insights. They can create transparency and accountability, and reduce information asymmetry by preventing that one party has more information than others. This is helpful for decision-making and to increase government trust (Mattheus et al., 2018). However, there are also a few challenges in the use and creation of dashboards.

2.2.1 Challenges in the use of Dashboards

The design of dashboards is important as a good design helps with an increase of transparency and accountability, however, shortcomings in the design can lead to reduced transparency and can make the dashboard too complex and difficult for non-experts to understand (Brath & Peters, 2004). There are different risks and challenges that are important to consider for the useability of dashboards (Mattheus et al., 2018)(Kitchin & Ardle, 2016). One issue that can lead to challenges is epistemology, where the question arises what the scientific assumptions are behind city dashboards. Dashboards visualise datasets and the underlying structures, relations or gaps. Here, the assumption often is that data is independent and objective and can be accurately measured, analysed and visualised. However, the data presented in dashboards is not purely objective, but is a result of ideas, instruments, practices, knowledge, and normative assumptions on how and what should be measured and which conditions and relations are relevant. (Kitchin & Ardle, 2016, pp.4-6). The design of the dashboard is made my decision-makers who can be influenced by specific or local circumstances, their own opinions and taste, or are under time pressure and make too fast decisions. Furthermore, there can be stakeholders and actors involved with their own agenda and goals, which leads to ‘management by objectives’ (Kourtit and Nijkamp, 2017, p. 27).

Another issue is related to scope and access. Here, the questions lies on how comprehensive and open for others dashboards are. First of all, the scope of dashboards are limited. There are

(12)

different indicators that have to be taken into account and these cannot all be measured in one measurement. Data collection is costly, and having to collect data on numerous factors can lead to high costs. This means that not all the relevant information is always on the dashboard. Furthermore, the information that is on a dashboard is influenced by the instruments and technologies that are used. Another limitation is whether or not the data generated for dashboards can be reused, as public data is often restricted for other usages due to copyright laws and licenses. Then, even when there is access to view the data or the analyses, the dataset itself might be closed off and can’t be downloaded (Kitchin & Ardle, 2016, pp.6-9).

The third issue is veracity and validity, and is about the data quality and whether the appropriate measurements are used. If the wrong measurements and parameters are used, the data that is generated has no validity. However, to control this, the data has to be open sources, otherwise this can be difficult to discover (Kitchin & Ardle, 2016, pp.6-9). Other issues that can lead to lower data quality can be a lack of maintenance and updating of the dashboards, which can lead to outdated data. Sometimes, the use of dashboards can lead to new questions that require new data collection. An inability to adapt to new developments can also influence the quality of the dashboard (Mattheus et al., 2018).

Fourth, there is a issue regarding the useability and literacy of dashboards. While dashboards in principle provide interactive maps that are easy to understand, this is not always the case. Dashboards can be too complex due to problems with the site navigation when too many layers and indicators exist. Dashboards provide numerous tools to change the maps, and it can be unclear how to interact with the data and change the layout. These difficulties not only make the dashboard difficult to understand for citizens, planners and policy makers do not always have knowledge on the usage of data and analytics. As a result, the transparency reduces, or information on a dashboard can be misinterpreted (Mattheus et al., 2018).

2.3 The Data Readiness Framework

As mentioned earlier is the data readiness framework based on three concepts; whether the organisation has enough capabilities to use big data, the maturity of e-government initiatives and whether the organisation’s structure and activities align with the use of big data. Then, the readiness of an organisation for the use of big data can be assessed based on the sum of how the organisation scored on each of these three components (Klievink et al., 2016, p. 268-269). These three components will be individually discussed in the following sections.

(13)

2.3.1 Organisational Alignment

Public organisations experience pressure from citizens and the private sector to adapt their institutional structures and transit towards new forms of data-driven policy making. Having the right infrastructure is necessary to allow open data to flow from one system to another. Open data can be defined as data that can be accessed by a large audience and can be downloaded or streamed, and it enhances new collaborations between citizens, public and private organisations. (Jetzek, 2015). (Janssen, Konopnicki, Snowdon, & Ojo, 2017).

The growing use of data can also be seen by the increasing number of smart cities. While there are numerous definitions of smart cities, this research refers to the definition provided by Meijer & Bolívar (2015) who include three elements of smart cities: smart technology, smart people, and smart collaboration. They define the smartness of a city as followed: “ _{the smartness of a city refers to} its ability to attract human capital and to mobilize this human capital in collaborations between the various (organised and individual) actors through the use of information and communication technologies” (Meijer & Bolívar, 2015, p.398).

For a smart city to be able to use information and technologies to foster collaboration, an adaptation of infrastructure is important. This can also be seen when looking at the definition provided by Batty et al. (2012, p. 481) , who define smart cities as _{“a city in which ICT is merged with} traditional infrastructures, coordinated and integrated using new digital technologies.”

However, there is not one type of infrastructure that suitable for big data, as it has to align with the activities of the organisation and their use of big data. Data can be used for different application types, such as object evaluation, research and continuous monitoring. The last application type is about collecting data and making it available for (near) real time analysis, for example through data visualisation in ‘dashboards’ where some dashboards are continuously updated. Different application types of data also require different strategies and infrastructures from the organisation, leading to four main types of organisations. For an organisation that mainly focuses on coordination and works project-based, there is often no need for data, and the intensity of data collection and use is low. However, organisations that are more focussed on administration and management, require a high intensity of data collection and usage, where data is often (near) real time so that the data can be monitored continuously. In between are organisations that focus more on research and evaluation, for which the use of data is high, but the collection of data is low. And there are organisations whose main task is registration and documentation. this mainly requires a high intensity of data collection to evaluate objects or subjects (Klievink et al., 2016, p.272).

The structure of the organisation is not only important for data collection and use within the organisation, but it is also important for other organisations. Governmental structures have to change from inward looking towards outwards-looking, with an open data infrastructure that supports the use

(14)

and sharing of free data. Therefore, there has to be coordination and collaboration between different stakeholders (Janssen et al., 2017, pp.190-191). This also means that organisational systems have to be interoperable with each other so that data can be exchanged and used, and services can more effectively operate together (Jetzek, 2015, pp. 91-92). Coordination and collaboration are further related to an organisation’s maturity, which will be discussed in the following section.

2.3.2 Organisational Maturity

As mentioned earlier is smart collaboration one of the elements that defines smart cities. Collaboration and engagement with citizens and different stakeholders, and to create a learning environment that stimulates innovation and knowledge exchange are therefore important aspects of smart cities. For this, open data is necessary as it enables different stakeholders to use and exchange data, leading to more knowledge transfer and in return a smarter city (Meijer & Bolívar, 2016, p.402). Therefore, smart urban collaboration between different actors requires a transformation of not only the internal, but also the external organisation as not only collaboration between departments is needed, but also of the external organisation where smart governance is about adopting a more community-based model where connectivity with different actors is more underlined (Meijer & Bolívar, 2015).

Considering this required transformation, it is not surprising that Klievink et al. (2016, p.273) argue that an organisation’s maturity is one of the three components where uncertainties can arise when dealing with big data. An organisation’s maturity describes the degree to which an organisation collaborates with other public organisations, and whether citizens demands and interests are central within the development of services and policies (Klievink et al., 2016, p. 273).

There are different stages of organisational maturity that describe the degree to which an organisation is oriented towards its customers, is flexible and collaborates. Klievink and Janssen (2009, pp.275-278) describe five stages of organisational maturity, that can be helpful to describe an organisations’ readiness. Organisations in the first stage are called stovepipes, as their systems are inflexible and fragmented, meaning that the departments within the organisation each use different systems and store their data on different applications making it more difficult to share data throughout the organisation. In the second stage, the processes and applications used by different departments within the organisation becomes more integrated. This improves the organisational efficiency, and in turn customer service. The second stage is often a stage of transition as it leads to the third stage, where organisations not only become more integrated within the organisation, but also with other organisations. Here, organisations share a government portal where citizens and clients can interact with multiple governmental organisations through one nationwide portal. This third step, where organisations are more integrated and connected, was also seen as an important transformation for smart governance, as mentioned earlier. Followed is the fourth stage, where not only a portal is

(15)

shared, the different services and activities of organisations are also integrated, and there is inter-organisational integration. Organisations work together in the service-delivery chain, and exchange data, knowledge and technologies with each other. The final stage is the demand-driven and joined up government. Here, organisations focus on the demand of citizens and have a high flexibility. Citizens can request what products and services they need through a single portal or application, which in turn will give a certain set of services or products. This requires a fundamental transition for which collaboration and communication with other relevant organisations is required (Klievink and Janssen, 2009, pp. 275-278).

While Joined-up governance is here described as the final stage for organisational maturity, literature on joined-up governance is also found outside the e-governance field. Joined-up governance can also be defined as collaboration between governmental organisations as a result of an uniformity between their objectives and activities. Here, organisations are coördinated; they communicate in their policies, decision-making and planning, and are integrated in their implementation. This can be horizontal, where organisational collaboration takes place on the same governmental level, or vertical, where there is coordination and integration between different governmental layers (6, 2004, p. 105). The latter is also called multi-level governance. Collaboration is not only an important step in organisational maturity for data readiness, it can also be important to solve cross-boundary problems, such as environmental issues related to climate change. Multilevel dependencies can therefore be a call to work closely together with regional and local governmental organisations. However, collaborating with other organisations to strive for a sustainable transition is not always easy. A hard approach in collaboration might lead to resistance from other organisations, while a soft approach might be more easily accepted but will not directly lead to high results (Martinelli & Midttun, 2012, p. 3).

For an organisation to make the transition from one stage to the other and become more mature, different organisational capabilities are required, which will be further discussed in the following section.

2.3.3 Organisational Capabilities

Organisational and technical capabilities are important for organisations to grow and become more mature (Klievink and Janssen, 2009, p. 281). When dealing with big data, it is therefore important for an organisation to have the required capabilities to extract value from data and to prevent negative consequences from the usage of big data, for instance when data is privacy sensitive (Klievink et al., 2016, p.274).

Different capabilities are needed for organisations in each organisational maturity stage. For organisations that are in the first stage, and are either stovepipes or integrated, technological

(16)

capabilities are mainly important so that they are able to deal with technologies that are needed for their services, or to gain access to new resources that are needed to benefit from new technologies. Technological capabilities can also include capabilities that are needed to protect privacy and regulate the ownership of data, to acquire and use knowledge, or to maintain the needed systems and infrastructure for the use of data (Klievink et al., 2016, pp. 273-275).

Organisations that are more mature and more integrated with other organisations will also need stakeholder capabilities for collaboration and interaction, and to maintain relationships. Service delivery capabilities are needed when an organisation works more demand-driven to ensure that the demanded services are available. Furthermore, transformation capabilities are important to enable organisational change when an organisations wishes to grow and to transform from one stage to the next (Klievink and Janssen, 2009, p.282).

Moreover, as one of the aspects of smart cities is the stimulation of knowledge innovation, additional capabilities are needed that can lead to data-driven public sector innovation. These can be collecting, sharing, combining and analysing data and creating open data, which is necessary to enable stakeholders to share and exchange knowledge (Janssen et al., 2017, p. 190). Open data can further help to increase transparency and to bridge the gap between the public and private sector. This can in turn foster collaboration between public and private organisations to solve societal issues through the use of data, leading to new organisational forms that emerge from data-driven public innovation. Data-driven innovation can then lead to new organisational forms such as cro-creation or crowdsourcing-based innovation with high participation from different organisations or citizen involvement, or lead to new innovations when it comes to public services or the development of policies (Janssen et al., 2017, pp. 192-193).

In order for these types to arise, different factors play a role that can enable or constrain data-driven innovation, such as trust among parties which influences the willingness to collaborate, policies that enable collaboration, the ability to share data or the availability of an infrastructure that supports the use of data analytics and to share data (Janssen et al., 2017, p. 191). These factors show that not only technical capabilities are important, but other capabilities, such as stakeholder capabilities, as well for an organisation to be innovative and to become more data-driven. Moreover, it can be seen that the capabilities also relate to maturity and organisational alignment showing how these three different components are linked to each other and should not be seen individually.

2.4 Hypotheses and Conceptual Model

According to Klievink et al. (2016), organisations should not use big data until they are ready. It is therefore likely that there is a relation between an organisation’s readiness and their use of big data. As data dashboards can be considered as big data according to the characteristics of big data that

(17)

Klievink et al. (2016) provided, it is likely that there is a relation between an organisation’s readiness and their use of data dashboards. An organisation’s readiness is influenced by three components, an organisation’s alignment, capabilities, and maturity.

For an organisation’s alignment, it is important that the infrastructure of the organisation supports the use of big data, sharing and collecting data, and collaboration. Since this research focuses on the use of dashboards that are developed by other organisations, an infrastructure that supports collaborations by allowing data and information to be shared is important, as it is likely to increase the likelihood that an organisation has knowledge on existing dashboards. Hereby it is not only important that knowledge is shared with other organisations, but also within the organisation as people working on energy related tasks and people working with data are not always in the same department. Therefore, the following hypothesis is formed:

H1: The more the infrastructure of an organisation supports the sharing of information and data, the more likely the organisation is to use data dashboards for policies.

Followed, organisational maturity is an important factor for an organisation's readiness. As this paper focuses on dashboards that are developed by other organisations, maturity is likely to be a significant factor as this increases the likelihood that knowledge on dashboards is shared with organisations. It can be expected that the more organisations collaborate and coordinate, the more likely they gain knowledge on dashboards that are relevant to their policy field. This leads to the following hypothesis:

H2: The higher the collaboration and coordination of an organisation with other organisations, and thus more mature, the more likely the organisation is to use data dashboards for policies.

Third, an organisation’s capabilities are important for the use of big data, or, in this paper, dashboards. Based on the same argumentation that is given previously, it is expected that stakeholder capabilities play an important role for the use of data dashboards, as these are also important if an organisation wishes to collaborate with others. Furthermore, it is important that an organisation has the capabilities to use and understand dashboards, for which more technical capabilities are important, specifically capabilities to acquire and use knowledge as this is what dashboards provide. This thereby leads to two hypotheses:

(18)

H3a: The higher the stakeholder capabilities of the organisation are to collaborate and interact with others, the more likely the organisation is to use dashboards for policies.

H3b: The higher the technical capabilities of the organisation are to acquire and use the information provided by dashboards, the more likely the organisation is to use dashboards for policies.

However, according to Kitchin et al. (2015) dashboards can be easy and simple to understand without the help of a specialist, as they visualise data by giving an overview at glance, leading to new insights, increased transparency, and reduced information asymmetry. However, this is dependent on the design of the dashboard as a poor design can increase complexity and make it difficult to understand. An issue that is important is therefore the useability and literacy of a dashboard. This can be dependent on the number of layers, tools and indicators of a dashboard. Too much layers or unclear tools can make site navigation too complex. Design is thereby important. Furthermore, it is important that the data presented on the dashboard is accurate, for which maintenance and regular updates are important. This leads to the following hypothesis:

H4: The better the design and the quality of a dashboard, the more likely the organisation is to use dashboards.

The different concepts of a supporting infrastructure, coordination and collaboration and technical and stakeholder capabilities are derived from the data readiness framework of organisational- alignment, maturity and capabilities. It is therefore expected that these concepts will increase the data readiness of an organisation, and in turn will increase the use of data dashboards. Furthermore, it is expected that a good design of a dashboard will further increase the use of dashboards, and as it can be used without the help of a specialist, the expectation is that a dashboard that is well designed could even increase the use of dashboards independent of an organisation’s readiness. These expectations can be visualised in the conceptual model presented below.

(19)

Figure 1. Conceptual model

Based on these variables, different indicators can be set to measure them in a number of cases. These cases and the selected indicators will be discussed in the research design chapter.

2.5 Summary

In the theoretical framework, an overview was on different concepts that provided useful insights for this research. First, the different benefits and challenges of using dashboards in the public sector were discussed. Followed the data readiness framework was discussed, that will be used to analyse if there is a relation between an organisations’ readiness and their use of dashboards. First organisational alignment and the importance of a supporting infrastructure for collaboration and sharing data in a smart city was discussed. Second, an overview was given of the different stages of organisational maturity, where each stage indicates a higher level of collaboration and coordination. Followed, the different organisational capabilities such as stakeholder or technical capabilities were discussed. Finally, different hypotheses and a conceptual model were provided based on the literature. In the next section, these will be operationalised into different indicators and the different research methods will be discussed.

(20)

3. Research Design

In this chapter the methodology will be described that is used to execute the research project and to analyse the different hypotheses that were given in the theoretical framework. The first section will describe how the hypotheses are operationalised in this research, then the research strategy will be given which describes how the research was carried out. Followed is the case selection, and possible threats and limitations of the research.

3.1 Operationalisation of concepts

In the theoretical framework, different hypotheses were formed based on the literature on the data readiness components organisational alignment, maturity and capabilities, and literature on dashboards. In the table below, it can be seen how these different concepts are operationalised in the elements from the hypotheses, which are then divided into different indicators.

Table 1. Operalisation of concepts

Concept Type variable Definition Elements Indicators

Organisational Alignment

Independent variable The alignment of an organisation’s business strategy, (IT) infrastructure, and IT strategy with the use of big data (Klievink et al., 216, p. 272)

Supporting Infrastructure for sharing data and information

- Organisational

systems are

interoperable so that data can be shared - services can operate together

(Jetzek, 2015)

Organisational Maturity

Independent variable When an organisation collaborates with other public organisations, and citizens demands are central in services and policies (Klievink et al., 2016, p. 273). Coordination and Collaboration with other organisations - Degree to which data and information is shared with other organisations (Klievink and Janssen, 2009) - Communication with other organisations about policies, implementation, planning - Integration between organisations in the implementation of policies (6, 2004)

(21)

Collaboration within the organisation

- Degree to which information and data is shared within the organisation

(Klievink and Janssen, 2009)

Organisational Capabilities

Independent variable Whether an organisation has the required capabilities to use and extract value from big data (Klievink et al., 2016, p.274). Stakeholder capabilities capabilities to: - Degree to which relations with other organisations are maintained

- Degree of

interaction with other organisations - Degree of collaboration with other organisations (Klievink and Janssen, 2009) Technical capabilities Capabilities to: - Collect data - Share data Combine data Analyse data - Create open data (Janssen et al., 2017) Dashboard

design and quality

Independent variable Whether dashboards are easy to navigate, have a clear understandable layout, and the data represented on the dashboard is accurate (Kitchin & Ardle (2016)(Mattheus et al., 2018) - Number of layers of the dashboard - Whether there is a manual present with the dashboard - amount of clicks needed to navigate through the dashboard - Whether the dashboard is regularly updated (Mattheus et al., 2018). Use of data dashboards

Dependent variable Whether

municipalities use dashboards when they are developing policies

- mentioning the name of related dashboards

- mentioning the use of the use of dashboards

(22)

3.2 Case Selection

For this research, a qualitative and comparative small n-case study design will be used. A small-n case study is chosen as these are often more suitable to test new ideas, in this case the relation between data readiness and the use of dashboards, and it allows to isolate the causal relationship that is the focus of this research. The latter can be done with a most-similar design by keeping the relevant control variables the same, while varying the explanatory or independent variables (Toshkov, 2016). Selecting cases based on the independent variable is important, as a small-n case study is too small to select cases randomly as this leads to high chances that it is not possible to identify the relation. In this research, cases are selected based on a co-variational analysis approach by making, a covariate table that shows all the possible outcomes of the results (Blatter and Haverland, 2012). Then, based on the table the appropriate cases can be selected.

For relevant variables, the different governmental characteristics described by Mergel et al. (2018) were looked at, who state that different governmental characteristics have an influence on the ability of a government to engage in agile methods, and thereby are adaptive and make use of new technologies. These characteristics can be size, centralisation, or their engagement in public-private partnerships. Therefore, municipalities are selected that are located in South-Holland, and are similar in terms of organisational size. Furthermore, municipalities are similar in terms of centralisation and engagement in public-private partnerships. The province of South-Holland was chosen because they developed multiple dashboards to help municipalities with renewable energy, and have advisors that work with almost all the different municipalities in the province. This therefore provides an example where the relation between contact with the province of South-Holland can be compared with the use of the dashboard of the province. Based on these characteristics, a covariate table was designed, which can be seen below.

(23)

Table 2. Covariate table with possible outcomes Variable Type of variable Municipality A Municipality B Municipality C Municipality D Organisational size

control variable around 500 employees around 500 employees around 500 employees around 500 employees Organisational capabilities Independent variable

High High High Low

Organisational alignment

Independent variable

High High Low Low

Organisational maturity

Independent variable

High Low Low Low

Use of data dashboards

Dependent variable

? ? ? ?

In order to select municipalities that vary on the independent variables, a survey was carried out which was send to all the municipalities in the province of South-Holland to get an indication of how they score on each of the data readiness variables. Then, based on the results, different municipalities were selected to conduct semi-structured interviews.

These interviews are conducted with public servants from the province and the municipalities. In the province, employees are chosen that give advice to municipalities regarding the heat transition, as these are expected to have an overall overview of the collaboration and communication between municipalities and the province. Employees from municipalities are selected that either work on the energy transition or with data and dashboards. Employees working on the energy transition are relevant for the research as the focus of this research is whether they use dashboards when they are working on policies related to energy. The employees that are working with data or dashboards are selected as these are more likely to have knowledge on the organisational alignment and capabilities as they have more expertise on the different systems and that are used and how data is shared within the organisation as they work with data themselves. More information on how the respondents are selected will be given in the next section.

(24)

3.3 Data collection

3.3.1 Survey

To collect the data for the research, different data collection methods were used. First, a survey was send to municipalities in South-Holland, asking about their IT department, infrastructure, collaboration with other municipalities, and whether they use one of the three data dashboards that the province of South-Holland provides in the field of sustainable energy, or other dashboards. The dashboard provided by the province are the ‘warmtetransitieatlas’, the ‘signaleringskaarten bodem en ondergrond’ and the ‘zonnewijzer’. For the survey, all municipalities in the province of South-Holland were contacted (Provincie Zuid-Holland, n.d.-a). This happened in two ways, first all municipalities were contacted through their info email address or through a general contact form of the municipality with the question to spread the survey to the relevant employees. Second, a mailing list from the province of South-Holland was used that contained different public servants from municipalities that are involved in sustainability and/or renewable energy. In the mail, the employees were asked to share the survey with colleagues from the energy department and from the IT department.

In the survey, the questions were mainly based on the data readiness theory from Klievink et al. (2016). First, to get a sense of the size of their organisation questions were asked regarding the number of employees in the municipality and the number of employees in the IT department. Furthermore, questions were asked about how they view big data and what they think the municipality can gain from big data in order to get a sense on their perspective regarding the use of big data.

Then, questions derived from the theory where asked. First, to investigate the organisational alignment, they were asked about their main organisational tasks, and whether they collect and/or use data for these tasks.

To look at the organisational maturity of the municipality, questions were asked whether the municipality shared information and or tasks with others and in what extent, and whether other organisations shared information with their municipality and their satisfaction in this.

To asks about the organisational capabilities, questions were asked regarding the main use of (big) data, and whether (big) data, and or data dashboards are used for policies. Then, respondents were asked to what extent they agreed with statements regarding the capacities of the municipalities to design, develop and maintain a good IT infrastructure, whether this offers structure to the organisation to help with decision-making processes, and whether there is collaboration with other parties.

28 public servants responded to the survey, of which 14 completed the survey. This can be seen in the table below. Based on the respondents, six municipalities were selected for interviews.

(25)

Table 3. Survey respondents

Survey respondents Completed the survey Date

Zwijndrecht Yes 23-04-2019

Capelle aan den IJssel Yes 24-04-2019

s Gravenhage Yes 24-04-2019

Sliedrecht Yes 25-04-2019

Leiden Yes 25-04-2019

Capelle aan den IJssel Yes 25-04-2019

Vlaardingen Yes 26-04-2019

Alphen aan den Rijn Yes 26-04-2019

Vlaardingen Yes 26-04-2019

gemeente Westland Yes 26-04-2019

Leiden No 29-04-2019 Zuidplas Yes 29-04-2019 Voorschoten en Wassenaar No 30-04-2019 Hardinxveld-Giessendam No 02-05-2019 Zoetermeer No 02-05-2019 Vlaardingen No 02-05-2019 Schiedam No 02-05-2019 Leiden No 02-05-2019 Den Haag No 02-05-2019

Alphen aan den Rijn No 02-05-2019

Nissewaard No 03-05-2019

Pijnacker-nootdorp Yes 03-05-2019

Zoetermeer No 06-05-2019

(26)

Rotterdam Yes 06-05-2019

HLTsamen Yes 07-05-2019

Delft No 07-05-2019

Capelle aan den IJssel No 09-05-2019

3.3.2 Interviews

Based on the results from the survey, municipalities were selected for an interview to gather more information and better understand what influences the use of data dashboards. Based on the results of the survey, cases are selected that vary in the independent variables. This led to a selection of six municipalities, Alphen aan de Rijn, Cappelle aan de IJssel, Vlaardingen, Westland, Pijnacker-Nootdorp, and HLTsamen, which is an organisational collaboration between Hillegom, Lisse and Teylingen, that all have around 500 employees.

Here, the municipalities Alphen aan de Rijn, Cappelle aan de IJssel and Westland, scored relatively high on data readiness in the survey, and the municipalities Vlaardingen and HLTsamen scored relatively low on data readiness in the survey. Pijnacker-Nootdorp scored neither high or low on data readiness. At each municipality, the aim was to have at least two interviews, one with an employee that worked on renewable energy, and one employee that worked with IT and/or data to get more insights in the organisation’s data readiness, and to control the answers that were given in the survey. However, at the municipalities Alphen aan den Rijn and Vlaardingen, only one interview was held.

In the interviews, the questions were based on the indicators described above. Respondents were asked about their knowledge on dashboards related to the energy transition, their use of it, their collaboration and coordination within the organisation and with other organisations, whether there were different systems in the organisation and if data could easily be shared, etc.

Furthermore, two interviews were held with public servants of the province of South-Holland who give advice to municipalities on issues related to the energy transition, and frequently visit the municipalities. One of the weekly meetings where all the twelve advisors at the province are present was also joined to gain general information on the municipalities, the energy transition and the dashboards that helped to sharpen the research questions. In the table below, an overview is given of the interviews that were held for this research.

(27)

Table 4. Overview of interviews

Organisation Respondents Date Duration Abbreviation

Province of South-Holland Employee of the province of South-Holland - proces advisor 20-06-2019 38m Interview 1a Employee of the province of South-Holland - proces advisor 11-07-2019 50m Interview 1b

Alphen aan den Rijn

Employee of municipality Alphen aan den Rijn - Data scientist

12-06-2019 1h 16m Interview 2

Capelle aan den IJssel

Employee of municipality Capelle aan den IJssel

12-06-2019 1h 20m Interview 3a

Employee of municipality Capelle aan den IJssel 12-07-2019 1h 12m Interview 3b HLTsamen Employee of municipality HLTsamen 19-06-2020 54m Interview 4a Employee of municipality HLTsamen 09-07-2019 56m Interview 4b Pijnacker-Nootdo rp Employee of municipality Pijnacker-Nootdo rp 26-06-2019 49m Interview 5a Employee of municipality Pijnacker-Nootdo rp 08-07-2019 1h 1m Interview 5b Vlaardingen Employee working for the municipality of Vlaardingen

(28)

Westland Employee of municipality Westland 21-06-2019 48m Interview 7a Employee of municipality Westland 09-07-2019 52m Interview 7b

3.3.3 Informed consent

The respondents of the survey and interview where both informed of the goal of the research. For the survey, respondents would see a message informing them that the answers they provided would be used for the thesis, and asked them permission to use their answers. If respondents did not accept, they would be led to the end of the survey. Respondents of the interviews were send an informed consent form through mail before the interview took place. Here, the goal and purpose of the research were stated, and they were informed on how their answers would be used, emphasising that their answers would remain confidential as their names would be anonymised, and whether they would give permission for the use of their answers in the project.

3.4 Research limitations and threats

For the survey, the population consists of 52 municipalities. This means that for significant results, almost all municipalities have to respond to the survey. However, as survey response rates are typically low, this is unlikely. This means that the population of the survey is too small for significant results. However, these results are compensated by interviews with municipalities.

A second threat of the research methods is that with surveys and interviews, results are sensitive to misinterpretation. Especially with surveys, respondents can misinterpret a question and you cannot explain a question better. It is possible to elaborate further on a question in interviews, but misinterpretation is still possible and a respondent might give a different answer than is actually the case.

A third threat of the research can be that when asking questions regarding data readiness, employees could feel more confident about the organisation than might be the case, and could say that they collaborate and share a lot of information and data, even though this might not always be the case.

Therefore, these threats are taken into consideration for the analysis and conclusion, and a variety of municipalities are interviewed to get a broader overview.

(29)

4. Empirical Findings

In this chapter, an overview will be given of the obtained results from the interviews that were conducted with the employees from the Province of South Holland, and the different municipalities. 12 interviews were conducted in total of which two were held with employees of the province of South-Holland who give advice to municipalities on the energy transition. In the municipalities Alphen aan den Rijn and Vlaardingen one interview was held with someone who works with data, in Vlaardingen this was someone who works for the municipality, but is not officially an employee of the municipality. In each of the municipalities Capelle aan den IJssel, HLTsamen, Pijnacker-Nootdorp, and Westland, two interviews were held; one with someone who works on policies regarding the environment or sustainability, often as a policy advisor; and one interview was held at each of these municipalities with some who works as a data scientist or analyst, or works with geographical information (systems).

4.1 Province of South-Holland - Energy Advisors

4.1.1 Organisational Maturity

The idea for the energy advisors came from a motion from the provincial executive with the argumentation that a lot of municipalities, and mainly the smaller municipalities, might not have the capabilities to deal with the energy transition. Therefore, they decided to support the municipalities in these tasks till 2021, when the transition vision for heat has to be finished (Interview 1b). In this vision, municipalities have to state how and when they will make the transition towards low emission heating within their municipality (RVO, n.d.). Initially, the idea of energy advisors was received skeptically by municipalities. This was also due to the more traditional relation between the province and the municipality where the province has a more supervision role of regulating, and stating what is what isn’t possible for municipalities (Interview 1b). Despite this, the relation between the province and the municipalities is improving and currently the advisors of the province visit around 80 percent of the municipalities in South-Holland (Interview 1a).

The energy advisors have different tasks. They advice the municipalities, and maintain a network function by stimulating the knowledge exchange and collaboration between municipalities and different stakeholders. This can be done by sharing information and good practices between municipalities themselves, and organising knowledge exchange meetings for municipalities. They further pass down information to the regional energy strategy (RES), which is a project with different municipalities in the region, and they sit in ‘steering groups’ of the RES together with different stakeholders (Interview 1a)(Interview 1b).

(30)

4.1.2 Organisational Alignment

According to one of the advisors, a main difficulty with the energy transition is that municipalities are organised per sector. This can be hindering as different departments are often involved, for example in issues related to neighbourhoods, besides the department that works energy. While departments used to focus on their own work, the energy transition requires collaboration between different departments. Therefore, municipalities are asked to reflect upon themselves and think if they are organised well enough to execute this task collaboratively at the municipal-, regional-, and provincial level. The response by municipalities to this varies, some will think more about it and think it is a strong point, while others respond more nonchalant (Interview 1a).

4.1.3 Organisational Capabilities

One of the advisors think that when they started with this project, the municipalities didn’t have the time, or the capacity, to deal with the energy transition alone. Therefore, for the past one-and-a-half-year, the focus was mainly on the organisation of the municipality and how they can improve the capacity so that more people in the municipality are available to work on the energy transition (Interview 1b).

It was also mentioned that mainly the bigger municipalities have the capabilities to work with data, and that smaller municipalities sometimes don’t have enough knowledge to work with raw data. An important aspect is that multiple people are often needed to discuss the data and to interpret it (Interview 1a). In smaller municipalities, there sometimes is one person who can work with data, or with GIS (Geographical Information System) when it is about dashboards, or there is someone available from the environmental service (omgevingsdienst). The environmental service does the monitoring and licensing regarding the environment in execution of the province and the municipalities. They can have an important role when it comes to data exchange as they have a lot of technological knowledge and can extract information from data and provide this to the municipality (Interview 1b)(Omgevingsdienst, n.d.).

Currently, consultancy organisations are often needed for the supply of information, which is not always ideal as a problem with consultancy organisations is that there isn’t always clarity about the boundary conditions that affect the outcomes of the study, such as the amount of years it takes before a heat pump has to be replaced, which determines the cost of the heat pump.

For the supply of information, consultancy organisations are needed. While the information consultancy organisations use for the reports sometimes comes from the municipality, they don’t always work with it themselves. However, one of the advisors does think that a strong point is that municipalities are able to ask the right questions and can critically assess information and judge

(31)

whether the information they get is correct. Even though they do not always have the technical knowledge. (Interview 1b).

4.1.4 Use of Dashboards

The province has developed different dashboards to aid municipalities with the energy transition. These are also shared with the municipalities through the advisors, who sometimes give additional explanations of the dashboards, such as with the ‘warmtetransitieatlas’ by indicating what the different options and costs are and what certain indicators mean. However, one of the advisors does think that municipalities in general understand the dashboards, and are doing quite well in this by using different maps and thinking about how they are going to do things (Interview 1a).

However, there are still different issues with the use of dashboards, for instance with the abundance of it. Besides the ‘warmtetransitieatlas’ from the province, PBL also develop a similar map, and the national programme RES published different datasets as well (Interview 1a). It is problematic that the information between these can vary, and that not all the dashboards are always useful. For instance with the dashboard from PBL. There, the starting point is the district division according to CBS, which is based on zip codes, instead of clusters that are based on the quality of houses which is needed for the energy transition as newer houses are more energy efficient (Interview 1a)(Interview 1b). Furthermore, there are also a lot of different criteria. For example when thinking about an alternative heat system, then you also have to think about the social class of citizens in a neighbourhood. Additional criteria are continuously added to dashboards and maps, which makes the dashboard more complicated.

4.2 Alphen aan den Rijn

4.2.1 Organisational Maturity

When talking with a data scientist from the municipality Alphen aan den Rijn, the employee mentioned that within the organisation, information is still spread across different departments and isn’t centrally organised in one place, nor is it always directly accessible. If you need information from another department, you have ask it from someone from another department. Then it is in general always shared, but you do have to actively go after. However, the data scientist is building an ‘omgevingsatlas’ (environmental atlas), an integrated viewer where different datasets from different themes are added. They also have such an atlas for energy, with opportunities for the heat transition, riothermie, limitations for wind turbines, and more components.

In terms of data use outside the organisation, the data scientist collects around 80% of his data from other organisations, these are often public organisations or publications from consultancy organisations, or open data portals such as ‘national geo’ registry, where bigger organisations and also

Data-Driven Policy Making for Sustainable Energy: The impact of data readiness on the use of dashboards for renewable energy policies