Smart Amsterdam - Defining the Future : Clarification of Smart City Objectives Through User Stories

Clarification of Smart City Objectives Through User Stories

A thesis submitted for the degree of Bachelor of Civil Engineering

University of Twente

Faculty of Engineering Technology

Report by

Casper Thostrup Oxand supervisor Msc. B.R.J. Visser UT supervisor Dr.Ir. R.S. de Graaf

Research was conducted over 3 months

From the 9th of April to the 8th of July

Preface

Throughout my bachelor’s degree, my interest in process-oriented working methods such as System Engineering increased greatly. Especially the fourth module of University Twente’s civil bachelor degree where we had to design function trees to understand a garage and see it as a system was highly interesting to me. In the 6

module of the degree, one of my good friends and I embarked on a project using System Engineering and Mechanics to design a river-sluice, which both satisfied structural and customer requirements. From this project, I realised that System Engineering was something that I wished to incorporate into my professional life later. In the minors leading up to the Bachelor thesis phase of the third year, I learned of the Smart City concept. Although I learned a lot about the idea it left me quite frustrated also. Are these just buzzwords? How does the municipality know if these systems are wanted? When can one truly say that a city is smart?

Having had previous contact with Ben Visser from Oxand at a lunch lecture presentation by the company. I knew that they, as a Civil Consultancy firm, would be a good fit for me with their focus on asset management, process architectures and tender consulting. Ben is at the time of writing consulting the municipality of Amsterdam for their asset management work as well as Smart City developments. This amalgamation of topics led to the research question this project focuses on.

Firstly, I wish to thank the two people that guided me through the jungle that is writing a thesis. Ben was a great guide for keeping within the set boundaries of the work and helping me keep spirits high when I felt like I was losing my way. Dr de Graaf added the critical component of ensuring that my work was up to scientific standard. I have a habit of following my own path, which is not desired when following the (rightfully so) set way of working as a scientist. Both gave me great insight and broadened my horizon further and for that, I wish to thank them.

Before starting my degree, I was incredibly nervous. Studying had never been my strong suit and knew this would be a battle for me. Luckily, throughout my three years at the University of Twente, I met some incredible people who kept my spirits high, helped me overcome my weaknesses and were fantastic to work with. I firstly wish to thank my study friends, which made the experience of studying hilarious as well as fulfilling. Our group, Wonk Stonks VOF, taught me how a great project team can work. My student house, Los Chichis, I want to thank for making sure I did more than just study in student time. They were there as a cushion in the hard times, and an amplifier in the great times! Lastly, and most importantly I wish to thank my parents for the advice and guidance they have provided, before and throughout, my study. Without my fathers, sometimes difficult to swallow honest opinions, I would have been studying Architecture without the technical knowledge I have now gained. My mother helped me through some of the most difficult time and gave me a good kick to keep working! The same counts for my friends Peter and Andrew who were the best housemates one could have wished for when I needed a helping hand. Thank you, my lovely friends and family.

Enjoy the thesis!

Casper Thostrup, 02/06/2021.

Casper.thostrup@gmail.com

Executive Summary

Smart City development is an increasingly important topic in the municipal context. However, a consensus amongst industry leaders and governmental bodies on the definition of Smart Cities has not been established. The clarification of goals within municipalities has subsequently also been difficult. Cities want to be smart, but it is unclear what functionality they want and what citizens desire. This work investigated to which extent User Stories can be utilized for the clarification of Smart City objectives in Amsterdam. The main research question being: To which extent can User Stories be utilized for the clarification of Smart City objectives in Amsterdam?

The User Story method begins with the identification of user roles of smart systems, which is followed by the writing of stories; here, desired functionality and the reasoning behind wanting that functionality is uncovered. The next step is filtering stories through fourteen criteria which streamlines their structure. The final step in the process is the testing of stories, to ensure stories have been implemented as well as uncovering more stories.

This research has investigated the potential of User Stories through interviews with projects, businesses and municipal experts working within Amsterdam’s Smart City themes. 6 semi-structured and 4 structured interviews took place. In the structured interviews, interviewees were asked per specific step of the User Story method whether they believe this could aid in the context of Amsterdam Smart City development. The insights per step of the method then paved the way for answering the main research question. The results from these interviews have given the insights that User Stories can aid in the clarification of objectives in municipalities within the context of Smart City development, with some notable comments on sub-steps and main steps of the method. The identification of user roles could aid in Smart development. The use of extreme characters should be included with caution if added and the use of personas was highly recommended by all, but one can never fully step into anyone’s shoes. The writing of stories can aid in capturing the functionality of Smart City objectives, although it is likely not possible to capture all functionality and writers may struggle to think towards the future rather than reflecting on their past. The filtering of stories was considered useful in Smart City development but may take a lot of time to perform. The testing of stories was considered beneficial, but the use of prototypes was recommended to uncover more stories and improve the process of ensuring stories have been implemented.

With the determination that User Stories can aid in the clarification of Smart City Objectives in Amsterdam, the following recommendations can be made. Other methods exist for uncovering functionality, such as Customer Journeys. These could be compared to the User Story method to evaluate which may be the most effective. The acceptance stories step has some hurdles in its design with regards to ensure that stories have been implemented.

New ways of conducting acceptance testing of stories could be researched. A case study could be conducted in

Amsterdam to further investigate the potential of the method. Perhaps comparing current Smart City systems in

Amsterdam to the functionality uncovered by stories. Many interviewees noted that engagement may be difficult for

citizens as they are not used to the User Story method. It is therefore recommended to investigate ways of improving

participation and making writers feel at ease when writing stories.

Table of Content

Preface 2

Executive Summary 3

Table of Content 4

Table of Figures 6

Table of Tables 6

1. Introduction 7

1.1 Problem context 7

1.2 Project context 7

1.3 Explanation of concepts 8

1.4 Aim of this research 9

1.5 Research Questions 9

1.6 Scope 10

1.4 Reading guide 10

2. Methodology 11

2.1 Semi-structured interviews 11

2.2 Stuctured interviews 11

3. User story method 12

3.1 Identification of users 12

3.2 Gathering Stories 13

3.3 Filtering stories 15

3.4 Acceptance testing stories 16

4. Gathered results from interviews 17

4.1 Contextual semi-structured interview results 18

4.2 Structured interviews 19

5. Analysis of results 21

5.1 Identification of user roles 21

5.2 Gathering stories 22

5.3 Filtering stories 22

5.4 Acceptance testing stories 22

6. Discussion 23

7. Conclusion 24

8. Recommendations 25

References 26

Appendix A: Interview questions 27

Contextual semi-structured interview questions: 27

Structured interview questions: 27

Appendix B: Gathering user roles 28

Brainstorming initial set of users 28

Organising initial set 28

Consolidate roles 28

Refine the roles 28

Persona creation 28

Appendix C: Cohn user story principles 29

Independent 29

Negotiable 29

Valuable to users or costumers 29

Estimable 29

Small 29

Testable 29

Appendix D: User Story Quality Framework Criteria. 30

Syntactic 30

Semantic 30

Pragmatic 31

Table of Figures

Figure 1 Story-Writing Workshop ...16 Figure 2 Quality User Story framework ...17

Table of Tables

Table 1 Conducted interviews ...19

1. Introduction

In this part of the report, the topic and problem will be introduced. Further elaboration on specific terms and themes will be provided. This is subsequently followed by the objective of this thesis, and the research questions that will be answered to reach the aim of this thesis. The scope limitations are described to tailor the expectations for both the researcher and reader. This chapter concludes with a reading guide for the rest of the thesis.

1.1 Problem Context

The Smart City is a worldwide phenomenon being both pursued by governmental bodies as well as private enterprises (Hatch, 2012; Hollands, 2020). Smart City technology and data management are being implemented within both existing and greenfield cities

; for the tackling of social and structural problems that are becoming increasingly prominent with the rise of urbanisation (United Nations, 2018). Although the concept of Smart Cities is already, and increasingly popular, a consensus amongst both experts and governmental institutions on what a Smart City is, has not been established. One definition by Dirks and Keeling (2009) for IBM is: “A smarter city is one that uses technology to transform its core systems and optimize the return from largely finite resources.” The British Standards Institute (2014) defines a Smart City as: “Effective integration of physical, digital and human systems in the built environment to deliver a sustainable, prosperous and inclusive future for its citizens.” Even more challenging has been the assessment of Smart Cities and at what stage a metropolitan truly becomes ‘smart’. Giffinger et al. (2007) use 74 indicators based upon 6 characteristics: Smart -Economy -People – Governance, -Mobility, -Environment and -Living. Lazaroiu and Roscia (2012) use 18 indicators that take a more statistical approach to the rating of Smart Cities. These methods can provide goals within specific smart themes for municipalities to strive towards but do not consider whether the citizens of a city are satisfied with the provided systems. One of the factors of Giffinger et al. (2007) is the availability of sustainable, innovative and safe transport systems. But what would these systems be? What functionality would inhabitants want from this transport? At what stage is the performance good enough for the users of the transportation? Often Smart City systems are innovative and therefore no previous examples of required functionality exist. This often makes it difficult to know what a city or company must fulfil to satisfy inhabitants with regards to Smart themed systems.

1.2 Project context

One city which has been engaged with Smart City development is the municipality of Amsterdam. Officially a city since 1342 (Gemeente Amsterdam Stadsarchief, 2016), Amsterdam is actively pursuing modern Smart City solutions for solving its current and future challenges. The city is presently working with 20 partners, which include governmental bodies, research institutes and companies that are pursuing the following four themes with regard to Smart City development (Municipality of Amsterdam, 2021):

1. Transition from a linear economy to a Circular Economy;

2. Transition to a Digital City with responsible data-driven innovation;

3. Transition to renewable, decentralized and variable Energy;

4. Transition to smart, clean Mobility solutions for people, goods and services.

The challenge within these four themes is how one can define what these would look like in a real city context. What does a circular economy in a municipality entail? Would citizens recycle everything and if so, how would they do this?

What would the day-to-day process look like?

Oxand, a civil consultancy firm based both in France and The Netherlands, which focuses on asset management and contract tendering is currently working with the municipality of Amsterdam on multiple different projects. Their collaboration also includes exploring Smart City development for Amsterdam.

Oxand is interested in exploring whether using User Story methods can determine the functionality that smart systems must fulfil to satisfy users, as well as clients of the system. Using such a method could lead to the goals and objectives of the city being more clearly defined. This could have the benefit of reducing development time, as municipalities will have a better understanding of what inhabitants desire in their systems, and therefore, possibly also increase the participation in the use of the system.

1 Greenfield cities are smart urban areas which are developed without any legacy infrastructure. This allows

developers to have a clean slate and full creative freedom.

1.3 Explanation of concepts

1.3.1 What are User Stories?

User Stories are typically a method for uncovering desired features from users in the context of software. It places less focus on requirement lists, and instead, concentrates on discussions with expected users on what features they would like to have included in the software. One of the most cited experts in this field, Mike Cohn, describes User Stories as follows: “A User Story describes functionality that will be valuable to either a user or purchaser of a system or software.” Cohn (2004) further describes three important aspects that stories are composed of:

• A description on paper of the story, which purpose is for planning and as a reminder.

• Conversations with users or purchasers, that help expand the details of a story.

• Testing certain functions where the results are documented, which can then be used to verify whether a story has been completed.

User stories are not commonly used in the engineering field. The author was not able to find any work directly relating to civil engineering, or Smart City project development, that utilized User Stories.

1.3.2 Defining terms

A lot of new terms for readers will be used within this and the following work. Here these words will be explained and create a mutual understanding between reader and author.

Stories

In the context of this work, stories are small written sentences that describe a desired feature. One of the most cited works in the field structures these sentences as follows: “As a < type of user >, I want < some goal > so that < some reason >” (Cohn, 2004). These stories, therefore, do not only convey the desired functionality in the form of a goal but also who would like this functionality as well as the reason for it. It is important to note that a story is not a hard- set requirement, rather, it describes what the user would want.

Users/Persona

Users and personas are the perspectives from which stories are written. These perspectives, therefore, do not mean that users have to be directly involved. A different person could place themselves in the shoes of a user, although it is preferable to utilize real users. Users capture the stakeholders which will be interacting with the system in some way. A persona is an addition to the typical description of a user, it adds a background story, an image, an educational background etc. (Hudson, 2013). This helps story writers to step in the shoes of somebody else.

System

User stories are typically used in software development, and authors of works contributing to the field, therefore, usually talk about software when relating the stories to something. This work will refer to the application of User Stories to ‘systems’, as this better encompasses all the elements of Smart Cities.

Operator

Within this research, the operator is the one applying the User Story method to some objective. This is important

as Cohn (2004) often in his work explicitly states that the stories and tests must be performed by the user and

customer of the system. An operator merely holds the workshops, ensures stories are in the correct format and are

stored correctly. In many of the User Story works, the operator is described as the developer. The developer is also

responsible for the creation of software in most User Story contexts. Within this work and the following that does

not have to be the case. Therefore, the operator term is used to describe the person which applies the User Story

method in the Smart City context.

1.4 Aim of this research

From the Project context, it becomes clear that Smart City, as a concept, is difficult to define. This makes

understanding what a Smart System must do to satisfy citizens also challenging, and hence implementation success of such a system can be limited. This work will attempt to contribute to Smart City development by looking at the extent to which User Stories could be applied to identify the functionality of Smart systems. The research objective can therefore be formulated as:

Explore the extent to which User Stories can aid the clarifying of objectives of municipalities with regards to Smart City development.

Within this research, the different themes of Smart City development are based on the four themes described by the municipality of Amsterdam in 1.1 Project Context. Namely the transition: from a linear economy to a Circular Economy; to a Digital City with responsible data-driven innovation; to renewable, decentralized, and variable Energy;

to smart, clean Mobility solutions for people, goods and services. The research will look at developments within Amsterdam, and therefore, other themes or projects outside of the municipality are not relevant in the scope of this research.

1.5 Research Questions

To fulfil the research objective, the following research questions will be posed to allow the researcher to complete the work in the allowed time for the bachelor thesis. These questions have been set in consultation with the client, Oxand. To complete the research objective the following main question research question is posed:

To which extent can User Stories be utilized for the clarification of Smart City objectives in Amsterdam?

To answer the main question of this research, three sub-questions will be posed which will give insight into whether the method can be useful in the given context.

1. What is the state-of-the-art of User Story method?

Although the most cited work in the field of User Stories is by Cohn (2004), many other works have been created in the years after. To analyse the effectiveness of User Stories in the context of Smart City objectives it must be found what the state-of-the-art process is.

2. To what degree can user roles successfully assist in understanding and capturing stakeholders of Smart Systems in Amsterdam?

The effectiveness of user roles has not been determined in a municipal context. The user stands central in the User Story method. therefore, if user roles cannot aid in understanding and capturing stakeholders, story writing from the perspective of such users will be ineffective.

3. To which extent can stories effectively aid in capturing the desired functionality of Smart City goals in Amsterdam?

Whether the use of stories can effectively be applied in the context of a municipality is yet unknown. In this research it will be investigated whether stories can aid in this context.

Based on the results of these three questions the main question can be answered and a recommendation on the

implementation of this method can be given.

1.6 Scope

This research was conducted over three months and therefore some elements had to be constrained. Here the limitations with regards to each sub-question can be found. Firstly, a general constraint should be given. In this work, it was not possible to implement the User Stories method in a real case from start to finish to assess its viability.

Because of this, other methods were used to evaluate the extent to which User Stories can clarify Smart City objectives. These methods can be found in the Methodology section of this report.

User stories is a method created mainly for software development. Due to this some of the works that contribute to the User Story method are based on software solutions such Lucassen et al. (2017), which uses software tools to produce an automated approach that creates a conceptual model based on high-quality User Stories. In this project scope, the tools themselves will not be considered but their underlying principles will. What criteria the software considers when analysing the quality of stories for example.

To answer sub-questions two and three the scope will be held within the context of this project, Smart Amsterdam.

This means that the sub-questions will be answered in the context of Amsterdam and the Smart projects being developed there. What is considered a Smart project is therefore determined by the four themes described in 1.1 Project Context. This could result in the possibility that User Stories may behave differently outside of the context of Smart City development in Amsterdam. As one city may have a different perception of what they deem as ‘smart’.

The User Story method also includes a validation stage which is named the acceptance testing stage (Cohn, 2004).

Certain parts of this will be used in the research, as this part also supports the exposing of underlying assumptions that users may have when they wrote stories. The actual testing, will, however, not be considered part of the scope as the research is only interested in clarifying the goals of a Smart City, not ensuring they have been implemented properly. This means the creation of the system, and the testing to see if stories have been implemented to the user’s satisfaction will not be considered.

1.4 Reading guide

This work starts with describing the methodology that will be applied for answering the sub-questions and main

research question. After the findings of the literature study are represented, which describe the User story method,

and how it would be applied in a Smart City context. This is then followed by the Gathered results from interviews,

where the results of conversations with municipal experts and businesses representatives will give insight into the

feasibility of the User Story method. This is then followed by an analysis of the results with regards to each User Story

method step. The results and limitations are discussed, and conclusions are drawn about the set sub-questions and

main research question. To finalise, recommendations are given about what could be done to further evaluate the

effectiveness of the User Story method in the context of Smart City development.

2. Methodology

This research starts with the application of desk research to find the current state-of-the-art User Story method.

The method that the author finds, will in the scope of this research be considered the state-of-the-art and will then be written out in its stages allowing the reader to understand the method. This is then followed by interviews with municipal experts as well as company representatives working on developing smart systems in the city of Amsterdam. Two different types of interviews will be conducted which are explained below. The second, and third sub-question, will be answered through conducting at least 10 interviews; one semi-structured and one structured interview per smart theme of the municipality, which are described in 1.2 Project context as well as with municipal experts. What smart theme a project or organisation falls in was deduced by showing the interviewees the four themes described in the project context and enquiring which they believe fits best with their work. The number of interviews required was determined through the work of Guest et al. (2006) and Galvin (2015), which both found that 12 interviews were found sufficient, where little new information was found with the addition of more interviews. Unfortunately, due to time restrictions, the limited number of smart experts and businesses working in Amsterdam and the challenge of acquiring interviewees, 10 were conducted, 4 structured and 6 semi-structured.

2.1 Semi-structured interviews

These interviews are less formal and semi-structured. Their purpose is to give further context to the problems experienced by the interviewee with regards to Smart City development in the municipality. This will give the researcher further insight into how the User Story method could aid in the clarification of Smart City objectives. The questions that were posed can be found in Appendix A: Interview questions. These interviews will be transcribed, and noteworthy elements will be discussed.

2.2 Structured interviews

Although the context setting interviews are of great use for understanding the current situation with regards to Smart City development, they do not specifically focus on what elements of User Stories the previously mentioned municipal experts and smart businesses could consider useful in the Smart City context. Due to this, specific questions with regards to the different stages of the User Story method were asked. Firstly, an explanation of the User Story method stage was given after which the interviewees were be asked to state whether they believe this step could be of use in the context of Smart City developments. These questions were related to the main four stages of the method namely: Identification of users, Gathering stories, Filtering stories and Acceptance testing stories. They were then asked to elaborate upon their answer.

The results from these interviews allow for a pattern to become visible with regards to the viability that companies as well as municipal experts see in the method and therefore assess to what extent these User Stories can be utilized for the clarification of Smart City objectives. The list of questions can be found in Appendix A: Interview questions. The results will first be shown in the section, Gathered results from interviews, after which they are studied in the section Analysis of results.

All interviews were between thirty minutes and an hour dependent on the interviewee’s prior knowledge of

the method, the time they had available as well as their openness to talk. Each participant was explicitly asked

for permission to be recorded; the recording was done through Microsoft Teams or Zoom. All interviews were

transcribed and can be acquired by contacting the researcher.

3. User story method

In this section of the research report, the state-of-the-art method for the User Stories method is described. This section, therefore, answers the first sub-question of the thesis: What is the state-of-the-art of User Story method?

The method was uncovered by literature research. The method is described in four main steps: Identification of users, Gathering stories, Filtering stories and acceptance testing stories. Three of the steps were based on the work of Cohn (2004) with additions made by other authors. The filtering stories step was added through the work of Lucassen et al. (2015). The steps are chronological and should be followed in the order that they are presented.

3.1 Identification of users

Cohn (2004) suggests organizing brainstorming sessions with customers, which in this context is the municipality, as well as developers. In the context of Smart Cities, the customer would be the municipality, but these can differ and depend on who requests the creation of the system. Developers could be central government, contractors, and councillors but this is also affected by the said project. The most important element is that the group is diverse and represents as many involved stakeholders as possible. Hudson (2013) suggests that users themselves, as well as employees not ranked highly in the executing organisation, should be included. They can give context that experts in a field may overlook or assume is not important. He argues further that technologists

could struggle with

understanding a problem if they try to derive this from their perspective of the system. People who are not so active in the technology field may not assume the same things as a technologist. The exact steps of this stage can be found in Appendix A: Gathering user roles. In principle the below four steps are performed from Cohn’s method with an additional fifth added from Hudson’s work:

1) Brainstorm an initial set of user roles;

2) Organizing the initial set;

3) Consolidate roles;

4) Refine the roles;

5) Creating of personas.

Most importantly is that participants are not judged on their suggestions and that open discussion is allowed. Once consolidation and refining of roles are actively pursued discussion and confrontation may take place. Djajadiningrat et al. (2000) add to this method by suggesting the introduction of extreme characters in the set of user roles.

Although Cohn (2004) is not in favour of using this in all situations as it could lead to unnecessary stories being implemented, it is useful in the context of a municipality. Many different types of people exist and should be represented when designing anything within a municipality. It is not ethical to disregard the less physically able or groups, such as children, with less ability and power to speak their mind. Because of this, it is suggested that at least a few extreme roles are added to ensure equal representation of all people. This way one can reduce resistance to the implementation of Smart Systems.

2 Technologists are scientists or engineers that specialize in a particular technology field.

3.2 Gathering Stories

Now that users have been identified what they find important in terms of functionality of the smart system must be found; this is done by the writing of stories. Cohn (2004) has for a long time set the standard for what a User Story must be. The structure of a story should be as follows: As a <role>, I want to <action>, [so that <benefit>]. An example of such a User Story is: “As a Cyclist, I want to pump my bike so that I can cycle more efficiently.” A good guide for the size of a story is that it should be able to fix within a post-it note, this limits the size but still allows for the use of extra notations. There are a few principles Cohn (2004) suggests holding to, these can be found in Appendix B: Cohn User Story principles. Hudson (2013) strongly opposes the structure of the universally accepted User Story. According to him, taking the perspective from the first person causes the following two problems:

Developers assume that users are similar to them, even when trying to approach the stories from a different

perspective; it has also been found that thinking about others is more effective than thinking about yourself (Polman

& Emich, 2011). Instead of writing from one own perspective, it is therefore recommended to write for one of the personas that were created in the previous stage (Hudson, 2013). This would change the structure of User Stories to the following:

<persona[:role]> <performs a task>[so that<unobvious goal>]

Hudson (2013) notes that the main benefit is that this makes stories descriptive, rather than prescriptive. Descriptive means that there is good reason to believe that a persona would perform such a task rather than describing what a user must do.

Four methods for gathering stories are suggested by Cohn (2004) which are chosen based on their lightweight and non-obstructive nature. Within software development, this is highly important as the systems are often developed iteratively. Despite Smart City systems being developed less iteratively, the methods are still suggested to be used here as the reasoning behind the application of each method still uncovers information for the clarification of Smart City objectives.

3.2.1 User Interviews

Interviewing users is a great way of understanding what functionality they desire in their work. Cohn (2004) considers three important aspects when performing interviews and asking questions. Firstly, it is highly recommended

to interview real users, in addition, one should also attempt to have a great diversity of users from different backgrounds. Questions should be context-free and open-ended. Context-free refers to questions that steer users in a certain direction; instead of asking what loading speed a user would want from a specific feature, they should be asked what performance they require. This allows for much broader answers to be given. Open-ended questions further help with allowing for widespread answers to be given. One should not assume users know the trade- offs when choosing between two features; still, one should also not steer users into considering specific negative or beneficial traits. When having to choose between a desktop or browser application Cohn suggests framing the question as follows: “What would you be willing to give up to have our next generation product run within a browser?” From this, the implementer can then decide what concession to make.

3.2.2 Questionnaires

Cohn (2004) is not a supporter of the idea of questionnaires for identifying User Stories. This is due to the one-way communication nature of the method. Rather, it is suggested to apply this when gathering large quantities of data about existing stories. Questions should be similar to the user interviews keeping them context-free and open-ended.

3.2.3 Observations

Letting users interact with the system and recording their comments is one of the best ways of receiving feedback

(Cohn, 2004). The unfortunate part of this is that, in comparison with software development, features cannot be

quickly created, tested, and integrated or dismissed for Smart City systems. This means that to create a prototype

often more time, money and energy is required when wishing to use observation. This method is recommended once

enough stories and subsequently, functions have been gathered to create a prototype which can then be refined

through observations. This could be highly beneficial before rolling out such a system on a large scale.

3.2.4 Story-Writing Workshop

A story-writing workshop should be inhabited by the users, developers and all other affected stakeholders.

Participants are each given a persona and asked to write stories related to the system from their perspective. At this stage, no priority should be given to any User Story, in this, the workflow is identified when interacting with the system. The session starts with an explanation of the system in general terms. An electric boat that can travel across a canal for example. The participants are then given an empty block and told this is the start of the system; standing on the side of the canal for example. From here the participants are asked to write what they can do next from the perspective of their persona. From these actions, new boxes can be added which are then expand further on the previously given stories. An example of this can be found below in figure 2.

Figure 1 Story-Writing Workshop

To identify missing stories the host of the workshop could ask the following questions: What would a persona most likely do next? What possible mistakes could a persona make? What might confuse the persona in this stage? What additional information would a persona require? It is best to focus on quantity rather than quality according to Cohn (2004). Stories can become redundant or replaced by better stories later on. Such an approach creates a low fidelity prototype of the system at play. Cohn (2004) suggests discarding this prototype as soon as possible as the low fidelity stories that are created in this process should be expanded upon in further stories which increase the resolution of what functionality the smart system must fulfil. Due to this increase in information the prototype quickly becomes irrelevant and is best to be discarded to avoid confusing.

As a commuter I arrive at the dock of the electric

boat yard.

As a busy commuter I want to call for the boat so that I can get to the other

side quickly As a Pensioner I

want somewhere to sit so that I can rest my body whilst I wait for the boat

As a busy commuter I want enough space on the boat so

that many people can travel across it As a pensioner I

want to be able to hold on to a railing or

have a seat so that I don’t fall over when

the boat moves As a pensioner I

want an easy step onto the boat so that I can cross the

waters safely

3.3 Filtering stories

Whereas the previous stage was mostly aimed at quantity rather than quality the following stage attempts to

increase the accuracy of stories so their usability rises. Lucassen et al. (2015) suggests the use of a Quality User Story Framework which only analyses the information gathered from User Stories. The framework uses fourteen criteria which are placed within three categories. These three categories are:

• Syntactic: Quality of the written part of the story. Not considering the context.

• Semantic: Quality of the relations between and meanings of different story parts

• Pragmatic: Quality of whether the form is acceptable for communicating a set of given requirements.

The relation between the criteria and the categories can be seen below in Figure 2. A description of the criteria can be found in Appendix D: User Story Quality Framework Criteria.

Figure 2 Quality User Story framework

All the written User Stories in the previous stage should be processed through the 14 criteria to adapt them to a standard form. This could increase the quality of the stories, and therefore, the data extraction. This stage should end with a final list of User Stories that describe all the systems functions that have been uncovered. There may be stories not yet uncovered. These will be found in the next stage.

USER STORY QUALITY

SYNTACTIC

SEMANTIC

PRAGMATIC

Atomic Minimal Well formed

Conflict-free

Conceptually sound Problem-oriented Unambiguous

Complete

Explicit Dependencies Full Sentence

Independent

Scalable

Uniform

Unique

3.4 Acceptance testing stories

The testing of stories has two important reasons. It exposes the assumptions made by story writers and gives a deeper understanding of how a system must function (Cohn, 2004). It also demonstrates that the system is satisfactory to the customer or user of that system. Cohn (2004) recommends that one can best write tests on the back of story cards. These can then later be expanded upon. All tests should be written by the customer or user of the system, not by developers. An example of a test for a Smart light post could be formulated as “Test automatic lighting up at night.” Such a test reveals the hidden assumption that the light post must automatically light up. Tests should be written before any prototypes are created. Cohn (2004) suggests that tests are usually written in the following stages:

• When the customer and user are speaking with the model operator about a story and wish to capture explicit details

• As a dedicated effort before designing of the system begins

• Whenever new tests are discovered when designing the system

It is recommended that all cards are gone through by the municipality and possibly users at least once before designing starts (Cohn, 2004). The following questions are recommended to keep in mind when thinking of possible tests:

• What else must be known to designers about this story?

• What are you assuming in this story that is not explicitly stated?

• What are situations where the story may act out differently?

• What can go wrong during the story?

When acceptance tests have been implemented into the system the municipality and users should be expected to

test the system on whether a test has been implemented successfully. This is considered out of the scope of this

work, however. As the goal is to clarify the municipal Smart City objectives, not to ensure that this functionality has

been implemented successfully.

4. Results

For understanding whether the User Story method can contribute to the clarification of Smart City objectives, the problems currently being experienced must be uncovered. As mentioned in the 1.2 Project context this thesis was written in the context of Amsterdam. Therefore, the analysis was conducted through interviews with municipal employees, companies and research projects which contribute to the Smart City themes of Amsterdam.

A chart of each person’s role in their organisation, the organisation’s relation to the four smart themes of Amsterdam and what type of interview was conducted can be seen below in Table 1.

Table 1 Conducted interviews

Municipal smart themes Semi-structured Interview Structured Interview Transition to smart, clean

Mobility solutions for people, goods and services

Organisation: Local Heroes Role: Head of operations

Organisation: Fynch Role: Product owner

Organisation: Local Heroes Role: Head of operations

Organisation: Fynch Role: Product owner Transition to renewable,

decentralized and variable Energy

Organisation: Elaad Role: Behavioural analyst

electric driving Transition from a linear

economy to a Circular Economy

Organisation: Swap Shop Role: Founder

Organisation: Swap Shop Role: Founder

Transition to a Digital City with responsible data-driven innovation

Organisation: Boombrix Role: Developer

Organisation: BRIDE Role: University of Twente

expert

Municipality

Organisation: Municipality of Amsterdam Role: Project Lead start-up

in residence

The first interview was with a municipal employee who works in the innovation team of Amsterdam. The second with an employee of Elaad, a company, founded by some of the largest energy suppliers

. They particularly focus on the transition to electrical vehicles. The representative’s position focuses on the user’s interaction with the charging stations that her company installs. The third was a start-up in Residence project

named Boombrix, this is a sensor that allows for the evaluation of tree health in Amsterdam. The fifth was Local heroes; A company focusing on improving the convenience of local market shopping.

Sixth was with Fynch whose app encourages greener transportation by offering rewards for certain behaviour as well as giving companies insight into employee travel behaviour. The seventh was with BRIDE, this is a project which has developed a 3D metal printed bridge. This bridge includes sensors that can identify pedestrians walking on it. The 8

was with The Swap Shop, which is a new take on recycling clothing. It is a physical and digital shopping platform where shoppers can bring old clothes for which they can then earn discounts on buying other clothes. Full transcriptions of all interviews can be requested through the researcher.

From the interviewed participants, Fynch and Local heroes had both applied User Stories in their development cycle. This is not surprising as both organizations have created services that are based in a mobile application. The Swap Shop founder used customer journeys, a method that was suggested to have commonalities with User Stories. The BRIDE university expert has experience with System Engineering, which includes stakeholder analysis. The other interviewees had not heard of User Stories.

3 Some notable large companies such as: Enexis Alliander and Stedin.

4 This is a training programme funded by the municipality of Amsterdam to invite new teams to solve Amsterdam’s

urban challenges.

4.1 Semi-structured interview results

In this section points of belief, with regard to Smart City development and User Stories, brought up by interviewees will be discussed. The comments will be separated into two groups. Comments that make a case for the application of User Stories in the municipal context will be placed in 4.1.1 Enabling User Stories. These could be, for example, problems experienced by companies and municipal employees with regards to uncovering functionality. The second, 4.1.2 Inhibiting User Stories, notes the comments that give insights that could limit the effectiveness of implementing the method in the context of Smart City development.

4.1.1 Enabling User Stories

The project lead of the innovation team from the Municipality of Amsterdam spoke in the interview about the process of how smart products are chosen. The team starts with the problem which is placed on the market. This is then put on the market as a tender where the team creates a campaign to get as many participants as possible. The team actively looks for start-ups. The gathered solutions are then filtered through a selection process from which one will emerge. The main challenge with this process is that a lot of energy and time is spent on the selection. The solutions may also not fully encompass the needs of the users. The interviewee noted that “A lot of projects stay pilots, and you know people are a bit tired of the word pilot”. The Elaad representative noted that their experience of creating a prototype was of high importance and gave a lot of valuable feedback. It was noted that some features were missed: “There were quite a few elements in there that we hadn’t thought of”. “Also, just system things but design things where people thought: ‘Hey I can click here and I have a choice’, while that is not the case”. The project lead of the innovation team commented on the ethics of Smart City development: “If you want to put up all kinds of smart technological cameras in the city, that’s fine, but if the citizens don’t know what it does and what it will be, then you’ll get resistance very quickly.” The behavioural analyst of Elaad corroborated this opinion stating:

“We are realizing more and more that there are people in these electric cars, and people have to go along with that [The charging stations]”. The innovation team lead noted that Amsterdam already has a platform for citizen participation named ‘Modern democracy’. Within this platfom, citizens can vote for what features they would like in their neighbourhood. The Boombrix team noted that when they were testing the data gathering device citizens would come to look at what they were doing in the street and were very interested. It showed the interviewee the willingness to participate in city development. The Elaad spokesperson also used user participation to figure out what the citizens wanted; brainstorm sessions, interviews with experts, and a political system debate workshop. The Local Heroes representative noted that the iterative process of User Stories is highly important and that, especially with a service, feedback should constantly be processed. Boombrix was very positive towards the idea of User Stories, the interviewee noted that solutions should be for the people and that the success of technology often rests on whether they have considered the people in their designing.

4.1.2 Inhibiting User Stories

The Elaad behavioural analyst thought the participation of users was always of great value when designing anything.

It was noted, however, that this does not have to be User Stories but that it could be one of the tools to encourage participation. The project lead of the innovation team noted that a development cycle is still highly important.

Many of the products developed through the start-up in residence program are completely ground-breaking and municipalities also like to see results in iterations before up-scaling. The developer of Boombrix noted challenges with implementing their product. Boombrix’s stakeholders, asset managers, did care about trees but considered that once planted, they would require no care. The arborist team is very small, and this could mean that even if a system that is desired is implemented the success could be limited due to resources being strained in certain areas.

The representative of Local Heroes mentioned that the application of User Stories does not guarantee that it is the

system people want: “I do notice is that you create a User Story and, in the end, it turns out to be slightly different,

but it does work better than doing nothing”.

4.2 Structured interviews

After a brief introduction and explanation of the research objective, the interviewees were taken through the steps of the User Story method and asked the questions posed in Appendix A: Interview questions. They were asked whether they believed the step would be effective or not after which they were asked to elaborate. If participants were unsure this was also documented.

4.2.1 Identification of users

From the interviewed 3 reacted very positively and 1 responded negatively to the identification of users. The Fynch representative spoke about the importance of users: “In my opinion, this is what it is all about. If you do not know the user, then how can you ever deliver?” The founder of Swap Shops corroborated this sentiment. The head of operations at Local Heroes noted that municipalities often decide to do something and then just do it. After which interviewee gave an example of how it was believed to be better: “I am that citizen. I live here, this is what it is now. I would actually like it to be like that, then I think it’s very effective If you define it like that instead of just trying some things.” The University of Twente expert representing BRIDE was less in favour of the method: “I don’t think it would be very effective because it would require a lot of time to go through these roles to analyse them and so on, and so I would say a little useless”.

Specific comments were also made on the two sub-steps of the identification of users that were mentioned as being additionally implemented works: the addition of extreme characters and personas. None of the participants had heard of extreme characters. Personas were a more known phenomenon. 1 participant looked negatively towards extreme characters, 1 was unsure and 2 thought it as positive. All but 1 participant did, however, have comments on the challenges that adding such roles could bring. The BRIDE representative had an issue with the exact definition of

‘extreme’: “Smart bridge with a wheelchair user would not be as extreme as something that I define as an extreme character. Extreme would be something like, truly extreme…” The Fynch product owner believed it better to focus on one group first but did see the importance of extreme characters in the municipal context: “If you focus on all these extreme characters as well, it will be impossible to make something concrete. These extreme characters can be added later in the process.” The Local Heroes head of operations found the concept interesting but was unsure of its implementation strategy and what it would mean for smart systems, the spokesperson had a similar sentiment to the Fynch representative. The Swap Shop representative looked favourably upon the use of extreme characters: “Yes yes, exactly, cities want to, and of course must be inclusive. I for sure think that they have to take this into account.” It was noted, however, that the use of extreme characters would not have been done by the Swap Shop. This was due to the problem of attempting to fulfil too much and then not satisfying any stakeholder.

All participants thought the use of personas was a positive addition. The Fynch product owner noted that the step may not be necessary but could aid: “I do think it could aid a lot. Especially if you work with a lot of different people.

From it, everyone can get a direct feeling for what is meant with a particular group.” The university expert for BRIDE

noted that personas could be useful but that he still would be biased. A similar sentiment was shared with the

Local Heroes head of operations. He noted the challenge of stepping into someone else’s shoes: “You know, it’s a

multicultural neighbourhood. Everybody there is very different…. I wouldn’t know how to walk in someone’s shoes

like that.”

4.2.2 Gathering stories

All the interviewees looked positively towards the gathering stories step. The Fynch representative noted the following: “The benefit of User Stories is that you have to think about what you deliver to the user, or what the added value is.” The Swap Shop founder did note a weakness of this method, referring to the method looking at the future of what people want: “That is very difficult for people to answer. If you take them to a situation of the past, okay. Once you wanted to go to the other side with a boat. What happened? And why? What did you think in that moment? If you approach it from that perspective, you get to know a lot.” In general, no specific comments were made on the various sub-steps, even by Fynch and Local Heroes who are familiar with the methodology.

4.2.3 Filtering stories

2 of the interviewees reacted positively to the step-in theory; there were however comments about the time and effort it would take to implement such a step. The Fynch product owner noted that this step is in essence her responsibility when discussing functionality in their smart app. The noted the following: “It is useful, but requires a completely separate function to be assigned. This way people don’t all have to have the criteria explained and one person can ensure the quality is consistent.” She was also sceptical of whether a software tool could potentially solve this issue. The university expert had the following opinion: “I see the added value, but if adding the value results in spending a lot of time, then I (if I was ‘the ministry’) would question it.”

4.2.4 Acceptance testing stories

From the interviewees, 3 looked upon the testing of stories step as positive, and 1 negatively. Most of the

interviewees positive about the stories did have comments on the method presented. Particularly, they were not

fond of only using this stage to uncover unmentioned functionality instead of the intended purpose of also validating

the systems. The Head of operations of Local Heroes found it important to note that writing tests does not uncover

all functionality. They must also be performed: “You come up with an idea, then you create the user Stories, then

that idea becomes something else again, then you test it, then it becomes something else again and then you get

to make the final adjustments.” The Swap Team founder understood that testing was difficult as the method is only

used for identifying functionality, still, she believed that creating prototypes and testing were the best methods

of capturing additional desired functionality. The university expert gave insights into the problems additional

information could give at such an early stage: “Yeah, it is devil’s in the detail. I would completely agree with

you that it aids a lot if you add the detail, but at the same time someone has to read all of this and then if

you›re thinking about one detail, you might think about all other possible details, and it just makes it way too

complicated.”

5. Analysis of results

In this section of the report, the data is analysed. This part of the report is separated into four subheadings, each heading represents one of the four main steps. Each step is then analysed in-depth with both positive and negative points from the gathered data is discussed. This analysis per step then leads to an understanding of whether it is useful or not in the context of Smart City development in Amsterdam. This section concludes with a summary of the

5.1 Identification of user roles

Here the analysis focussed on answering how effective the identification of user roles can be in the Smart City context. Overall the application of user roles was considered positively by all the interviewees when considering the industry standard steps of Cohn (2004). Although the interviewees were told about the various sub-steps such as the brainstorming, organizing, and consolidating of user roles, they focussed more on the larger picture of the step and its implications. It is believed that this could have two reasons: 1) the participants thought the sub-steps were useful, or, 2) they could not give any insight as they had themselves not had experience with the application of these sub-steps. This work skews more towards the first reason as many of the interviewees had experience with User Stories or similar methods. This means that if there would have been specific trouble with these sub-steps the interviewees, with the experience of implementing stakeholder like methods, would have most likely commented on it. The consensus over the Cohn steps was that the method was a good way of capturing what stakeholders are important and getting a feel for who the system is being built for. The BRIDE representative’s scepticism of the time this identification would take is appreciated. The researcher did not note, in any of the interviews, the time this step would take. In Appendix B: Gathering user roles some time estimation is given by Cohn who, for example, suggests that the brainstorming should only take 15 minutes. Both the Fynch and Local Heroes representatives, who are both acquainted with the User Story method, also did not have any comments on the long duration this step would take.

This then leads to the conclusion that the time this step should take is, most likely, not as long as expected by the BRIDE representative.

A lot of remarks were made about the extreme character roles and persona steps added to the method. This is believed to have been caused by two factors: The researcher explicitly stated that these steps were added from separate literature and the steps, are perhaps, more controversial. Particularly the addition of extreme characters was seen as controversial whereas the persona roles were mostly approached similarly to the Cohn steps and less discussion took place. The extreme character role has a few key issues that were brought up by the interviewees.

The first, discussed by the Fynch and Local Heroes representatives, was the issue of adding functionality for minority groups which could lead to a product that does not satisfy anyone. This is a real issue when considering budgetary restraints in the municipal context. Tax money is used and therefore if no large population group is satisfied with the system, it could reflect badly on the ruling parties. The note of the BRIDE representative is a valid one. What truly is an extreme character? The work by Djajadiningrat et al. (2000) does not give a quantitative answer to this and leaves it very ambiguous. It could therefore be very time consuming to come to a consensus with municipal experts, the developer and the operator of the system on what an extreme character is. Although the persona character addition was seen as a positive addition, the bias that was mentioned with regards to stepping into the shoes of such a character could be difficult. Such a problem could be lessened by ensuring that people similar to the persona are included in the design team. Amsterdam has 180 different nationalities (Shorto). Because only a handful of personas would be created, it could therefore be challenging to know how to represent such a persona due to the many different cultures and types of people in Amsterdam. This was something the Local Heroes representative noted.

Gathering stories

Here the analysis focussed on answering how much stories can aid in capturing the functionality of Smart City

goals well. Like the user identification step, no specific, negative or positive remarks were made about the different

methods for gathering stories. Although both the Fynch and Local Heroes representatives did note that they used

workshops to gather stories. Many of the unstructured interviews gave insight into the willingness of citizens to

participate in the design process, examples being the Elaad and Boombrix interviews. It shows that citizens are

interested in new systems and willing to give comments and work with the developers on these systems. It can

therefore be inferred that citizens would also be willing to write stories with the municipality for clarifying what

they would want from Smart systems in Amsterdam. The modern democracy programme in Amsterdam also shows

that infrastructure for implementing the method exists and could therefore lessen the difficulty of integrating the

method. The critical thinking step of considering what the reason is behind wanting certain functionality adds an

important step concerning reflection. This can limit the number of functions being pursued that do not have clear

reasoning for their implementation behind them. This was something noted by the Fynch product owner.

Despite the method being received well, it was noted by a few of the interviewees that it did not have to be the method of writing stories to capture the desired functionality that citizens want. This could mean that there are other ways of capturing functionality that could be more effective. An example of this was provided by the Swap Shop founder who used customer journeys to figure out what stakeholders wanted. She noted that people often find it difficult to think of something new and are much better at describing what they experienced and how this was frustrating and good. Another weakness brought up was the lack of prototyping in the method described in this method. The Elaad representative noted that the use of prototypes made conversations with users easier as well as capturing additional functionality was more effective. A similar sentiment was shared by the Project Lead of the start-up in residence programme who noted that they wanted to see prototypes and were not interested in only vague concepts. The goal of this research is to clarify goals but this clarification could be hindered by users struggling with letting their imagination go wild by the nature of the method as well as the lack of physical examples of smart systems. Another interesting point brought up by the Boombrix founder was the challenges with wanting certain smart functionality but stakeholder or users then not using these systems. A user may be able to consider a function and give a good reason for wanting it but then never using it. This is a challenge that could be difficult to overcome in the goal-setting of municipalities and certainly would require debate. Particularly once the

5.2 Filtering stories

Here it was analysed how much story filtering can aid in improving the capturing of functionality related to Smart City objectives. This step had a shared consensus amongst all experts. They saw the value in the application of this step but were worried about the time and energy that would have to be put into going through each story. A sentiment that is understood but cannot be confirmed without further research. The suggestion of the Fynch product owner to have a dedicated person on this task to ensure consistency and quality of the stories is kept high throughout the process is perhaps a suitable solution. Her scepticism on whether a software solution could be effective cannot be confirmed or denied due to the limited knowledge of the programme as it fell outside the scope. Having not applied the software on stories its effectiveness cannot be confirmed. The time factor seems to be the main possible disadvantage in this step.

5.3 Acceptance testing stories

In this section analyses focussed on how much story tests can aid in the clarification of functionality in Smart

City goals. The testing and questioning story writers further to uncover additional functionality had a plurality in

opinions from most interviewees. In some sense most agreed that the testing of stories was of benefit, but most of

these did think the way it was executed within this work was of less value. Many argued that real-life tests should

be conducted. This plurality was perhaps caused by the interviewees not fully understanding that the goal of the

research is to analyse the User Story method for the clarification of Smart City goals and not ensuring that the system

validates the wishes of the user. This is a contentious point, however, because through testing of real-life prototypes

one could uncover a deeper understanding of desired functionality in the smart systems, but these systems for

municipalities can often be of large scale and difficult to prototype. An example of this is the BRIDE project where

the 3D printed bridge has been tested for a year at the University of Twente campus before soon being placed in

Amsterdam. It shows that perhaps other methods should be explored for testing stories before creating a real-life

prototype. The devil is in the detail argument by the BRIDE representative also brings up interesting questions. The

clarification of goals in the municipal context of Amsterdam will need a certain level of detail. But what level of detail

is enough? If a citizen notes in a story that he would like to have transparent charging poles, does one have to go into

deeper detail at this stage? Could such a story not already describe enough before development starts? A benefit

of the use story method is its flexibility in levels of detail. A municipal employee could decide they would first want

to explore broadly what citizens would like within a certain smart theme after which they could decide to focus on

one written story and explore it further. This, therefore, allows the municipality to choose their level of detail on an

individual basis and avoid additional information where it is not required.

5.4 Summary analysis of results

Based on the analysis conducted on the viability of the main steps of the User Story state-of-the-art method the following conclusion can be drawn.

The application of user roles can be effective in the Smart City context, this was a consensus by all interviewees except for one. The BRIDE representative who disagreed with this method said it could take too much time. This belief was found not to be based in reality as two of the interviewees, which were experienced with the method, did not make similar comments. Many of the interviewees, did, however, make comments on the extreme character sub-step which was considered difficult to add. These roles could cause systems to have functionality that doesn’t satisfy any group fully and the term ‘extreme’ is vague and could mean many things to different people. The persona additional step was considered great by all interviewees, but it could be difficult to step into other people’s shoes.

From both the semi- as well as structured interviews it became clear that many different types of people live in Amsterdam.

The gathering of stories can aid in the capturing of the functionality of Smart City Amsterdam as all interviewees agreed such was the case. Many programs such as Modern democracy and the experiences by the Boombrix representative shows that citizens are willing to participate and work on improving their city. Some interviewees noted the limitations of the story writing process. The functionality that could be captured using this method will, most likely, not be able to capture all desired functionality. The Swap Shop representative also noted that participants often struggle thinking about the future and are better equipped to look towards past experiences.

story filtering can aid in improving the capturing of functionality related to Smart City objectives. The two

interviewees who were asked about this step thought it to be useful in a municipal context. Both did, however, note that such a step could take a long time to process. The Fynch representative job was doing similar work to this step.

The acceptance testing of stories could aid a little in the clarification of functionality in Smart City goals. This

limitation in aid was based on the opinion of interviewees that real testing of prototypes is a much more effective

method of capturing further functionality. Merely thinking of what functionality was missed can aid but not to the

same extend as prototype testing. Many interviewees also noted that testing should certainly be performed on the

system once they are created.