REQUIREMENTS IN SCRUM: STANDARDIZED OR MULTIPLE OPTIONS?

R

EQUIREMENTS IN

S

CRUM

:

STANDARDIZED OR MULTIPLE

OPTIONS

?

ing. N. (Nick) Holthuijsen (10416048)

nick.holthuijsen@avanade.com

Master Thesis Information Studies – Business Information Systems University of Amsterdam Faculty of Science

Final Version: June 30, 2014

Supervisors: drs. R. (Ronald) Kleijn & dr. G.P.A.J. (Guus) Delen

drs. R. Kleijn, signature: dr. G.P.A.J. Delen, signature:

Page | 2

A

BSTRACT

Requirements is one of the reasons for either an IT-project success or an IT-project failure. Popular Agile methods are emerging and becoming a standard in IT. These techniques do not have a clear standardized method to cope with the requirements. Scrum does not prescribe how to deal with elicitation and specification, but only states that all the requirements should be placed on the backlog. This study tries to describe the optimal approach for requirements in Scrum. The qualitative study resulted in fragmented elicitation and specification techniques used. Rather than one optimal method, it is concluded that requirements in Scrum are done optimally when considering the following aspects: Inspect & Adapt your technique, enough Business Involvement is required, use slicing techniques to make requirements fit in a Sprint, system documentation is not part of the requirement specification but is part of the development, and last having a dialogue about requirements is more important than completely specifying it.

Page | 3

T

C

1. Introduction ... 5

2. Theoretical Framework ... 6

2.1. Agile & Scrum ... 6

2.1.1. What is Agile and how does Scrum relate to that ... 6

2.1.2. Scrum Explained ... 6

2.2. Requirements Engineering ... 8

2.2.1. Definition & Purpose of a Requirement ... 8

2.2.2. Types of Requirements ... 9

2.2.3. Requirements Engineering Process ... 9

2.3. Requirements Engineering in Scrum ... 10

2.3.1. What are the elements that the Requirements Engineering process should comply with in Scrum? ... 11

2.3.2. What existing Requirements Engineering methods match the elements of Scrum?...12

2.3.3. Requirements Engineering Model in Scrum ... 18

3. Methods ... 19 3.1. Interviews ... 19 3.2. Participants ... 19 3.3. Validity ... 20 4. Results ... 21 4.1. Techniques ... 21 4.1.1. Elicitation ... 21 4.1.2. Specification ... 22

4.1.3. High-level & low-level requirements ... 24

4.2. Business Involvement & Feedback ... 25

4.3. Formalized Requirements ... 26

4.4. Slicing Requirements ... 27

Page | 4

5. Discussion ... 29

5.1. Threats to validity & future research ... 32

6. Conclusion ... 33

Acknowledgments... 33

References ... 34

Appendix A: Glossary & List of abbreviations ... 38

Appendix B: Scrum Framework ... 40

Appendix C: Specification by Example key process patterns ... 41

Appendix D: Story Split Cheat Sheet ... 42

Appendix E: Requirements Refinement Model ... 43

Appendix F: Interview Questions ... 44

Appendix G: Interview Gunther Verheyen ... 45

Page | 5

1. I

NTRODUCTION

We are all known with the fact that a lot of IT projects fail (stopped, cost overruns & schedule overruns)(Al Neimat, 2005; Bloch, Blumberg, & Laartz, 2012; informatie.nl, 2009). One of the main concerns is that the requirements are not understood or formulated incorrectly (Bloch et al., 2012; Gilb, 2010). Reasons for not understanding could be caused by the ever changing wishes of the end-user/customer. One of the principles of Agile is to create a development process which can handle the changing requirements even in the development state (Beck et al., 2001).

Agile development has gained momentum in the real world of software development but Agile research has been lagged behind (Abrahamsson, Conboy, & Wang, 2009). Forrester research (Hammond & Brown, 2013) found that Agile is now used on 19% of the formal development processes, while 37% is not using any methodology at all. Abrahamsson et al. propose to research the extension of the applicability of Agile. Requirements Engineering is as stated one of the issues in software development. But a combination of Requirements Engineering and Agile software development is still lacking. This study focusses on researching the relationship between those phenomena.

Scientific Agile research has fallen behind on the real world. IT industries are using Agile more and more, yet there is little scientific research (Abrahamsson et al., 2009). As the University of Amsterdam suggests (according to the proposed research themes); Information Science is researching on improvements to perform requirement analysis as a general principle. This study will help us to better understand Agile and requirement analysis in specific.

Abrahamsson et al. (2009) stated that although more research is done to the new emerging phenomena in Information Systems Development, a deeper understanding of how Agile methods are deployed in practice is needed. They also stated that there is a need for combining Agile methods with other existing approaches to extend the applicability of the Agile methods. The theoretical framework elaborates the popularity of Scrum as an Agile development framework, explaining the focus of this study is on Scrum.

This research aims to study how requirements are analyzed in practice when using the Scrum framework and how that is different from traditional Requirements Engineering theories. Therefore the following research questions are proposed:

RQ: How can the requirements be optimally determined when using the Scrum software development framework?

This research question develops the following sub questions:

SQ1: How are requirements determined according to the Scrum framework and according to Requirements Engineering studies?

SQ2: How are requirements determined in practice using the Scrum framework?

SQ3: To what extent are current theories on Requirements Engineering integrated in the Scrum framework in practice?

Page | 6

2. T

HEORETICAL

F

RAMEWORK

2.1.

A

GILE

&

S

CRUM

2.1.1.

W

A

S

Agile software development methods attempt to offer a lightweight and faster software development method for delivering value to the customer. The Agile Manifesto (Beck et al., 2001) describes the twelve principles of these development methods. Agile practitioners are convinced that the best software emerges from self-organizing teams. Abrahamsson, Warsta, Siponen, & Ronkainen (2003) present an overview of the different development methods, based on the Agile principles, currently available. Most used methods are DSDM, XP, FDD and Scrum. VersionOne (2014) claims that Scrum is used 55% of the times when Agile development has been applied. Dingsøyr, Nerur, Balijepally, & Moe (2012) researched the current state of scientific research on Agile. The research “found that most studies focused on eXtreme Programming, and very few on

the Scrum development process, which was gaining significant traction in the industry” (p.1219).

This increased the desire to perform further research focusing on the Scrum framework and its application.

2.1.2.

S

E

The foundations of Scrum are originally introduced by Takeuichi & Nonaka (1986). They discussed that software development had to change to follow the fast changes of organizations. The teams had to have more responsibilities and had to be self-organizing. Based on these tenets the Scrum framework was introduced by Ken Schwaber and Jeff Sutherland in 1993 (Sutherland, 2012). Sutherland and Schwaber (2013) describe Scrum as: "a framework within which people can

address complex adaptive problems, while productively and creatively delivering products of the highest possible value." The framework offers an easy to follow set of rules which can be used to

efficiently manage product development efforts in an iterative, incremental approach.

In this thesis we will not discuss the entirety of Scrum, only the fundamentals to understand the concept of the framework, in Appendix B the framework is displayed.

2.1.2.1. S

R

Scrum defines three different roles (Sutherland & Schwaber, 2013):  Product Owner

The Product Owner (PO) is responsible for maximizing the value of the product and the work of the Development Team. This person is the main contact of the stakeholders and makes decisions on priority. The Product Owner creates and manages the Product Backlog.

 Development Team

The Development Team consists of professionals who deliver a potentially releasable increment at every end of the Sprint. The team is self-organizing and no one tells the Development Team how to turn the Backlog into the increments. The teams are cross-functional and should include all the necessary skills to achieve the Sprint goal. All

Page | 7

specialist roles are referred to as developers (for example: a Business Analyst develops requirements).

 Scrum Master:

The Scrum Master is responsible for ensuring Scrum is understood and enacted. He is responsible for removing Sprint impediments and should coach all the involved professionals in how to apply Scrum.

2.1.2.2. S

E

The Scrum events defined by Sutherland & Schwaber (2013), are time-boxed with a maximum amount of time per event based on the size of the team and Sprint cycle.

 The Sprint

The Sprint is the main element of Scrum. It is a time-boxed event of four week or less, in which a potentially releasable product increment is created. The Sprint contains Sprint planning, daily Scrum, Sprint review and Sprint retrospective. During the Sprint no changes are made that would endanger the Sprint goal.

 Sprint Planning

The Sprint Planning is used to determine which items will be included in the Sprint Backlog. The plan is created by the entire Scrum team. Primary focus of the planning is to see what can be done in this Sprint and how it will get done. The Sprint goal is set and selected Product Backlog Items are placed on the Sprint Backlog.

 Daily Scrum

The daily Scrum is a 15 minute daily event for the Development team to synchronize activities and create a plan for the next 24 hours. “What did I do yesterday?”, “what will I do today?” and “do I see any impediments towards the Sprint goal?” are the three main questions.

 Sprint Review

The Sprint Review is held at the end of the Sprint to inspect the Increment and adapt the Product Backlog. At this meeting, the stakeholders are also participating, giving feedback and changing the Product Backlog if needed.

 Sprint Retrospective

The Sprint Retrospective is a meeting for the Scrum team to inspect itself and create a plan for improvement. Main questions are: “what went good?” and “what needs improvement?”

2.1.2.3. S

A

The defined Scrum artifacts are (Sutherland & Schwaber, 2013):  Product Backlog

The Product Backlog is an ordered list of everything (features, functions, requirements, enhancements and fixes) that might be needed in the product. The Product Owner manages the Product Backlog, including the content, availability and ordering. The estimation of the Product Backlog Item (PBI) is done by the Development Team.

Page | 8

 Sprint Backlog

The Sprint Backlog is the set of PBIs that are forecast for the current Sprint, plus a plan on how to deliver the Increment. The Sprint Backlog consists of all the work to meet the Sprint goal.

 Increment

The Increment is the sum of all the PBIs completed in the Sprint and any preceding Sprint. The Increment must be done according to the Definition of “Done”. The Increment must always be in useable condition, even if it is not released. The Definition of “Done” is the shared understanding of the Development team what is perceived as finished.

2.2.

R

EQUIREMENTS

E

NGINEERING

2.2.1.

D

&

P

R

Before explaining what the Requirements Engineering process actually is, first an understanding on the definition of a requirement is needed. The term requirement has different definitions. The Oxford Dictionary defines a requirement as: “(1) A thing that is needed or wanted (2) A thing that

is compulsory; a necessary condition”. A requirement is defined by the IEEE (IEEE Computer

Society, 1990) as:

1. A condition or capability needed by a stakeholder to solve a problem or achieve an objective.

2. A condition or capability that must be met or possessed by a solution or solution component to satisfy a contract, standard, specification, or other formally imposed documents.

3. A documented representation of a condition or capability as in (1) or (2).

The International Institute of Business Analysis (IIBA) gives the following definition:

“Requirements include, but are not limited to, past, present, and future conditions or capabilities in an enterprise and descriptions of organizational structures, roles, processes, policies, rules, and information systems. A requirement may describe the current or the future state of any aspect of the enterprise” (IIBA, 2009).

According to Hull, Jackson & Dick (2005) requirements are the basis for every project, defining what the stakeholders of a potential new system need from it and also what the system must do in order to satisfy that need.

These definitions clarify that a requirement is a stakeholder request for a condition or capability that should be met with an organization structure, role, process, policy, rule and/or information system. A requirement is a functional need that ensures understanding of the requested service. But why should you specify the requirements formally?

Nuseibeh & Easterbrook (2000) discus that requirements need to be defined, because of estimating project costs, schedules and technical feasibility from the specifications of requirements.

Page | 9

Hull et al. (2005), Sommerville (2007) and the IEEE Std. 830-1998 (IEEE Computer Society, 1998) describe that requirements form the basis for more than development of software, it is utilized for project planning, risk management, (acceptance) testing, tradeoff and change control. The level of detail of the requirements differs with the purpose of the requirement (Sommerville, 2007), more stakeholders equal more detail, because every stakeholder may require different details (For example: testers need to know what to test).

Requirements are not only used for the initial design of the software, but are also used in a later moment for testing and maintenance (Read, 2013). To measure if the software does what it envisions, IEEE (1998) states that quality traits of the requirements are: correctness, lack of ambiguity, completeness, consistency, verifiability, modifiability and traceability. Read (2013) stated: “In iterative software development cycles, requirements may not reach such a high level

of quality in documented form, but are confirmed through face-to-face interactions with the customer (Cohn, 2004), sometimes while discussing already built software.”

This paragraph can conclude that requirements are not only associated with programming the software but are also associated with project planning & estimations, risk management, (acceptance) testing, tradeoff, schedules and change control.

2.2.2.

T

R

Software requirements are made in different distinctions. Organizations and researchers have done studies on very specific distinctions (IIBA, 2009) and very broad distinctions (Hull et al., 2005). This distribution of definitions creates a chaotic view on the essence of Requirements Engineering, which is: clarifying the requested services and constrains.

Sommerville (2007) defines two layers of requirements. First a distinction should be made between user requirements and system requirements. The user requirement contains statements in a natural language of what services the system is expected to provide. The system requirements set out system's functions, services and operational constraints in detail. In the second layer the system requirements have a distinction between the functional and non-functional requirements. The non-functional requirements are statements of the services that the system must provide. The non-functional requirements (also defined as quality requirements) are requirements that are not directly concerned with specific functions, but may relate to system properties such as: reliability, response time and occupancy.

2.2.3.

R

E

P

We have identified what a requirement is and what type of requirements are known. Requirements Engineering is the process of finding out, analyzing, documenting and validating the requested services and constraints (Sommerville, 2007).

The Requirements Engineering process is separated in four areas (Bourque & Dupuis, 2004; Nuseibeh & Easterbrook, 2000; Paul, Yeates, & Cadle, 2010; Sommerville, 2007), namely:

Page | 10

1. Requirement elicitation & analysis:

The elicitation and analysis is the process of retrieving the requirements from the stakeholders.

2. Requirement specification:

This process is focusing on describing the requirements defined in step 1. 3. Requirement validation:

This process is focusing on checking the requirements for validity, consistency, completeness and verifiability. Requirement reviews and prototyping are the principle techniques.

4. Requirement management:

Focusing on managing planning, creating policies and manage changes in the requirements.

Besides these four main sub-process, an additional sub-process is suggested by Sommerville (2007); the feasibility study is focusing on the business case of the IT project, before deciding if the project should continue or not. During the feasibility study, the focus is on getting the business goals of the project and define a few high-level business requirements to assess the profitability of the project.

If we place these steps in a model, it would look like Figure 1: Requirements Engineering Model. De Swart (n.d.) describes that these steps are not separated steps that you should perform when the other is finished, but that these steps in Requirements Engineering should be integrated. For example: when you are eliciting a requirement you should specify it at that moment, and also check if you understand the stakeholders by continuously validating what is written down.

2.3.

R

EQUIREMENTS

E

NGINEERING IN

S

CRUM

In traditional software development (the waterfall or V-method) the Requirements Engineering process is the first phase of software development (Wikipedia, 2014). Hull et al. (2005) discuss that Requirements Engineering in traditional software development is not a one-time phase but should be done in every phase of the V-method.

Page | 11

Using an Agile Software development methodology, Requirements Engineering continues throughout the lifecycle of software development. Scrum does not extensively describe how to deal with the requirements of a project. All of the requirements should be placed on the Product Backlog. The requirements should be assessed by the Development team and should “refine” the requirements for the next Sprints. This refinement should take a maximum of 10% of the capacity of the development team. The Development team creates the estimates, but may never change PBIs. This is the responsibility of the Product Owner. The Development team may add new PBIs, but the Product Owner is managing the Product Backlog.

Scrum describes that it is important to know what to develop, but gives the self-organizing Development team the opportunity to decide how to acquire this insight. They should come up with a proper way that is enough for them to understand what to develop in the next Sprint.

2.3.1.

W

R

E

S

?

Trying to identify what could be suitable processes that fit into the Scrum framework, we first have to describe what elements the process should comply with in order to fit.

To be a highly effective team when using Scrum, the primary idea is to develop according to the Lean principle: eliminate as much waste as possible (Lean Enterprise Institute, n.d.). Every task should be performed as absolutely required and extra work (gold-plating) is fundamentally wrong. This also applies to the requirements. The requirements should be described clearly enough so the developers know what to do, but no more than that. Writing large documents on explaining the requirements is also out of the question.

To deal with the ever changing requirements, the Requirements Engineering process is not a pre-development phase but should be integrated within the pre-development. Sutherland (2012) defines this as JIT(Just In Time)-requirements.

A famous quote by Steve Jobs (Reinhardt, 1998) when working with design by request is: “people

do not know what they want, until they see it”. Asking the stakeholders why they want something

instead of what they want could help. Acquiring feedback in an early stage of the development is a solution to show the customer: is this what you meant or does this solve your problem? Prototyping is an example of a method to acquire feedback on the understanding of requirements. Some teams prefer to show real work, rather than prototypes.

The specification of the requirements is dealt in the next paragraph, but one of the used techniques is user stories (Cohn, 2004). Bill Wake (2003) created the INVEST acronym to remind teams of the good characteristics of the user stories. This acronym cannot only be applied to the user stories but should be applied to each of the other techniques to describe requirements. INVEST is an acronym for: Independent, Negotiable, Valuable, Estimable, Small and Testable.

Page | 12

Concluding this paragraph, we could state that the Requirements Engineering process should at least have to following elements: iterative, incremental/independent, eliminate waste, JIT and acquire feedback for understanding.

2.3.2.

W

R

E

MATCH THE ELEMENTS OF

S

?

The process of Requirements Engineering is separated in four sub-processes. Based on these four processes we try to identify the work to be done.

To start with identifying which techniques to use, first Specification by Example is discussed. Specification by Example contains the elements dealing with the requirements in Scrum. It might not be the optimal technique, but gives a guideline in how we can create a process that is suitable for Requirements Engineering in Scrum.

2.3.2.1. S

E

Specification by Example is a collaborative approach to define requirements and business oriented functional tests (Adzic, 2011), see Appendix C for the figure. The key idea is that there should be one document representing the specification of the software, instead of different document from analysts, developers and testers. Every involved participant of the specification of the requirements should collaborate on one single definition. This one definition is in the end of the Specification by Example process an automated test. The process is based on the practices of Behavior Driven Development (BDD) and Acceptance Test Driven Development (ATDD) (Adzic, 2011, p. 29). BDD and ATDD focus on creating automated tests before actually developing the product. After the automated tests fail, the code will be created to make the test pass. The code can then be refined and after that marked as complete.

Based on the literature described until this point the following process steps are proposed: Step 1: Sprint n-11 _{Product Backlog Refinement: Collaboratively specify requirements}

Step 2: Sprint n-1 Product Backlog Refinement: Refine requirements with key examples Step 3: Sprint n Development and test: Create automated tests, develop software and acquire early feedback when developing. Living documentation should be ready at the end of the Sprint.

Not bounded by Sprint: Managing requirements (Manage Product Backlog).

Because you create the tests beforehand you get double feedback when developing the software if it actually does what it should do according to the specifications. You get feedback from the automated tests and feedback when communicating with the stakeholders when something is finished. This is possible because the specifications of test are described in a way the business actually understands.

1 _{Might be sprint n-2 or 3, based on the size, complexity and the team. But refinement should always be done as late}

Page | 13

Using the Specification by Example process we got a process framework that can be used when developing with Scrum. The validate requirements step is covered because of the feedback when using step 2. The software is validated in step 3 with automated tests and early feedback. How to elicit and specify is not described by Adzic. Management of requirements is done with the management of the Product Backlog. These three aspects are discussed in the following paragraphs.

2.3.2.2. E

&

A

Nuseibeh & Easterbrook (2000) describe in their research different elicitation techniques, divided in separated classes. These classes are: Traditional, group elicitation, prototyping, model-driven, cognitive and contextual techniques. In this paragraph we shortly discuss these classes and elaborate which are suited for the proposed process.

 Traditional techniques

This technique includes traditional techniques such as surveys, interviews and document analysis. Interviews can be used when needing in depth knowledge from a stakeholder. Surveys can be used when large amount of data is required.

 Group elicitation techniques

When multiple stakeholders are required to elicit information on requirements, group elicitation techniques can be used. These techniques include Joined Application Design, brainstorming and focus groups. Because requirements need to be lean, only focus groups might be suited. Adzic (2011) discuss that focus groups at least need a programmer, a tester and an analyst to interview the stakeholders. These focus groups are based on the introduction of the three amigos method in XP (Dinwiddie, 2011).

 (sketch) Prototyping

If early feedback is required on the system, prototyping might be a good solution. Prototyping can be combined with other techniques in order to get a better view on the requirements. The requirements techniques that can be used in this context are: story boarding and model storming. These storyboards or models should not be too realistic to ensure expectations of delivery time are not reduced. Besides these techniques, early products are shown to the product owner to acquire feedback. This can also be seen as a form of prototyping.

 Model-driven techniques

Model-driven techniques provide specific models of type of information. Known models are KAOS and I* (Nuseibeh & Easterbrook, 2000). Other methods, such as UML, can also be seen as a model-driven technique. Using model-driven techniques are dependent on the expertise of the stakeholders, some might understand UML, while others may understand KAOS. It is also a possibility none of the techniques are understood, at which the model-driven technique should be avoided or explained to all involved parties.  Cognitive techniques

Cognitive techniques are originally used for knowledge acquisition for knowledge based systems. Techniques include protocol analysis, laddering and repertory grids (Nuseibeh & Easterbrook, 2000). This technique requires a large amount of time to perform in waterfall, therefore it would be a challenge to use it iteratively.

Page | 14

 Contextual techniques

Contextual techniques include observation and conversation analysis. These are fine-grained techniques that focus on the detail in performed activities. This can for example be used after some Sprints, you then have some increments, at which you can observe the use of the system.

Based on this overview, we cannot rule out any of the techniques. However we can divide the applicability of the different techniques. The techniques; traditional, group elicitation and model-driven, are more applicable when eliciting requirements when there are no detailed requirements yet, and no related increments are delivered. The prototyping technique can be used to further clarify requirements and create sketches to better understand the requirements. The last two techniques; cognitive and contextual, are more suitable to improve the product, because you can observe the use of the earlier developed increments.

Analysis of the elicited requirements is limited because requirements are acquired according to the JIT principle. The requirements are specified a Sprint before development. Analysis are generally used to acquire additional information regarding requirements and create consistency. When performing Requirements Engineering in Scrum these analysis should be performed during the elicitation of the requirements. When multiple development team members are available (focus group) these should all get the requirements precise for their skillset.

2.3.2.3. S

Specifying the requirements can be done with many different techniques. Based on the elements that Requirements Engineering should have in Scrum, there are three viable options to specify the requirements, namely using: user stories, use cases or process models. Each of the next paragraphs gives a short introduction on these specification techniques.

2.3.2.3.1. USER STORIES

User stories is a popular way to specify the requirements in Scrum (Adzic, 2011; Sutherland, 2012). User stories was originally introduced in the XP process (Cohn, 2004), but is also suitable for Scrum.

Cohn (2004) describes a user story as “functionality that will be valuable to either a user or a purchaser of a system or software”. The three key elements of a user story are (Jeffries, 2001) :

 Cards: The card is a small description on what the stakeholder wants. On the front of the card the requirement is described. On the back of the card (acceptance) test criteria for the requirement are specified.

 Conversation: The conversation is to further clarify the requirement specified on the card. It is essential when applying user stories that a conversation is held on the requirement, when the details are not yet clear. The result of the conversation may be (but not required) notes, diagrams, sketches or other information that clarifies the requirement.

Page | 15

 Confirmation: The confirmation is to confirm if the requested requirements are met. This is done based on the (acceptance) test criteria. The tests can be done manually or automated.

If we model these three steps, we see a similar model in Figure 2: User Stories model. These steps are very similar to the Requirements Engineering areas (elicit & analyze, specify and validate).

Large user stories that still need to be split are called epics. Epics are used when defining high-level requirements (Cohn, 2004), they can be split into two or more stories of smaller size. Adzic describes that a user story describes how a user will get a specific value out of a system. The following structure is advised by Adzic (2011) and Cohn (2008):

As a <stakeholder role>

In order to <achieve something valuable> I want <some system function>

This structure provides insight on what the value of a specific user story is. The benefit of using user stories is that it is easy to understand for all the stakeholders. The customer understands the written text and so does the developer. If the developer does not completely understand the user story, it always has the opportunity to have another conversation about the story and add additional notes if needed.

Using only a user story is (probably) not enough to develop the software. To elaborate on the detail of the user stories, the conversation is described. In Scrum it is widely acknowledged to add acceptance criteria to the user story (Adzic, 2011; Cohn, 2008; Hassa, 2013). Describing the acceptance criteria gives more insight on what the customer actually wants. The introduction of the acceptance criteria is based on the ATDD principle. Creating formal criteria Dan North created a structure to write test criteria (North, 2006):

Given <initial context> When <an event occurs> Then <ensure some outcome>

FIGURE 2: USER STORIES MODEL

Card

Conversation Confirmation

Page | 16

Several tools support this structure in order to automate the tests2_{, as is defined in the}

Specification by Example process pattern.

Combining the user story with acceptance criteria, makes it easy to understand for both the stakeholder as the development team what is needed. It also creates a more formal way of describing the requirements and is easier to automate and maintain. And this with a minimal of documentation.

2.3.2.3.2. USE CASES

Use cases are another specification technique, which are often associated with the Rational Unified Process (RUP). Use cases are part of the Unified Modelling Language (UML). It is widely used in requirement practices in both Agile and traditional development methods. Use cases are used to create a graphical overview of the requirements, displaying actors, use case function and the use case relationships.

Fowler & Scott (2003) define a use case “as a set of scenarios tied together with a common user

goal”. The use case can both be written text or a graphical presentation (diagram) of the

requirements. Agile Modeling (n.d.) describes the use case as a sequence of actions that provide a measurable value to an actor. The use case encompasses different scenarios of the same requirement. The use case diagram, is used to visualize the use case, but according to Fowler & Scott this is not required in order to write a good use case.

Use cases are often supported with additional models. Requirements according to UML are based on use cases, sequence diagrams and class diagrams. Besides these three diagrams; state, activity and physical diagrams are other supporting diagrams in UML (Fowler & Scott, 2003). Applying Scrum principles; only the diagrams that are necessary for understanding the requirements are used when specifying the requirements, and should thus be created only if they are needed and when they are needed. The previous paragraph discussed the acceptance criteria that should be added to user stories. Acceptance criteria can be used to further specify a use case, when the other UML diagrams are undesirable or acceptance criteria are sufficient.

2.3.2.3.3. PROCESS MODELLING

It is very common that requirements are displayed as processes, since some Information Systems have to support a process. Known process modelling illustrators are activity diagram or BPMN. Creating a model of all the organizational processes up-front is not very “agile”. A method to do Agile process modelling is to first identify high-level processes and detail the processes that will be supported in the next iterations.

DSDM is an Agile method that uses this type of requirement specification during the workshops (“15. Requirements,” n.d.). The workshops to elicit the requirements are first held with the process owners to receive an indication of the high-level requirements. The end users of the new

Page | 17

system are asked how they perform their activities of the process when more details of the process are required. These high-level and low-level specifications might also be suited for Scrum.

2.3.2.3.4. BEST SPECIFICATION TECHNIQUE

Now that the three viable specification techniques have been outlined, we need to elaborate on what would possibly be the most suitable technique in Scrum. Based on the literature study, we cannot state that there is one best option. User stories is the “leanest” of the three suggested techniques. If the user stories technique is sufficient for development teams to develop the software, it is suggested -based on the literature- to only use user stories. Use cases and/or process modelling can be used when the user stories alone are not sufficient.

User stories in general are by design very broad, which is one of the reasons acceptance criteria are added. The Specification by Example process describes this suggestion, but also suggests to add examples to the specification. To have the lean aspect, it should be considered by the team if it is required to add examples or that a user story with acceptance criteria is sufficient. Adding clarification to an already well understood story could result in waste. When a design is required other techniques might be included, for example model storming, in order to fully refine the requirement. The primary rule in specifying the requirements is: do not specify more than required to understand what needs to be developed.

2.3.2.4. M

Management of requirements is less represented in this study, because Scrum tells what to do when you have the requirements. The requirements have to be described, ordered, refined and estimated. Scrum is very clear in changing requirements. Requirements may always be changed by the Product Owner, if the requirement is not yet in a Sprint. If the requirement is in Sprint and needs to be changed, that can be done at the end of the Sprint and a new PBI will be created. The Development team may not change requirements, but can be the author of a requirement and may advice the Product Owner to change a requirement. The Product Owner is the manager of the Product Backlog and therefore the manager of the requirements.

Before requirements can be actually developed, they must be of a “workable” size, meaning as small as possible and should at least be realizable in one Sprint. When using user stories, this is referred to as splitting or slicing user stories (Cohn, 2004). Splitting use cases can be harder, because the use cases are mostly connected and look inseparable. Two techniques to split stories are: story mapping (Patton, 2005) and a pattern described by Richard Lawrence (2009, 2012). Story Mapping is a practice where you horizontally map the user stories as activities in order of priority and on the vertical axis as increasing sophistication of the implementation. The top row is a sort of “skeleton” that is a first version, but usable product. The next row represents additional functionality on top of the “skeleton”.

The pattern described by Lawrence are nine patterns on how to split user stories. The main focus on the patterns is that the stories will always be conform the INVEST acronym. Every small

Page | 18

independent user story must have value for the business. The pattern sheet is available in Appendix D: Story Split Cheat Sheet.

Impact mapping (Adzic, 2012) is a technique for ordering and managing large and small user stories to describe value and priority of the user story. It provides focus for delivery, by putting deliverables in the context of impacts they are supposed to achieve and visualizes the assumptions. The main goal of impact mapping is to achieve answering the question: why are we

doing this?

2.3.3.

R

E

M

S

Based on the defined techniques in the previous paragraph the following model is proposed in Figure 3: Requirements process in Scrum (Sprint n-1) (also available in Appendix E: Requirements Refinement Model).

The steps are discussed in the previous paragraphs. These steps describe the requirements according to the Requirement Refinement in Scrum which is done in Sprint n-1. Requirements Engineering in Scrum starts with the Elicitation and Analysis, using the methods described in the model as elicit method 1. The other described methods can be used to further clarify a requirement or analyze the interaction with the system that is already created in previous Sprints. The requirements are specified, using user stories with acceptance criteria, use cases or thru process models dependent on the project. The requirements are validated by both stakeholder and developers after which the specification can be marked as OK, as “too large” needing a split in several user stories, or it may need further specification. The cycle continues per requirement until it is refined enough to move it to PBI ready for development.

Page | 19

The second part of the requirement is during Sprint n. As described in the theoretical framework requirements in Scrum are dealt with from the start of the project until the end. Therefore also shortly the last steps of the process are according to Scrum and Specification by Example are discussed. First create the automated test cases, then you develop the software, acquire early feedback from both the customer and the tests and refine the software & update the specification if necessary. When feedback is acquired, it is essential for the Product Owner and the Development team to decide to apply the feedback in the same Sprint or put them on the backlog in order to achieve the Sprint goal.

FIGURE 4: REQUIREMENTS SPECIFICATION IN SCRUM (SPRINT N)

3. METHODS

The function of this research is to compare and evaluate the Requirements Engineering techniques in the context of Scrum. Therefore qualitative research is the most suited method. The study started with a literature study on Requirements Engineering techniques and the definition of Scrum. Based on this literature study the most suitable techniques in the context of Scrum are outlined. Using semi-structured interviews the goal is to figure out how these theories are aligned with current Scrum practices.

3.1.

I

NTERVIEWS

The interviews for this study are semi-structured, meaning that there is a baseline of questions, but in order to really answer the questions follow up questions are required on the provided answers. The interview questions are described in Appendix F: Interview Questions. Besides these interview questions it is also tried to understand how “agile” the project is. This is done because weak Scrum projects may affect the perception of good or bad Requirements Engineering.

3.2.

P

ARTICIPANTS

The participants are outlined in Table 1: Participants Interviews. The data is anonymized by using an alphabetical numeration on the organization and projects. Quotes used in the results are referred to the role of the interviewee.

The roles of the participants interviewed are the primary roles in Scrum (Product Owner, Scrum Master & Developer) and additional roles inside the projects (Business Analyst & Project Manager). In total 21 interviews are conducted, with 22 participants. Grouping the participants

Page | 20

a total of 12 teams were interviewed. Two interviewees also elaborated on previous team experiences. The minutes of the interviews are displayed in Appendix H: Interview Results. The minutes are not in chronological order, but are categorized by organization and project.

TABLE 1: PARTICIPANTS INTERVIEWS

After defining the first draft of the conclusion, the results are discussed with Gunther Verheyen (Director Professional Programs at Scrum.org). Based on the results some extra questions are asked to get the opinion of a Scrum representative. The questions and minutes are displayed in Appendix G: Interview Gunther Verheyen.

3.3.

V

ALIDITY

To ensure the validity of the study, three organizations are interviewed. Because Organization A are the largest source, the projects in this organization are performed at large customers, whom all have their own ideas on how to run the project. Organization A does not have one formal way for the project teams to run their project, therefore each project has slightly different implementations of Scrum. This ensures that the sources of interviews are on a higher validity level. Having an expert interview to review the final results improves the quality of the data, because of his long experience with Scrum projects.

Organization Project Role

Organization A Project A Product Owner/Scrum Master

Developer Project B Developer

Project C Developer/Agile Trainer Scrum Master

Developer

Product Owner 1 & Product Owner 2 Project Manager

Project D1 Developer Project D2 Developer Project E Business Analyst Project F Scrum Master

Scrum Master Project G Business Analyst

Project H Developer/Agile Trainer Project I Developer

Organization B Manager Software Development

Product Owner Business Analyst

Page | 21

4. R

ESULTS

4.1.

T

ECHNIQUES

The requirement techniques used in Scrum differ per project. There is a thin line between eliciting and specifying the requirement. One interviewee stated the following:

“When using waterfall, creating use cases was a goal. At the end of the requirement phase you have to finish the use case in order to continue. In Scrum you have to define what you need to build, the use of different techniques, both elicitation and specification, are only a tool to define it.”

If we look primarily at the theoretical framework a distinction is made between elicitation and specification. Although these are intertwined, a separation is made in order to better understand the different techniques that are used to elicit and specify a requirement.

4.1.1.

E

Elicitation techniques that are used are: large workshops, focus groups, brainstorming sessions, (sketch) prototyping, observations and interviews. In Figure 5: Elicitation Techniques the results are visualized. Note that the numbers do not add up to 21, because some teams use multiple techniques.

According to three Product Owners, large workshops are primarily used when to elicit high-level requirements. Focus groups are specifically used when to go into more detail on a requirement. In both types of workshops brainstorming techniques were used. A Scrum Master stated:

“In large workshops the focus was on eliciting as much requirements as possible. The focus groups focused on as much detail as needed for a requirement.”

Two Product Owners stated: “When having a lot of stakeholders, first a large workshop is created

to discuss the high-level requirements. Focus groups are specifically used to clarify a requirement.”

Prototyping is used in different matters. For example model storming and story boarding is used when a requirement is not specific enough. Twelve interviewees argued using sketch prototyping techniques. A common answer is:

“Model Storming or Story Boarding is used to clarify a requirement when there is discussion on the requirement. Some quick models sketched on a white board can clarify the requirement very fast.” – Scrum Master.

One Business Analyst stated also using rapid prototyping in order to elicit 150 requirements as fast as possible.

Page | 22

Observations are done in two occasions, namely when an old system needs to be replaced, the system functionalities are observed and reviewed. The second type of observation is to elicit feedback from the stakeholders, the stakeholder is observed what their actions on the software are in order to understand the feedback and thus to clarify the requirement. Interviews are used sometimes. Seven interviewees replied to use an informal interview to elicit a requirement one on one. This is done with an expert or when the functionality only affects a few users.

FIGURE 5: ELICITATION TECHNIQUES

4.1.2.

S

The interviewees used five types of requirement specification; user stories, use cases, Specification by Example, plain English and sketches. User stories are used by most of the teams, as indicated by ten teams (seventeen interviewees). One team stated only to use mockups/sketches to define the requirement. Two teams make use of use cases. Although most teams use user stories, almost all the interviewees perceived that this is not “the holy grail”. The results are visualized in Figure 6: Specification Techniques used and further elaborated in this paragraph.

FIGURE 6: SPECIFICATION TECHNIQUES USED3

3 _{Explanation of 16 in total teams: 1 team combined User Story with Use Cases, 1 team combined User Stories with Use Cases and Specification}

by Example, 1 interviewee described two teams. 1 team had difference between high-level and low-level requirements. 5 8 3 12 2 5 7 0 5 10 15 Elicitation

Elicitation Techniques

Large Workshops Focus Groups Brainstorm Sketch Prototyping

Prototyping Observations Interviews

10 2 ₁ 2 ₁ 0 5 10 15

Specification Technique per team

Page | 23

One of the developers stated the following on user stories in Scrum:

”I perceive that it is best for every team and the customer to decide how to describe the

requirements. In my three years of experience with Scrum development I have never used anything else then user stories, since this worked very well in our teams.”

A user story as elaborated in the theoretical framework, is perceived as not enough for teams. Acceptance criteria and enough conversations are perceived as essential in requirements. Eleven interviewees specifically argued that acceptance criteria are important. Regarding the acceptance criteria the following quotes are stated:

“A good requirement should consist of enough acceptance criteria to clearly understand the

demands of the requirement” – Scrum Master

“I can’t imagine that you use something else than user stories when you develop software with Scrum. A scenario or acceptance criteria is required though.” – Product Owner

When describing the acceptance criteria, this can be done in plain English or according to the Given-When-Then-structure. Four teams are using the GWT-structure, two other teams wish they used the structure after hearing about it. The GWT structure is perceived as readable, structured and easy to understand. A Product Owner stated:

“The criteria of the user stories were at this project defined as very small test scripts, such as: when you click on this, this should be returned. In the new project we use the GWT structure, if I could go back to my previous project we should have used the GWT structure then as well.”

As stated the conversations of the user story are essential. One developer stated: “User stories

in combination with acceptance criteria are limited, but in combination with discussions it is sufficient.”

So the most common result is that development teams are using user stories, but that other methods are also a viable option. Most of the teams commented that sketches or mockups are used together with the user story in order to make the requirement clearer. In that case some interviewees stated to start off with a very high-level user story and keep detailing (with sketches or mockups) until it was clear enough. Two teams create software based on business processes, using user stories as a baseline and added process models to clarify what the system should support. On the other hand two teams did not even use a user story but a one line description of the requirement. The Scrum Master in one of those teams stated the following:

“It was agreed with the customer to implement a CRM system out-of-the-box, with customer representatives involved. A high-level sentence or some observing was enough to understand the requirement. We wrote only the high-level sentence and were continuously asking the customer for feedback.”

Page | 24

Regarding use cases, seven people argued that it may be a viable option to use in Scrum. Although most interviewees could not state ever using use cases in Scrum, most have a distinct opinion. Two quotes from first a Business Analyst and second from a developer.

“Do not use Use cases, they take a lot of time to define correctly and could be complex for unexperienced customers. Rather try using Specification by Example to clarify requirements for the customer and development team.”

“Use cases are also a possibility. In order to make a use case work, the team has to decide on how much detail you are going to define in the use case. With less details in the use case it is also a very reasonable option to use.”

Organization B does describe the requirements in use cases. The organization is in the process of implementing Specification by Example to replace most of the requirements. Although Specification by Example is perceived as better, the organization believes that use cases will still play a role on the more flow-based requirements. “The application-flow is not really possible in

Specification by Example, because you have to extend it with sketches or something similar, but this makes the overview less visible. The coherence of user stories is harder to see than when using use cases.” All three interviewees of organization B believe that use cases still need to be used in

some occasions.

Ten interviewees argued that there is not one way to describe requirement, but that per project and per team should be decided what should be the best technique suited for the job. The manager software development stated: “There is not one way to describe requirement. I think

that best is to look at feature type specific requirement specification. Calculating intensive features might require another specification than process driven or visual features. You could say in the baseline use one type of requirements specification, and then based on different types, use additional specification techniques.” A Business Analyst stated: “Get one level of abstraction, that one of the customer.”

Mister Verheyen argued that “In Scrum you can describe [requirements] as abstract as needed.

Because of the transparency principle, the requirement must be described in a language that both the team, the Product Owner and the stakeholders can understand it.” The team should decide

how to describe the requirements.

4.1.3.

H

-

&

-

This paragraph was introduced to study if there was a difference in the techniques used for high-level and low-high-level requirements. The findings on this subject are that no or very little difference in both elicitation and specification techniques between high-level and low-level requirements. It was stated in one of the interviews that the high-level requirements contain less details and that sometimes it is small sentence with the wish. Most teams tried to describe the high-level requirement in user story form to have consistency in the different levels of requirements.

Page | 25

4.2.

B

USINESS

I

NVOLVEMENT

&

F

EEDBACK

According to five of the interviewees; business involvement is an important aspect in the process of a requirement. The amount of business involvement could be related to the formality in which requirements should be documented. A developer stated: “Commitment of the customer is

crucial in succeeding with user stories.” If you can have enough discussion with the customer

there is no need to write a large requirement document. A Business Analyst stated: “The key to

really understand the business is to have a lot of conversations and discussions about the wishes of the customer.” A Scrum Master stated: “Because there was enough business involvement, this [refining at the last moment] was not a problem.”

In one of the Scrum projects interviewed, the project had a setup where the customer was continuously involved reviewing the software and providing feedback. As also stated in the previous paragraph: “Because the continuous feedback with the client, it was unnecessary to

describe requirements at all.” A developer stated:

“When involvement is limited, user stories should be described more detailed and additional sketches or documentation should be provided.”

Other projects had a team setup where some of the Scrum team members were from the customer, usually as a tester. One Product Owner stated: “Business stakeholders were involved

in the team as tester, creating a very short feedback loop.” Mister Verheyen argued that a

requirement is never complete until it is implemented. Having a dialogue about the requirements is most important, this is only possible with business involvement. He stated the following: “From

the more traditional methods, an illusion is created that as long as you think long and hard enough about requirement and describe everything, that it may guarantee success. But when we look at it practically, we see that this is not the case (…) Most important is to have this dialogue.”

Feedback on developed software could be done at several points in time. The team should always endeavor to get feedback as early as possible, but this is only possible with sufficient business involvement. “A general guideline is that if feedback is given in the Sprint, as long as it fits in the

size of the PBI and it does not affect the Sprint goal, it can be dealt with in the same Sprint.”

Otherwise the feedback should be placed on the backlog. The second option to have the client give feedback is during the Sprint review. Feedback in the Sprint review is dealt with in a different way. Feedback is then placed on the backlog and “is refined as soon as possible, so an estimation

and priority can be assigned.” Some teams struggled with feedback in the Sprint review, as it was

very important and had to be done as soon as possible. “We always create a buffer to fix bugs or

apply feedback in every Sprint.” But in most projects Sprint Review feedback was placed on the

Page | 26

4.3.

F

ORMALIZED

R

EQUIREMENTS

In paragraph 2.2. is stated that requirements are associated with other elements than only knowing what to develop. In this paragraph the study tries to find out if the requirements are documented in a formal way and if any of these other aspects are associated during the project. The findings on this topic are fragmented. Seven of the teams discussed not having to deal with formalities when describing requirements, while five teams had more formal requirements because of signoffs.

“As Product Owners we only have sign-offs with the board on the vision. The created roadmap is discussed with the board and then accepted or not. Other requirements do not need any formal signoff.” – Product Owner.

Other teams that had a more traditional client, where formalized requirements were needed, had different options to deal with these formalities. Most Product Owners try to keep the formal process away from the Scrum Team.

“The client demanded that the requirements were formal and signed before development. In order to do that the requirements were created in small chunks and signed one by one.” – Business

Analyst.

The requirements that were created in small chunks were not specified in a large document, but as just-enough and signed based on those minimal described requirement. If the requirements should be described in a formal way, it is perceived as not a problem, but the stakeholders must accept that the amount of time needed to describe requirements might be higher than the 10% defined by Scrum. Another team dealt with the request to formally sign the requirements as following:

“To have a formal kind of signoff the team decided that the PO sets a new PBI to the status ‘Approved’ as a formal signoff for approving the requirement. The requirement was set to status ‘Accepted’ if the requirement is tested and perceived as done by the PO.” – Business Analyst

One could state that organizations with less trust in the Scrum team require a more formal approach on requirements. One of the interviewees stated the following about that:

“Because you are documenting requirements extensively you need to have trade-off and acceptance testing etc., because you can then blame someone if the requirement is not met. With Scrum we say we do this together and we are responsible together.” – Product Owner/Scrum

Page | 27

4.4.

S

LICING

R

EQUIREMENTS

In the theoretical framework was stated that teams have problems with slicing the requirements. The findings according to the interview state otherwise. Almost all teams say that they do not have problems with slicing the requirements. There were four interviewees that stated having problems with slicing. One of the interviewees that had difficulty with slicing stated:

“Slicing was a problem. Sometimes it was just decided that the requirement was not able to fit in one Sprint, causing it to have overlap in more Sprints. Because the PO, stakeholders and the development team were okay with this, no action was required. If I look back now, I think it would be better to slice.” – Scrum Master

Other teams had no difficulty slicing. The teams used three techniques to slice a requirement. First technique was slicing the requirement to a bare minimum and an optimal implementation of the requirement, this was used by three teams. The Scrum Master of one of those teams stated:

“Slicing was done to first create a requirement in the ugliest form and in the second Sprint refactoring the UI. This not the best method because you can’t go live, but the stakeholders can see the functionalities and give feedback.”

The second method, used by one team, was to slice in a horizontal way, meaning slicing in architectural areas. For example: first Sprint develop database, second Sprint the middle tier and third the UI. The discussion in Scrum is that you cannot slice horizontally, because you cannot give value with only the database (Ratner & Harvey, 2011). There was one team doing this and the developer stated the following:

“This [slicing in layers] means that we are slicing horizontally (…) but because a platform is created for other developers only a web service will still deliver value for our stakeholders.”

As long as these teams make sure that they still deliver value each Sprint, they are not violating the key ideas of Scrum. A third option of slicing is to split functionalities in half. If the customer can still work with half the functionality in the first Sprint, you are still delivering a value. The same applies for slicing per UI, for example: decide to first slice for mobile UI and in a later Sprint make the UI also desktop compatible.

In the expert interview two slicing methods are suggested. He argued to slice on either the user/actor and on the story/action. “An example is to do it for different user, so slice on user level.

A requirement for every user type.” The second method is to slice on the story level is for example

to slice every step of a workflow. Although the stakeholders cannot use the entire workflow, you can make sure that the specific step is finished. “At the end of the Sprint the requirement does

not have to be in production, but a rule of thumb is that from a technical point of view, it should never be a problem to do that.”

Page | 28

4.5.

R

EQUIREMENTS AFTER DEVELOPMENT

An underexposed aspect of requirements is what to do with requirements, after the requirement is implemented. Requirements are in sometimes used in order to understand the system after it is delivered. Use cases for example give an overview of how the system is developed. Though looking at the results in most interviews, use cases are not used. In order to understand how the system works, how do we deal with that documentation? Product Owners 1&2 stated:

“One thing that is unknown is the amount of documentation that is required. Because the user stories are very small chunks it will be hard in the future to understand the entire system when looking back at the user stories. [We believe] it is also not possible to go through the entire code. When the system goes into operation or a new team arrives, how is it possible for the new team to understand the system?”

The organization that is implementing Specification by Example also does not know how to deal with that documentation. The manager at organization B stated: “We do not yet know how to

document the functions of the system. The user stories can be linked to an epic, but the question is, will this make visible enough how the entire system works. It is something that we are currently thinking about, by using inspect & adapt.”

A discussion based on these answers with an Agile Trainer lead to the following opinion: “We

should think about if documentation should be part of a requirement. Shouldn’t we focus on describing the requirement what to build and look at another document to use when to better understand the system? Maybe a user manual is sufficient, or maybe if you write test cases in a readable way it might be a solution to understand the software.”

The Scrum expert argued that documenting the system should be part of the Definition of Done, if this is requested by the stakeholders or the operational team. He stated the following: “Some

teams that I worked with used use cases as the method to document the system, they did not focus on getting the requirements 100% up front, but creating the use case during the development. So instead of documenting the expectation, document the reality.” Although use

cases are used to document the system, this is still not a reason to describe requirements also in use cases. “You still don’t need to describe your requirements up front as use cases. The user story

is an essential use case, the switch between those is made pretty fast. Plus you can get the tendency to describe the requirement (if using use case) up front more exhaustive because then you have to do less during the Sprint.” He also argues that if a wiki or other tool is sufficient, the

team should not make it harder for themselves and do that.

A finding of this study is that teams are struggling dealing with this issue. Some teams are using a general rule of thumb that complex functionality should be explained in supporting technical documentation. Two teams discussed to create technical documentation at the end of every Sprint to understand the software. The expert stated a possible solution to help to solve this finding.