The holy grail: Business value in agile
software development
Name: Raico van ElsenStudent number: 12832928 Word count: 12407 Referencing style: APA Contact info: 0683222400
Address: Kanaalstraat 46, 3531 CK Utrecht Mail: raicovanelsen@outlook.com
Supervisor: Hans Borgman Second reader: -
Ethics approval: EC 20200412110406 MT: Digital Business
Theme: Business value in agile software development Date: 14-8-2020
Statement of originality
This document is written by Raico van Elsen who declares to take full responsibility for the contents of this document.
I declare that the text and the work presented in this document is original and that no sources other than those mentioned in the text and its references have been used in creating it.
The faculty of Economics and Business is responsible solely for the supervision of completion of the work, not for the contents.
Table of contents
Abstract ... 4
1. Introduction ... 5
2. Literature review ... 8
2.1. Stakeholder value creation ... 8
2.2. Business value of software development ... 10
2.2.1. The Business value model ... 11
2.2.2. The Software Value Map ... 11
2.2.3. Diversity of value aspects ... 13
2.3. The customer perspective on business value ... 14
2.4. Agile requirements engineering practices ... 16
2.4.1. The release planning process ... 17
2.4.2. Scrum Sprint Planning ... 17
2.4.3. Assessing business value ... 18
2.5. Business value and story points analysis ... 19
3. Research Method ... 21 3.1. Respondent selection ... 22 3.1.1. Agile expert ... 22 3.1.2. Product owners ... 23 3.2. Characteristics of respondents ... 23 3.3. Interview procedure ... 24
3.4. Data analysis strategy ... 25
4. Results ... 27
4.1. Stakeholder satisfaction as driver for value creation ... 27
4.2. Constructing business value ... 29
4.3. Customer value perspective ... 32
4.4. Assessing business value ... 34
4.5. Business value and Story points analysis ... 36
5. Discussion and Conclusion ... 38
6. Implication ... 40
7. Limitations & future research suggestions ... 41
Literature ... 44
Appendix A – interview guide ... 47
Appendix B – Data analysis scheme ... 49
Abstract
Since business value creation has become a central focal point for new agile software development projects (Racheva, Daneva, & Sikkel, 2009), the purpose of this research is to explore how business value is defined and created in agile software development projects. This to ultimately make progress in establishing a common ground on which the concept can be understood and used in agile software development. A qualitative research approach was chosen to enable the exploration of this central concept in agile software development. In-depth semi-structured interviews were conducted with ten product owners and one agile expert. The data collected through these in-depth interviews were analysed and described in the results and discussion & conclusion section. The creation of valuable software can be described as a process of managing the diverse and sometimes opposing value perceptions regarding business value from different stakeholders in the development project. The most important stakeholder in this value creation process is embodied by the customer of the software product. In the construction of the business value concept, two main value perspectives are considered important: the customer value perspective and the internal business perspective. The intuitive assessment of business value by product owners is substantiated by customer interaction and a long-term product vision embodied by a product roadmap. In this assessment, story points are differentiated from business value estimations.
Keywords: agile; business value; stakeholder satisfaction; customer value perspective; agile requirements engineering; in-depth interviews
1. Introduction
The agile way of working has become the new standard in software development now that most organisation shift their cost perceptions by a value-centric view on software development (Racheva, Daneva, & Sikkel, 2009). The first and foremost promise of agile software development (ASD) is that it is focussed on satisfying the customer through creating software products with maximum business value (Fowler & Highsmith, 2001). In the normative agile literature, it is assumed that by employing iterative development and close collaboration with the customer, this way of producing software results in more valuable software products for the intended customer (Fowler & Highsmith, 2001). In line with this customer-centric perspective, some researchers even argue that business value is the most important dimension on which requirements elicitation and prioritization should be based in ASD (Cao & Ramesh, 2008; Ramesh, Cao, & Baskerville, 2010). This focus on business value as a central concept is however rather new for the software development world in which software was long perceived from a value-neutral approach. In these times software was perceived as necessary but not value-adding for organisations (Boehm, 2003). Nowadays this perspective is totally reversed as organisations spent a large part of their investments on software as a way to gain competitive advantages (Lee, Choi, Lee, Min, & Lee, 2016). These large investments result in the fact that creating high business value software products is one of the most pressing themes in software development world (Dingsøyr & Lassenius, 2016). Even though business value is such a critical and key concept in agile software development, currently our understanding of what the business value of software encompasses is however very limited (Alahyari, Svensson, & Gorschek, 2017; Heidenberg, Weijola, Mikkonen, & Porres, 2012; Racheva et al., 2009).
Therefore, the purpose of this research is to explore how business value is defined and created in ASD projects. This to ultimately make progress in establishing a common ground on which the concept can be understood and used in ASD.
Better understanding the business value concept in the context of agile software development is relevant both theoretically as well as practical. As the business value of software becomes increasingly important in determining the success of software development projects, one would suspect that there is sufficient research on this critical success factor. Currently, however, the research field on the value of software is very underdeveloped (Alahyari et al., 2017; Heidenberg et al., 2012; Racheva et al., 2009). The general view on business value is that it refers to some form of monetary worth or revenue (Heidenberg et al., 2012). This definition is however too narrow for a software development context in which the concept can also refer to some form of intangible value as shown by the Software Value Map framework (Khurum, Gorschek, & Wilson, 2013). In this framework, the value of software is captured by four value perspectives which are: financial value, customer value, internal business value, and innovation and learning value. In ASD, Alahyari et al. (2017) found that business value of software is still mainly defined through a customer value perspective. The researchers however also exposed that business value of software is a complex concept, in which value can relate to a wide variety of value components. Besides the SVM framework and the business value research of Alahyari and colleagues, little progress has currently been made on better understanding the business value construct in ASD (Racheva et al., 2009; Alahyari et al., 2017). By exploring how business value is defined and created in ASD, an important step in developing our understanding of this key concept can be made.
In the agile way of working it is assumed that practitioners through practices such as face-to-face communication and close collaboration with the customer should be able to understand customer needs and transform a product vision into a software product with business value (Fowler & Highsmith, 2001). These principles are however more an ideal situation then the reality of most agile practitioners since customer access and participation is most of the time limited (Ramesh et al., 2010; Inayat, Salim, Marczak, Daneva, & Shamshirband, 2015).
Together with a very limited understanding of what business value of software encompasses, estimates and assessments of business value are mainly based on previous experience or an intuitive feeling of the business stakeholder or product owner (Heidenberg et al., 2012). As business value becomes one of the most important dimensions on which decisions in ASD are made, agile practitioners encounter challenges regarding requirements prioritization and implementation during development. Moreover, communication between business stakeholders and development teams about value creation is rather challenging (Heidenberg et al., 2012). Clearly, our current understanding of the business value construct in ASD is rather limited and there is a strong need to better understand and more accurately define this key concept in ASD so that practitioners can make better use of the concept (Racheva et al., 2009; Alahyari et al., 2017). Therefore, the following research question is proposed:
How is business value of agile software development defined and how is business value created during agile software development projects?
The research question is answered by conductingin-depth interviews with product owners of agile teams about the software project or product they were currently working on. A semi-structured interview guideline was constructed based on the propositions established from the general agile literature and software value literature. The theoretical grounds on which the propositions are based is described in section 2. Followed by an explanation of the data collection and analysis method used to explore the propositions in section 3. The results of this research, structured by the propositions indicated in section 2, are presented in section 4. In section 5 the main findings of the research are discussed. This research concludes by describing the implications of this research followed by the main limitations and future research suggestions.
2. Literature review
As of the exploratory nature of this research, the theoretical framework that constructs the concepts used in this research is structured based on five propositions that will together answer the proposed research question. First, the business value concept is placed in the context of project success factors, and the role of stakeholder satisfaction is made explicit. Followed by a description of the multitude of value aspects that is related to the business value concept in software development. After having described the multiple perceptions on the concept, the importance of the customer perspective will be explained. The theoretical framework will conclude by describing the importance and challenges of business value in the field of agile requirements engineering and the role of story points in the assessment of business value of software development.
2.1. Stakeholder value creation
As indicated previously, business value has become one of the key drivers for decisions in agile software development (Fowler & Highsmith, 2001). To define and better understand the concept, it is important to first of all place the business value concept in context. Ultimately, business value can be seen as an outcome of a particular software product that is produced in a software development project. The outcome of software development projects can be placed in the broad context of project success factors. In this field of research, project success has been described as a composition of two components, project management success and product success (Baccarini, 1999). One important element in both project management success and product success evaluations is stakeholder satisfaction (Baccarini, 1999).
The concept of stakeholder satisfaction has not been immediately adopted in the software development world, because software as a product has long been perceived in terms of a cost perception. In this perception, software was perceived as necessary but not beneficial or value-adding for organizations or stakeholders (Boehm, 2003). With the introduction of the
concept of Value-Based Software Engineering, Boehm (2003) contrasted this value-neutral perception in the software development field with practices such as "Value-based requirements engineering". In this practice critical stakeholders are identified, their value propositions are made explicit and these value propositions are translated into objectives for the systems. Nowadays, agile software development projects can be seen as value creation processes, in which all steps taken should be focussed on value creation for the stakeholders considered important in the project (Hannay, Benestad, & Strand, 2017).
In the agile way of producing software, the main stakeholder that should be satisfied is the customer of the software that is produced (Fowler & Highsmith, 2001). It is even assumed that business value from a customer perspective is the main driver for decisions regarding value creation in software development (Cao & Ramesh, 2008; Ramesh et al., 2010). This central focus on the customer as the only stakeholder to consider is in practice however challenged as product owners and representatives responsible for software projects have to look at value creation from a broader perspective. From this perspective, a more diverse group of stakeholders should be recognized (Hannay et al., 2017; Paetsch, Eberlein, & Maurer, 2003). In today’s agile development projects, development is focussed on different stakeholders that can include, enterprise managers, system developers, deployers, and maintainers, users, the general public, or even society (Hannay et al., 2017). Stakeholder involvement has therefore even become a key practice in agile software development (Daneva et al., 2013; Paetsch et al., 2003). This means that the whole development process has become a value creation process in which the product owner responsible for the software product should take into consideration the value propositions of all these different stakeholders (Hannay et al., 2017). Only by considering all these different value perceptions, a valuable software product can be developed.
From this line of reasoning, it can be argued that not only the customer as the single most important stakeholder involved has to be taken into account in order to create valuable
software products. Instead, the value perceptions of different stakeholders included in the project should be considered in this value creation process. This leads to the first proposition regarding business value creation from a stakeholder satisfaction perspective:
Proposition 1: Stakeholder satisfaction is the core driver for decisions regarding value creation in agile software development
2.2. Business value of software development
The fact that multiple stakeholders are considered important in the construction of valuable software still does not say anything about what then the actual value of software is. Even though the idea of placing business value in the center of software development projects is rather new for the software world, the business value concept itself has already a long-standing history in other fields. Business value is mainly discussed and defined in management and economic streams of research (Khurum et al., 2013). In these fields, business value is described in terms of different viewpoints such as production value, differentiation value, and shareholders' value (Khurum et al., 2013). In these streams of research, business value most of the time relates to some form of monetary worth or revenue for the organisation (Heidenberg et al., 2012).
As indicated previously, this pure focus on the monetary aspect of value is however too narrow for a software development context. Now that this enlarged perspective on the value of software has become a more pressing theme, some researchers have made progress in developing frameworks to define the business value concept in relation to software. These frameworks will be described to show the diversity of value components that are present when describing the business value of software development. In the following sections, first, the business value model presented by Heidenberg et al. (2012) will be described. Subsequently, the Software Value Map presented by Khurum et al. (2013) will be explained.
2.2.1. The Business value model
The first framework that will be discussed, is the Business Value model proposed by Heidenberg et al. (2012). The researchers propose a model for assessing the amount of business value of any given feature that will be included in the software product. The main goal of the model is to make the term business value more explicit so that discussions between different stakeholders about this central concept can be more unambiguous. The model as proposed consists of six attributes that any given feature can have. The first attribute is Monetary Value which relates to the estimated monetary worth of the feature. The second attribute is the Market
Enabler. The feature is considered a market enabler if it facilitates an introduction to or
retaining a market position. A feature is considered a Technical Enabler when a feature functions as a foundation for future functionality. Competence growth of the feature focusses on the degree to which the feature requires that new knowledge and tools are used by the developers. The attribute Employee satisfaction is included in the model since software development and programming are regarded as creative work activities in which the satisfaction and well-being of employees can be seen as important indicators of success. The last indicated attribute is Customer Satisfaction. Customer satisfaction is important in the creation of software, as satisfied customers often return and will continue the collaboration. Each of these attributes consists of four ordinal values in which four is the highest ranking. The total business value is constructed by the expression of the aggregation of these attributes (Heidenberg et al., 2012).
2.2.2. The Software Value Map
A more detailed and comprehensive framework on the business value concept in software development can be found in the work of Khurum et al. (2013). The researchers conducted an extensive literature review combined with an industrial case study research and captured the business value of software in their so-called Software Value Map (SVM). The researchers
constructed this framework based on the categories used in the Balanced scorecard. The Balanced scorecard is a measurement tool from business research that integrates the four perspectives of business performance; the financial, the internal processes, innovation & learning, and the customer (Kaplan & Norton, 1996).
The researchers used these different perspectives to categorize the different value aspects related to software from the literature. Underlying each of these perspectives, the researchers defined several value aspects, sub-value aspects, and value components that are hierarchically organized and altogether describe software value through these different lenses in an overall framework (Khurum et al., 2013). In this construction, value components are the endnotes of the hierarchy, which are measurable value factors. To portray the diversity of value elements related to the construction of business value of software, the different value perspectives, and their main value aspects will be discussed subsequently based on the work of Khurum et al. (2013).
The financial perspective is concerned with the strategic long-term objectives of the software producing firm and the related financial measures of these objectives such as revenue growth and earned value (Kaplan & Norton, 1996). In the SVM framework, this value perspective is constructed by the value component shareholder value which solely refers to the overall monetary worth of the organisation.
The innovation and learning perspective sheds light on the intangible assets needed to support the internal processes of developing software such as skills and capabilities that are of value for the software developing organisation (Kaplan & Norton, 1996). In the SVM framework, this perspective is composed of three different value aspects which are: Intellectual
capital value, the value of technology, and Innovation value for the market.
The internal process perspective is concerned with the process of delivering value to the customer and looks at value elements regarding efficiency and productivity of software
development (Kaplan & Norton, 1996). In the SVM framework, this perspective is translated into the internal business perspective on software value. From this internal business perspective, production value and differentiation value of the software product are considered important.
Lastly, the customer perspective is twofold, it first of all looks at the perceived customer value in terms of the performance and quality of the software that is delivered. Secondly, it looks at the outcomes of these perceived value propositions such as customer satisfaction (Kaplan & Norton, 1996). In the SVM framework, this division is captured by the perceived
value and customer lifetime value of the software product for the customer.
The integration of all these different value perspectives, value aspects, sub value aspects, and value components into one overarching framework can be seen as one of the first comprehensive models that gathered different perspectives and dimensions on the value of software.
2.2.3. Diversity of value aspects
What becomes apparent from these proposed models is that business value of software is based on different value aspects and can be seen from different perspectives. In the Business Value Model (Heidenberg et al., 2012) besides customer satisfaction also technical value, monetary value, market value, and internal value (competence growth and employee satisfaction) are considered in constructing the total amount of business value. The same idea applies to the SVM framework (Khurum et al., 2013) in which a financial, internal business, innovation & learning and customer perspective generate the total value of software. The multiple perspectives on the business value of software development are also confirmed by the work of Alahyari et al. (2017) who used the SVM framework to look at the business value concept in ASD. The researchers found that agile practitioners indicate that business value is constructed based on value aspects in all the value perspective domains of the SVM framework. This
contrary to the normative agile literature that phrases business value only in terms of the customers’ perception of software value (Fowler & Highsmith, 2001). This reasoning provides support for the following proposition:
Proposition 2: Business value in ASD is defined based on a variety of value perspectives.
2.3. The customer perspective on business value
Even though based on the prescribed models and the current literature on business value it can be argued that the business value concept encompasses a broader range of perspectives than only the one from the customer, research does indicate that value from a customer perspective is highly important in ASD (Alahyari et al., 2017; Misra, Kumar, & Kumar, 2009). This, in line with the promise of the agile manifesto that states that agile is focussed on delivering valuable software for the customer (Fowler & Highsmith, 2001). Since this customer perspective is so important in the construction of business value in ASD, this customer perspective will be further explained. Currently, there is very little empirical research that specifies value from a customer perspective concerning software products. Therefore, the earlier presented SVM framework from Khurum et al. (2013) will be used to explain which value aspects could be considered to determine value from this customer perspective.
In this perspective, value for the customer is essentially split into the direct perceived
value of the software product and the outcomes that are a result of these value propositions
which are clustered in the customer lifetime value (Khurum et al., 2013). The perceived value of the software product has essentially three main components. The intrinsic value relates to the characteristics of the software product such as reliability, functionality, portability, usability, and maintainability. The delivery process value captures the value of the delivery process in terms of the time, quality, and cost of this process. And the last value aspect is related to the user of the software product. In which the user experience value includes the same
elements as indicated in the intrinsic value element such as reliability, functionality, and maintainability of the software product.
The customer lifetime value of the software, which relates to the outcomes of the previously described value elements, is composed of revenue value and cost value for the customer. In summary, this customer perspective can be categorized in the intrinsic value of the software product for the customer, the intrinsic value of the software for the user, the quality of the software delivery process, and the outcomes that are a result of these value elements.
From this customer perspective, Alahyari et al. (2017) found that the most important value aspects from a customer perspective are Delivery Process w.r.t. time, Perceived & Actual
quality of the software, and the Cost of the software. The researchers related these value aspects
to the classic golden triangle factors (time, quality, and cost) from project management (Westerveld, 2003). Interestingly, these more short-term project-related value aspects were considered more important than the more long-term product related value aspects such as
Usability and Performance of the software product. This is interesting because, in the agile way
of working, product owners should be mainly concerned with transforming a product vision into a valuable software product for the customer (Fowler & Highsmith, 2001; Alahyari et al., 2017).
One important limitation in the research of Alahyari et al. (2017) is the fact that the researchers did not include the actual customers of the software product in their research. Therefore, it is questionable whether the project related value aspects are actually considered more important than these so-called product-related value aspects. What however becomes apparent from this exploratory research and the SVM framework, is that in the construction of business value from a customer perspective both short-term project-related value aspects, as well as long-term product related value aspects, are considered. Therefore, the following proposition to construct value from a customer perspective is proposed:
Proposition 3: From a customer value perspective both short-term project and long-term product value considerations are made to assess business value
2.4. Agile requirements engineering practices
As a result of the fact that business value has gained such a central position in software development, several practices in ASD have adopted the concept as their single focus point. The most important domain in ASD that is focussed on business value creation, is the practice of agile requirements engineering (Cao & Ramesh, 2008). Requirements engineering includes activities to identify, analyse, document, and validate requirements for the system to be developed (Ramesh et al., 2010, p. 450). Business value has such a pivotal role in this specific area of software development because during requirements engineering the functionalities of the system that should generate value are discussed and agreed upon.
The practice of requirements engineering in the field of agile development is different from traditional requirements engineering settings because agile projects are more iterative in their nature and therefore initially created requirements can evolve over the course of the project (Inayat et al., 2015). Due to this more iterative nature, practices such as face-to-face communication, direct customer involvement and interaction, iterative requirements engineering, the use of user stories and constantly prioritizing requirements are introduced so that the customer needs can be better understood and therefore more effectively transferred to the development team (Cao & Ramesh, 2008; Inayat et al., 2015; Ramesh et al., 2010).
Most of these practices are focussed on the involvement of the customer or business stakeholder in the development project so that the product that is being developed is better aligned with the customer needs and therefore holds more business value for the customer. Besides these more general practices previously indicated, in two of the most used agile development methodologies, Extreme Programming (Beck, 2000) and Scrum (Schwaber,
2004), business value creation is at the center of attention. These will be discussed subsequently.
2.4.1. The release planning process
The release planning process is part of the Planning game in extreme programming (XP), which is a method for software developers to create consensus between business stakeholders and the development team (Beck, 2000). The Release planning process has the ultimate goal to maximize the value of the software produced by the development team (Beck, 2000). To reach this goal, the development team should take the most valuable functionalities, based on the customer’s perception into production as soon as possible.
The game consists of three phases. The first phase is the exploration phase in which the customer describes what kind of software they want and how this software should work. These descriptions are written on so-called user story cards which serve as the guiding elements for the development process. The second phase relates to the commitment phase in which customers and developers further discuss the functionalities of the software that were created in the exploration phase and agree on the release date. The last phase is the steering phase. This phase is aimed at updating the plan that was created based on what was learned during the development process by business and development. In this phase, iteration, recovery, new story and reestimate are the so-called moves that can be used by the development team to make sure that only the most valuable stories are implemented in the ultimate system or software product. Overall the planning game should contribute to creating software functionalities that are considered most valuable for the customer (Beck, 2000).
2.4.2. Scrum Sprint Planning
Another methodology that is dependent on the business value construct and is most often used in agile software development is Scrum. The Scrum methodology will be explained based on
the work of Schwaber (2004). Scrum is an agile project management methodology. The methodology is based on the idea that software development projects cannot be fully planned upfront. Therefore, the work should be conducted in iterative sprints, which are periods of no more than 30 days in which the development process unfolds (Schwaber, 2004).
One specific element in the Scrum process that relates to the creation of the highest level of business value, is the sprint planning meeting (Schwaber, 2004). During these sprint planning meetings, the product owner comes together with the development team and discusses which functionalities or in Scrum called backlog items will be produced in the upcoming sprint period. This discussion should have the ultimate goal to produce those functionalities that hold the most business value. Therefore, prioritization of backlog items should be based on the value that each item holds. By assessing which items have the highest business value, a plan on which value-adding items the team will work in the upcoming sprint period is constructed. Ultimately, the product owner of the agile team is responsible for this prioritization process and the business value that the created software delivers (Schwaber, 2004).
2.4.3. Assessing business value
The release planning process in XP and sprint planning in Scrum are built on the assumption that the business value of the requirements that should be included in the system or software can be assessed and are clearly understood by the product owner, business stakeholder, and other stakeholders involved in the development project (Heidenberg et al., 2012). Furthermore, these practices assume that developers can closely collaborate with the customer, and the business stakeholder is readily available to steer the development project based on business value perceptions (Ramesh et al., 2010). These assumptions are however challenged since close collaboration with the customer is in practice rather difficult due to limited customer access and participation (Ramesh et al., 2010). Additionally, practitioners indicate that the business value concept is rather ambiguous and vague in most development projects (Racheva, Daneva, Sikkel,
Herrmann, & Wieringa, 2010). This means that assessing requirements based on their business value, is a process that lies predominantly in the expertise of the product owner and is most often based on previous experience or an intuitive feeling (Heidenberg et al., 2012). In the more traditional plan-based software development projects, this common understanding of what is considered valuable was mainly established through specifying requirements for the system extensively upfront (Bjarnason, Wnuk, & Regnell, 2011). In ASD it is however assumed that documentation is counterproductive as it decreases velocity and hinders productivity (Fowler & Highsmith, 2001). By utilizing direct customer interaction these requirements should evolve over the course of the development project (Ramesh et al., 2010).
This, however, results in the fact that a clear and generally understood idea that reconciles the different business value perceptions is missing in these indicated practices. And more important, business value assessment is predominantly dependent on the intuition and experience of the product owner that is responsible for the development project (Heidenberg et al., 2012). Resulting from this line of reasoning, the following proposition regarding the assessment of business value is established:
Proposition 4: Assessing business value in agile software development is mainly based on the previous experience and intuition of the product owner.
2.5. Business value and story points analysis
Due to the complexity of constructing the business value concept itself, currently, little progress has been made in finding approaches to measure business value as an outcome of produced software products (Dingsøyr & Lassenius, 2016). The earlier recalled work of Alahyari et al. (2017) tried to capture the measurements in use by agile practitioners that explicitly focus on business value assessment in ASD. Interestingly, the researchers found no explicit measurements or metrics used to assure that value is delivered to customers. The researchers
argue that such measurements to assess business value are missing because most software measurements are internally oriented (Niessink & Vliet, 1999). Such internally oriented measurements to assess performance and progress are mainly focussed on cost estimation and monitoring (Hannay et al., 2017). Monitoring and estimating the cost of software projects is a very complicated task due to the complex nature of the job at hand and the high level of uncertainty related to most software development projects (Jorgensen & Shepperd, 2006). This results in the fact that most software cost estimations have extreme high error rates (Jørgensen & Moløkken-Østvold, 2006)
Assessing and monitoring the business value of software is even more complicated since there is currently not enough methodological support to determine and monitor business value through particular measurements (Hannay et al., 2017). To overcome this absence of measurements to assess business value, some researchers have proposed evaluation tools that can be used for assessing the amount of business value of software functionalities. Hannay et al. (2017) propose the use of Benefit Points to assess the amount of business value of any given epic (bundle of user stories) that is created during software development projects. These Benefit Points essentially estimate how much a particular epic will contribute to the overall objectives that were set for the software development project. By this, the researchers state that it becomes possible to get an estimation of the amount of business value that will be generated when a particular epic is produced (Hannay et al., 2017).
Benefit Points analysis is however a rather new measurement tool that is not yet widely adopted in the agile development field. Due to the absence of such widely adopted value measurements, sometimes cost estimations are therefore used instead (Hannay et al., 2017). One often-used measurement to estimate cost and effort in agile software development is story point analysis. Story points are a unit for measuring the size of a given user story or feature that the development team is going to work on (Coelho & Basu, 2012). Story points are assigned to
each user story that is going to be developed in an agile development project. The underlying aspects on which story points are based can be the effort involved, the complexity, and the inherent risk of developing a particular story or feature (Coelho & Basu, 2012). It is important to recall that story points are a relative measuring tool and that the methodology is based on factors such as complexity of the software, effort involved, and risk related to production. These factors do not inherently relate to the amount of business value as a story with a high level of complexity or effort to produce does not automatically result in a high level of business value (Hannay et al., 2017). This reasoning leads to the last proposition regarding the measurement of business value:
Proposition 5: In agile software development the assessment of business value is differentiated from story point analysis.
3. Research Method
To answer the proposed research question, a qualitative explorative research approach was chosen for this research. To create as much rigor as possible, every research step in this thesis will be described and substantiated with arguments. By this, it becomes possible to create a clear chain of evidence and overcome limitations of reliability (Yin, 2011). Eleven in-depth interviews were conducted in the scope of this research. The interview guidelines for these semi-structured interviews were derived from the propositions as indicated in the theoretical framework. Semi-structured interviews can generate an in-depth understanding of the described phenomenon through the lenses of the interviewee and get an understanding of the underlying constructs of the phenomenon (Symon & Cassell, 2012). This is particularly important for the business value construct for two important reasons. First of all, as indicated in the research of Khurum et al. (2013) and Alahyari et al. (2017) the business value construct can be described
from different value perspectives. Understanding these perspectives through the lenses of the interviewee is essential to describe the concept in detail. Secondly, the current literature on the business value concept shows that the business value construct is based on several underlying value aspects and components. Furthermore, the different value elements as presented in the SVM framework are in no way exhaustive, and overlap between different value perspectives could present. By conducting semi-structured interviews, it became possible to capture these different elements and construct a comprehensive understanding of business value in the context of agile software development.
3.1. Respondent selection
3.1.1. Agile expert
The in-depth interviews were conducted with ten product owners of agile teams and one agile expert. Due to the exploratory nature of this research and the fact that the business value concept in a software development context has not been extensively researched besides the work of Khurum et al. (2013) and Alahyari et al. (2017), one interview with an agile expert was conducted. According to Bogner and Menz (2009), three types of expert interviews can be distinguished based on their function. One of these expert interview types is called the exploratory expert interview that was integrated into this research. The exploratory expert interview was used in this research to gain a better understanding of the complexity related to the business value concept in agile software development. Additionally, based on the knowledge gained from this interview, the developed propositions were further improved.
A person is considered an expert and knowledgeable of a subject based on their detailed knowledge of a particular subject, their particular status or community position (Kaiser, 2014). The agile expert included in this research was selected based on his extensive experience as a consultant, coach, and trainer in the field of agile software development. Since he had worked
with and coached a large number of agile teams, he possessed specific knowledge regarding business value creation in the context of ASD.
3.1.2. Product owners
Product Owners were specifically chosen to explore the business value concept in ASD for two important reasons. First of all, product owners are responsible for translating a particular product vision into specific software functionality (Schwaber, 2004). Therefore, product owners should be able to relate the business value concept to specific features and functionalities of the software that should generate this value. Secondly, product owners have the ultimate responsibility for the prioritization of the product backlog of the agile team. As indicated in the theoretical framework, a product owner should make this prioritization predominantly based on the expected business value that each item on this product backlog holds (Cao & Ramesh, 2008; Ramesh et al., 2010). Hence, product owners are constantly assessing the business value construct during software development and should, therefore, be able to describe how business value considerations are made during software development.
The product owners and agile expert included in this research were accessed via a large IT-benchmarking organisation and the personal network of the researcher. The selection premises that were set for the inclusion of respondents will be discussed in more detail in the following section.
3.2. Characteristics of respondents
All product owners included in this research were at the time of the interview working in a software development project. This was the first important premise because in order to relate the business value concept to a specific software product it was important that the product owners could discuss their value considerations in relation to this specific product and did not theorize on the concept. Some product owners were responsible for two software products at
the same time. To guarantee the in-depthness of the interview, one software product was chosen on which the interview was focussed.
Secondly, all product owners used the Scrum project methodology in their development project. This enabled the research to place the business value concept in the specific context of ASD. The fact that all product owners used the Scrum project methodology also supported the exploration of the requirements prioritization process in their software development project. This is essential in developing our understanding of the concept since this prioritization process should be mainly guided by the business value that each item or requirement holds (Cao & Ramesh, 2008; Ramesh et al., 2010).
The product owners developed software products for different industries and markets. Four product owners developed software products for the municipality market. The other product owners were developing products for the banking industry, e-commerce sector, insurance market, plantation industry, healthcare industry, and one product owner was working on a new ERP solution for his own organisation. The consequences related to the fact that four product owners produced a software product for the same industry will be discussed in the limitations section of this research.
3.3. Interview procedure
The in-depth semi-structured interviews that were conducted in the scope of this research lasted between 35 and 60 minutes. The interview guideline that was used to structure the interviews can be found in Appendix A. All interviews were digitally conducted due to the corona regulations that were in place in time of the research.
To relate the business value concept to a specific software development project, the interviews started with questioning the respondents about the software project they were currently working on. To make sure that all value considerations could be mentioned by the respondents, the product owners were asked when they knew they had delivered value in the
scope of their development project. This broad introduction to the value aspect of their software project enabled the respondents to deliberate on those value aspects and components that they considered important. This was essential to guarantee the explorative nature of this research. Then, the product owners were asked about the main functionalities of their software products and the value of these functionalities. After having explored how the respondents construct value in their development project, it became possible to focus the interview questions on how business value was constructed and assessed, how business value was defined from a customer perspective and how story points were related to the business value concept. By first of all extensively exploring the software product the product owners were working on, it became possible to phrase the questions regarding value creation concerning a specific feature or functionality. Secondly, generating a comprehensive understanding of the software product the product owners were developing also increased the transferability of the research results which is essential in qualitative research due to its more flexible character (Runeson & Höst, 2009).
At times respondents tend to talk about the business value concept in more general terms not related to a specific software project. To keep the respondents from thinking and conceptualizing the business value concept for themselves, the respondents were as much as possible asked to relate their value ideas to the specific project or product they were working on.
3.4. Data analysis strategy
The data analysis process was based on the process as described by Miles and Huberman (1984). The researchers split the process of qualitative data analysis into the reduction of data, the schematic projection of data, and abstracting conclusions from the found analysis to ultimately verify the found conclusions (Miles & Huberman, 1984).
After all the data was fully structured, the data was analysed in steps with the use of Nvivo software. First of all, all interview transcripts were read, and the first initial codes were
assigned to parts of the transcript that were considered important in relation to the proposed research question. At this stage, the data needed to keep its flexibility so that important elements not directly related to the propositions could also be identified. After having identified the first codes, themes were created that clustered these different codes into categories. Having constructed these themes, it became possible to group and integrate the data from the different interviews and specify sub-themes whenever this was necessary. After having created the themes from the data, these themes were then compared with the initial propositions derived from the literature.
After the analysis process was finished, the themes and underlying sub-themes that were related to proposition 3 regarding the diversity of value perspectives were compared with the SVM framework from Khurum et al. (2013). By this, it became possible to compare the different value aspects indicated by the product owners with the value aspects indicated in the framework. A detailed description of these value aspects will be discussed in the results section of this research.
By following the analysis process as indicated, it became possible to, first of all, make sure that the data kept its flexibility and that there was room for new insight. Additionally, the analysis process enabled the research to relate the data to the propositions that were proposed from the literature. The created higher-order themes, the relating propositions, and sub-themes can be found in Appendix B.
4. Results
To provide a clear context for the business value construct, an overview of the different software products and the related target markets of the products that the product owners were working on can be found in Appendix C. In the following sections, each of the five propositions is discussed.
4.1. Stakeholder satisfaction as driver for value creation
In line with the agile philosophy which significantly focusses on value creation (Fowler & Highsmith, 2001), almost all respondents highlighted the importance of value as a central concept in their development project. Product owner H explains: “In the development of this
product, you of course take the things that have the most value first.” This value-based thinking
is not only envisioned by the importance of high-value functionalities but also by highlighting that items that were considered not valuable, simply disappeared from their backlog. Product owner D explains this by stating that when he sees that some items on his backlog are constantly shifted downwards, he can conclude that it has no value for the software product he is developing and will therefore not be built.
This value-based thinking is directly related to the interplay between the product owners and their stakeholders. Almost all product owners mentioned the importance of their stakeholders in the production of valuable software. Product owner A portrays the importance of stakeholder satisfaction by stating: “You could say stakeholder satisfaction or management
basically covers everything.” He then further explains: “You could equalise value with whatever stakeholders want and it is your job to make a good translation and make sure that the interaction between stakeholders is facilitated and there is understanding.” Product owner
B further exemplifies the focus on his stakeholders in the development project by stating: “If
you produce for the stakeholders of a product and those stakeholders are satisfied, then you automatically expect that the product increases in value.”
Dealing with the needs of different stakeholders in the development project most of the time meant for the product owners integrating rather opposing value considerations. This because these different stakeholders most of the time have different interests in the software product. This means that these different stakeholders demand different things and requirements, on which the development team is expected to work on. Product owner A explains how he deals with the opposing demands of his stakeholders by stating: “Well look, you have done a great
job if you have listened to your customers, to your internal organisation, to your developers, and with the internal organisation I mean the management team, the sales department, and all other parties.” He continues: “As soon as you have listened to all of them and you have put that all together and then communicated that back in the right way, then you have done a great job.”
In this field in which different stakeholders demand different requirements for the product, it becomes apparent that product owners have to deal with a large group of different stakeholders. In line with previous research (Hannay et al., 2017), product owners considered that value creation is not merely grounded on the value perception of the customer as the only stakeholder to consider. The large diversity of stakeholders that were considered important in the development project was best captured by product owner K who explains which stakeholders he had to deal with during development by stating: “Well, product managers,
other product owners, architects from the activation team, more the lead developers from the module development teams. Account managers, also external parties so the supplier and the customer, consultants, project managers.”
The amount and diversity of stakeholders that the product owners considered important did however strongly vary. This variance was largely explained by the nature of the software product and the organisation in which the product owners were active. Product owner D expressed that his field of stakeholders was very limited as the platform he was developing was extremely “externally oriented”. As such, his main concerns were related to the customer of
the software product. He explained that this was facilitated by the fact that he and his team were given full autonomy to develop the product and therefore he was not bound by “internal
stakeholders”.
In their role as product owner, all product owners did however consider the importance of satisfying their most important stakeholders as a way to generate valuable software products. In this process, these stakeholders had a pivotal role in specifying and agreeing on the requirements that the development teams were planning to work on. Hence, it can be concluded that the data provides support for P1 - Stakeholder satisfaction is the core driver for decisions
regarding value creation in agile software development.
4.2. Constructing business value
Based on the software product each product owner was working on, they were asked how they would describe the value of the product or in specific cases, the value of the functionality. Previous research (Alahyari et al., 2017; Khurum et al., 2013) has indicated that business value of software can be defined based on different value perspectives and relating value aspects. The following section will describe the main value perspectives and the relating value aspects through which the product owners defined the business value of their software product.
The first value perspective considered important is the internal business perspective on the value of the software. In which a lot of attention was paid to the maintainability and
reliability of the software product. These elements were framed in terms of technical
improvements in the underlying infrastructure of the software product by which the long-term performance of the software product could be guaranteed. These value aspects were framed in relation to either a cost value perspective or a revenue value perspective.
First of all, the product owners explained that by making improvements in the maintainability of the software product, configuration and maintenance costs could strongly decrease. Which ultimately results in an overall operational cost reduction for the internal
organisation. Product owner F elaborates on this value by stating: “Especially with maintenance
so to say. If you have to configure things that always cost a lot of time, then you can convert the number of hours needed for this maintenance and the hourly rate and then you can express the costs per period. If you can improve that by making it more efficient, then you clearly add value.”
Secondly, the product owners explained that making improvements in the software product that increased the reliability of the overall product was essential to guarantee the long-term performance of the software for the customer. This was most clearly expressed by product owner H who elaborated on the value of the reliability improvements he was working on by stating: “The developers are leading in this, they make sure that we keep innovating on the
technical basic structure, something the customer does not ask for. But if we want to keep relevant in the long run, we have to keep investing in this.” From this line of reasoning, the
product owners explained that when the reliability of the software product decreased, customers are no longer willing to pay for it and therefore revenues for the internal organisation would decline. So, keeping the software product reliable was considered essential to guarantee the long-term revenue stream from an internal business perspective.
Interestingly several product owners expressed the tension between these technical improvements that increase the reliability and maintainability of the software and the customer's perspective on these value aspects. In which they explained that customers do not directly see the value of keeping the software product reliable and maintainable but they simply expect this as something self-evident. They furthermore highlighted that this internal business perspective sometimes clashes with the value perspective of the customer. This clash was described as a direct result of the fact that some of their customers are constantly requesting new functionalities which they consider valuable. Incorporating too many functionalities into the software product can eventually lead to the so-called "functionality creep" which refers to a
situation in which the extra functionalities go beyond the original basic foundation of the product and therefore the product loses scope and performance in the long run (Murphy, 2001). Product owner J explained this tension: “If we constantly commit to demands from the
customer, we won’t be able to work on real technical improvements, which eventually results in a collapse of the foundation of the software product.”
In the SVM framework provided by Khurum et al. (2013) reliability and maintainability of the software can relate to both the internal business perspective as well as to the customer perspective on the value of software. Interestingly, contrary to the SVM framework the product owners did not relate the maintainability and reliability value of the software product directly to the customer perspective as product owners indicated that their customers did not explicitly consider the maintainability and reliability of the software product of value for them.
One could argue that the outcome of the described maintainability and reliability value of the software could be framed from a financial value perspective since decreasing the operational cost and increasing revenues for the organisation is ultimately a financial value improvement. This would then however be more of a secondary perspective since the product owners did not specify their value consideration directly from this financial perspective. This since the financial perspective from the SVM framework (Khurum et al., 2013) solely captures the value of the software in terms of shareholder value. This direct link between the software product or development process and shareholder value was not found in this research.
In line with the agile philosophy, the main value perspective through which the product owners defined the value of their software was the value perception of their customers. The importance of this customer perspective and the underlying value aspects considered in this perspective will be elaborated on in the next section on the customer value perspective.
Since the product owners described the business value of their software both in terms of an internal business perspective and a customer perspective with different underlying value aspects
it can be concluded that the data provides support for P2 - Business value in ASD is defined
based on a variety of value perspectives.
4.3. Customer value perspective
In line with the premise of the agile software development philosophy, most product owners described that the core value of their software could be described in terms of customer-related value aspects. From a customer perspective, it is first of all important to mention that there is most often a clear distinction between the customer in terms of the user of the software product and the customer in the role of buyer of the software product.
In terms of the user perspective, the product owners predominantly described how their software functionalities contributed to efficiency improvements in the work processes of their users. This was mainly established by increasing the usability of the software product. Since they considered that by increasing the usability of the software product, work processes were more aligned with each other, and therefore fewer activities were demanded from the users of the software. Besides efficiency improvements that were a result of increasing the usability of the software, a lot of attention was paid to the quality of work processes improvements facilitated by the software product. Product owner I described how his form generator functionality improved the quality of information that his user in the form of a municipality consultant, received from its citizens. By this he explained, consultants who used the software were able to gain more insight into the personal situation of the citizen and could, therefore, give better advice and as a result improved the quality of their work.
From the perspective of the customer in the role of buyer of the software product, efficiency value was also indicated as highly important. In this case, the product owners explained that most of their new functionalities were focussed on efficiency improvements that result in a reduction of costs for their customers. Meaning that when a new functionality could
automate certain work activities, the customer perceived efficiency value in terms of a decrease in FTE’s needed to accomplish the job.
Even though the perspective of the customer was most considered by the product owners when they described the value of their software product, other than the efficiency and quality of work process value of the software, they were not specific in what actually constitutes the value of their software from this perspective. Most product owners therefore described the value of their software in more general terms as they were unable to specify the exact value for their customer. Product owner K best captures this more general idea on the business value of his software product by stating: ‘’Simply put, value is the thing customers are willing to pay for.” This is interesting since the product owners did mention that they spent a lot of time discussing what functionalities were considered important by their customers. Indicating the actual value of the functionalities that were considered most important by their customers was however most of the time not clearly defined.
In one specific case, the product owner could directly relate the value of his software product to revenue gains for his customer. In this case, the software product included a platform that gave insight into market developments. In the platform, the customers of the software product were able to use the information provided on the platform to make better deals with their buyers so that their profits could increase. This specific link between the main functionality of the software product and the value that this functionality generated was however exceptional.
Contrary to the earlier research of Alahyari et al. (2017), little emphasis was given to the so-called project-related (time, quality, and cost) value aspects from a customer perspective. Only product owner C who framed the management team of his organisation as his customer, elaborated on the value of quickly finishing the software project by stating: "In practice, you
delivery of the product." In which he further explained that this project value perception was
decreasing the actual business value of the software product.
One could argue that these project-related value aspects were not explicitly mentioned because the product owners perfectly followed the agile way of working in which they predominantly focus on creating valuable software products. This would however be too simplistic since except for product owner C all product owners were developing a software product with no specific defined end date. The ongoing character of the development project could be the main reason for the fact that the product owners did not consider these project-related value aspects from a customer perspective. However, since so little attention was given to these short-term project value aspects it can be concluded that the data does not provide support for proposition 3: From a customer value perspective both short term project and
long-term product value considerations are made to assess business value
4.4. Assessing business value
Assessing business value in the course of the software development project was considered very complex by most product owners. Product owner A explains this by stating: “Yes, it is
rather complex. It is a certain feeling.” He continues by stating: “But it is very subjective. Also, the assessment is very subjective.” Product owner I further substantiate this: “Also the prioritisation of items, most of the things are still based on a certain feeling, let say on my gut feeling.” In this, it is important to mention that these assessments of business value, based on
their intuitive feeling, were mainly related to smaller adjustments and requests in the software product. When looking at the more complex and large functionalities of their software products their intuitive feeling was substantiated by two important mechanisms.
As indicated in the agile literature, customer interaction and communication are considered key practices in agile requirements engineering to make sure that only those requirements with the highest amount of business value are included in the software product
(Inayat et al., 2015). To assess the importance of a particular functionality and therefore the amount of business value, product owners heavily relied on the interaction with their customers. Product owner F explains his dependence on the involvement of the customer by expressing:
“Without the input of that customer, I am unable to improve my product. He continues by
elaborating on the process of creating new functionalities: “I show particular things, then I
communicate this to my customer and the customer tells me this could be interesting, we will look at it. Then we receive feedback from that customer that it is correct, and we start working on it.” Product owner D even explained that he regularly visits his customers on their plantation
to get a better feeling with the product and is therefore better able to understand which functionalities are of value for the software product he builds.
Another important mechanism to assess business value that was indicated by seven of the product owners is the software product road map. This road map is essentially a long-term product vision the software developing organisation has created. The product road map describes the main building blocks of the software product that should generate business value based on the organisation’s perception. This roadmap supports the product owners in their assessment of requirements and the expected business value they will be able to deliver. If a particular requirement is related to one of the main components or functionalities in the road map of the software product, they know that this particular requirement holds business value and is therefore considered more important. Product owner C explained this when he was asked how he got the information to substantiate his intuitive feeling of the business value construct:
“Well yes I got that from the road map, which is a tooling for product owners. The end product is going to be realized with my road map.”
This road map was considered important in the assessment of business value. This contrary to the agile philosophy that states that software development projects should not be guided by predefined requirements upfront (Fowler & Highsmith, 2001). Product owner G
