Medical device software development: extensions for successfully
implementing Scrum in safety critical environment.
Master thesis research, MSc Supply Chain Management University of Groningen, Faculty of Economics and Business
Sebastiano Dall’Anese S3160424
Abstract
TABLE OF CONTENT 1. Introduction ... 1 2. Theoretical background ... 3 2.1 MDD (2007/47/EC) ... 3 2.2 Agile: ... 4 2.2.1 Scrum ... 5
2.2.2 Scrum Artefacts and roles: ... 5
2.2.3 Scrum Events: ... 6
2.4 Barriers: ... 7
2.4.1 Safe Scrum ... 8
2.4.2 Quality assurance ... 8
2.4.3 Safety oriented Scrum ... 9
2.4.4 Scrum at Qumas ... 10 2.4.5 Lack of planning ... 11 2.5 Scrum’s extensions ... 12 3. Research Design ... 13 3.1 Case Selection ... 13 3.2 Data collection ... 14 3.3 Data analysis ... 14 4. Results section ... 15
4.1 Scrum software development at Diagnoptics ... 15
4.2 Scrum software development at Brisp. ... 17
5 Discussion ... 19
5.1 Product Backlog and priority Items ... 19
5.2 Planning phase ... 20
5.3 Traceability and continuous testing ... 21
5.4 Risk management ... 22
6. Conclusion ... 22
6.1 Further research ... 23
6.2 Limitation ... 25
LIST OF TABLES Table 1. Standards harmonized into the amendment mdd (2007/47/ec). ... 4
Table 2. Scrum roles ... 6
Table 3. Barriers when adopting agile in safety critical environment ... 7
Table 4. Possible Scrum’s extensions ... 12
LIST OF FIGURES
Figure 1. Scrum framework ... 7
Figure 2. Safe scrum ... 8
Figure 3. Safety oriented scrum ... 10
Figure 4. Quamas r-scrum ... 10
Figure 5. Planning phase at diagnoptics ... 16
Figure 6. Scrum cycle at diagnoptics ... 17
Figure 7. Planning phase at brisp ... 18
Figure 8. Scrum cycle at brisp ... 19
Figure 9. Risk analysis after code testing ... 21
Figure 10. Software security best practices applied to various software artifacts ... 24
1. Introduction
Commonly, medical devices used to consist of merely physical components but at the present time they are made of a combination of hardware and software. Thus the development process is highly interlinked and requires high technical knowledge, which is expensive and uncertain undertaking (Cawley et al. 2010). Before 2007, software was seen as an element of the medical device and its compliance to the regulations was verified by testing the physical device (McHugh et al. 2015). In 2007 the European Union liberated its latest amendment to the Medical Device Directive (MDD) 1993/42/EEC known as 2007/47/EC, which went into effect in 2010 (Censi et al. 2015). The new amendment lead into great changes; one of the most important, which has direct effect on software developers, is that in the current situation stand-alone program can be categorized as an active medical device (McHugh et al. 2012).
In 2001 the Agile Alliance was established and it published the Agile Manifesto (Fowler et al. 2001). The Agile manifesto was originated by the practitioners’ need for developing new software development methods because the traditional ones, like the well-known waterfall, were seen as too rigid and inefficient to accommodate the growing complexity of the software development (Fitzgerald et al. 2013). Some authors have studied how Scrum, the most used method by Agile firms, is implemented in different sectors, like the construction industry (Streule et al. 2016) and in the teaching field (Opt et al. 2015). Most of the literature focuses on the use of Scrum in the IT sector (Lei et al. 2017; Nidagundi et al. 2017) and little research has been addressed to the adoption of Scrum framework in regulated safety critical domains, such as the one of medical device (Hugh et al. 2015; Cawley et al. 2010).
For medical device software developers it isn’t sufficient to improve the development process to increase the value for the customer, but it is also important to assure that the safety requirements are recognized and matched (Łukasiewicz et al. 2016, Regan et al. 2013). “For the medical device industry it is essential to have clear linkages and traceability from requirements - including risks - through the different stages of the software development and maintenance lifecycles.” (McCaffery et al. 2012, page 1). In general, Scrum emphasizes the importance of producing minimal documentation and doesn’t provide detailed instructions on how to manage a proper quality control system, which can guarantee traceability (Mwadulo, 2016). Consequently, many authors have claimed that pure agile methods cannot be applied in regulated environment (Turk et al. 2014, Rasmussen et al. 2009, Rottier et al. 2008).
RQ: What Scrum’s extensions do Medical Device Software developers need to ensure regulatory compliance with the European legislation?
This paper is exploratory; multiple case studies are used to collect rich data in order to answer the research question and to identify the extensions that are needed by Medical Device software producers. This research will also be helpful to practitioners because it will provide the necessary tools to overcome these impediments.
2. Theoretical background
In this section the amendment MDD 2007/47/EC and its consequences for the software developers are presented. It follows a description of Agile methodology with particular focus on Scrum framework. Through a literature research the issues related to Scrum implementation in regulated environment are identified as well as the possible solutions to overcome them.
2.1 MDD (2007/47/EC)
Medical device software must satisfy customer/user requirements but, unlike unregulated products, they must also meet requirements dictated by the regulatory entities of the geographic area in which they will be commercialized (McHugh et al. 2013; Rust et al. 2016). Regulatory requirements in contrast to user requirements can be predicted as regional laws determinate them. Thus, this section presents the European law and the standards, which medical device software must comply to be marketed in the EU.
All medical devices that are used and commercialized in EU must conform to the latest MDD amendment known as 2007/47/EC, which entered in force on March 21st, 2010 (Klumper et al. 2010). The definition of medical device can be found in Article 1, section 2 of MDD (2007/47/EC): “any instrument, apparatus, appliance, software, material or other article, whether used alone or in combination, including the software intended by its manufacturer to be used specifically for diagnostic and/or therapeutic purposes and necessary for its proper application” (McHugh et al. 2011, page 99)
The Medical Devices Directive, MDD (1993/42/EEC), stipulates that software can be a component of the medical device but the regulation doesn’t classify it as stand-alone software, which is now defined as an active medical device. This is one of the most significant changes that the amendment MDD (2007/47/EC) has brought to the attention of software developers because before 2007, software was seen as a “subsystem embedded in a medical device” (McCaffery et al. 2015, page 134) and its compliance to regulations was verified by testing the physical device (McHugh et al. 2015). In the current state, medical device software has to pass through a validation test to demonstrate that is safe and can be commercialized. However, even if with the delivery of the new amendment MDD (2007/47/EC) extends the definition of a medical device, the mechanisms necessary for medical device developers to achieve conformity compliance have not been modified with respect to the MDD (1993/42/EEC).
Standard: IEC 62304: 2006* ISO 13485:2012* ISO 14971:2012* Application: Software development
standard for medical device software Medical Devices Quality Management Systems Application of Risk Management to Medical Devices Purpose: -This standard proposes the
“software lifecycle model specific to the development of medical device software” **
-IEC 62304 classifies software components based on the potential risk that a patient, clinical or a 3rd party can encounter during the usage.
-Three classes are defined: • Class A – No injury or damage to health possible; • Class B – Non-serious injury is possible;
• Class C – Death or serious injury is possible -The standard requires that a quality management system should be implemented and maintained throughout the medical device software lifecycle. -The standard gives explicit guidance to the methodologies and procedures for identifying hazards in medical devices, accessories and software. Compliance rules:
-The standard is not
mandatory, but its cohesion facilitates the regulatory approval in EU and US markets.
-However, compliance to IEC 62304 standard is not sufficient to achieve validation and final release of the medical device software.
-Compliance to the standard is not mandatory.
-Compliance to the standard is not mandatory.
Table 1. Standards harmonized into the amendment MDD (2007/47/EC). Source: McHugh et al. 2011, McHugh et al. 2015, and Santos et al. 2014. *Year of introduction.
**McHugh et al 2013, page 3. 2.2 Agile:
In the same year, they submitted the Agile Manifesto [2], which is based upon four values:
- Individuals and interactions over processes and tools - Working software over comprehensive documentation - Customer collaboration over contract negotiation - Responding to change over following a plan
Moreover, its authors presented twelve principles supporting the “Agile Manifesto”. Based on these values and principles many agile methodologies have arisen such as: Scrum, extreme Programming, Crystal, Dynamic Systems Development Method, Agile Modeling, Lean software Development (Dyba et al. 2008). These methods became popular in the early 2000s and are now endorsed worldwide. Stravru (2014) conducted a survey to verify the usage of agile practices and it resulted that about half of the software developers use agile methods. The most used method by agile firms is Scrum: “53% of Agile practitioners use Scrum in their projects” (Rola et al. 2014, page 49). For this reason the paper focuses on Scrum.
2.2.1 Scrum
Ken Schwaber and Jeff Sutherland (2013), creators of scrum, define it as a framework to support and manage complex software development. They advise that scrum “is not a process or a technique for building products; rather, it is a framework within which you can employ various processes and techniques” (page 3). The authors define the basic rules for applying this framework. Of course developers are allowed to tailor or scale up/down this methodology in order to adjust it to the environment in which they operate. From section 2.3.2 to 2.3.4, the Scrum framework and its artefact are shortly presented based on the paper: “The Scrum Guide” (Ken Schwaber et al. 2013)
2.2.2 Scrum Artefacts and roles:
• The product Backlog is an informal document that contains all the product’s requirements. The requirements are usually called features or user stories. • The Sprint backlog is a set of Product Backlog stories that have to be
accomplished during a Sprint; it also represents a plan for the Development Team, which forecasts what software functionalities are developed during the Sprint. The Sprint Backlog must be clear and visible to the Development Team.
Table 2 presents the three roles contained in the scrum framework and their main characteristics.
Table 2. Scrum roles 2.2.3 Scrum Events:
The Sprint represents the time lapse in which the scrum team works to create a defined number of software functionalities. Its duration should be consistent during the project and it is usually between 2 to 4 weeks. This helps to control risk that may rise during the product development. The Sprint is divided in different phases: Sprint Planning, Daily Scrum, Sprint review and Sprint Retrospective.
• During the Sprint Planning, the team organizes the work that has to be accomplished during the Sprint. To help the team in this phase, the following questions should be addressed:
o “What can be delivered in the Increment resulting from the upcoming Sprint?”
o “How will the work needed to deliver the Increment be achieved?” (Ken Schwaber and Jeff Sutherland, page 8, 2013).
• Daily Scrum is a daily meeting of 15 minutes in which the Development Team synchronizes and plans the team’s work for the following 24 hours. This phase is also used to monitor the development progress.
• Sprint Review is performed at the end of the sprint; the Scrum team and different stakeholders evaluate the quality of the work that has been completed.
The Scrum team :
is self-organized and cross functional; these characteristics are important
as the scrum’s aim is to improve 8lexibility, productivity and creativity
The product owner :
-is responsible for creating the Product Backlog and
its modification. -ensures that the Product backlog is clear and can be
understood by all stakeholders and not only
by the team. -is also responsible for guiding and training the team to achieve its goal.
The Scrum master : -has to make sure that all
the team members understand the scrum methodologies and rules. -trains and helps the team.
-is responsible for the communication between
the Scrum team and the external stakeholders
The development team : -consists of professionals
in charge to deliver incremental product releases at the end of each
Sprint.
-within the team there are no job titles other then developers; the creation of
sub-teams is also not allowed. - size constrain: 3 to 9
• During the last phase, the Sprint Retrospective, the Scrum Team self-assesses its own work and analyses what went wrong during the Sprint. This phase serves the Scrum team to investigate potential improvements and eventually to create an improvement plan.
Figure 1. Scrum framework
Source: Takahira et al. 2012, page 2667. 2.4 Barriers:
McHugh et al. (2012) conduct a survey among 160 medical device manufactures, to identify the issues that medical device software developers face when implementing agile. The companies interviewed range from small organizations to large multinationals, which produce products compassing all classes of risk, based on the definition given by the MDD 93/42/EEC. Half of the firms develop software following the V-Model, 25% the Waterfall model and 25% used agile methods. Within the agile family, Scrum is the most used methodology by the medical device developers (Cardozo et al. 2010).
Table 3 gives an overview of the barriers that medical device developers face when develop safety critical software following agile technique.
Regulatory Control
Insufficient coverage of Risk Management activities Requirement Management
Traceability issues
Lack of up-fronting planning Lack of documentation
Table 3. Barriers when adopting Agile in Safety critical environment Source: McHugh et al. 2012 page 6.
2.4.1 Safe Scrum
Hanssen et al. (2016) propose a modified version of Scrum called Safe Scrum to help companies overcome the above-mentioned barriers. The authors develop some extensions (Appendix 1) that must be added to the classic Scrum methodology, and they test their new model against the IEC 61508 standard’s requirements. Moreover, Stålhane et al. (2012) claim that Safe Scrum could be used to achieve compliance with any standards.
However, these new extensions are created to satisfy the requirement of a specific standard, the IEC 61508, which is not included in the European amendment identified in section 2. Its effectiveness should be proved against other standards and regulations. Figure 2 represents Safe Scrum and its extensions.
Figure 2. Safe Scrum
Source: Hassen et al., page 97, 2016.
In addition, Hanssen et al. (2015) observe in their work the introduction of Safe Scrum in an industrial environment. The company chosen is Autronica Fire & Security AS, which operates in safety critical environment with the focus on delivering high safety performance products. They conclude that Safe Scrum supports Scrum Teams in producing the documentation required by the regulatory bodies and stated that a tool as automated document creation makes this process more Agile as it decreases manual work.
2.4.2 Quality assurance
projects are “run in large organizations with high expectations for quality and efficiency” (Gulstuen et al. 2015, page 239). Quality is essential when developing Medical devices software and for this reason the finding of this study can give some interesting insight about the role of QA team, which support the Scrum team during the software development. Gulstuen et al. (2015) report that Scrum Teams can achieve higher quality if the software testing begins at the early stages of the software. This particular observation supports what Safe Scrum introduced as extension. Both Safe Scrum (Hanssen et al. 2016) and the observations made by Gulstuen et al. (2015) suggested that in order to achieve high quality software, the agile practice Test Driven Development must be used in each Sprint, from the start of the project. Morsicano and Shoemaker also advise that it is important for agile projects to introduce continuous testing during medical device software development in order to meet the FDA requirements.
Moreover, a QA resource/tester has to support the Scrum Team in order to assure quality of design documentation Gulstuen et al. (2015) identify the tasks that the QA resource/tester needs to accomplish during the software development. However, in contrast to Safe Scrum it has been observed in Gulstuen et al. (2016) report that companies use an external dedicated QA resource, which works in close relation with the Scrum Team and Product Owner. It seems to be a good idea to have a supportive QA team, but despite that it is not clear which solution could work better in the case of medical device software developers. This point will be analyzed in more depth in the case studies.
2.4.3 Safety oriented Scrum
Guo and Hirschmann (2012) propose, as does Safe Scrum, to have non-safety-critical functions and safety-critical functions (Figure 3). The difference between the two models consists that in Safety oriented Scrum there is only one backlog and in the way this backlog is managed. They also claim that a Safety Manager is in charge of ensuring safety throughout the software development. They state “one crucial factor for use of such an integrated process model is the correct definition of the sprints according to the separation of the non-safety-critical functions from the safety-critical functions and the decomposition of the safety functions regarding their criticalities and dependencies” (Guo et al. 2012, page 648).
Figure 3. Safety oriented Scrum
Source: Guo and Hirschmann, page 648, 2012. 2.4.4 Scrum at Qumas
Fitzgerald et al. (2013) observe how the company Qumas adapts Scrum to work in a safety critical environment. In their work new elements, roles and artifact (Figure 4) have been added to the base Scrum.
Figure 4. Quamas R-Scrum
Source: Fitzgerald, Stol, O'Sullivan & O'Brien, page 868, 2013.
2.4.5 Lack of planning
2.5 Scrum’s extensions
For the Scrum approach to be adopted in safety critical environment, some modifications are needed. The Scrum’s extensions derived from the literature review are presented in Table 4.
Table 4. Possible Scrum’s extensions
*Source: Douglas et al. (2012), Fitzgerald et al. (2013), Ge et al. (2010), Gulstuen et al. (2015), Guo et al. (2012), Hanssen et al. (2016), Rafi et al. (2015) and
Stålhane et al. (2012).
The extensions presented in the theoretical background section are applicable with respect to the standard IEC 62304:2006 “Software development standard for medical device software”, which states that any software lifecycle model can be used for development of medical device software, so scrum can be modified and negative
Basic Scrum Scrum’s extensions *
One Backlog -Two Backlogs, ordinary
requirements and one for safety requirements. Ordinary requirements are linked to their respective safety once.
-One Backlog and distinction
between non-safety-critical functions and safety-critical functions
Three Roles: Scrum Master, Product Owner, and Development Team. Creation of sub-team is not allowed.
-Safety Manager
-Internal Quality Assurance Team -External Quality Assurance Team Priority Item is determined by
customers
If two items have the same priority, the first to be developed is the one with the highest risk
Minimal planning Dedicated planning phase: software
design generation so Risk and Hazard analysis can be accomplished
Software functionality is tested at the end of each Sprint
Continuous Testing during the Sprint,
TDD
Traceability should be maintained during the development process. Automatic traceability.
conflict will not be generated. Moreover, it will investigate if medical device software developers use these extensions to comply with ISO 13485:2012 and ISO 14971:2012.
RQ: What Scrum’s extensions do Medical Device Software developers need to ensure regulatory compliance with the European legislation?
The case studies will help us to answer these two sub questions:
1. What kind of Scrum’s extensions do Medical Device Software developers use?
2. Is the support of an external Quality Assurance Team needed or is the Scrum Team capable of providing this function internally?
3. Research Design
The objective of this master thesis was to investigate how medical device software developers adapt the Scrum framework to comply with the Medical Device Directive (MDD) 1993/42/EEC and its related standards. Robson et al. (2002) classified the different research methodologies based on their purposes and suggested that case study should be used for exploratory objective. Therefore, for the purpose of the research a multiple case studies, which aim is to investigate a contemporary phenomenon in its natural context were most appropriate (Runeson et al. 2009). This method helped to confirm the Scrum’s extensions detected in the literature review and to explore if new extensions were created and used in the medical device context.
3.1 Case Selection
A first contact with the possible participant was sent by email; where the aim and goal of the research were explained as well as the effort and the relevance of their participation to the study.
Two cases were selected for convenience accessibility and proximity to the
researcher. The cases were two companies based in Groningen, NL and they used Scrum successfully for developing medical devices software. However the two companies were different in structure and area of competence.
reason, regulatory bodies impose high safety and security standards. Security is a primary concern during the development of this software, because in case of failure the system could be compromised and it can interfere with other software, for example medical devices.
3.2 Data collection
Corbin et al. (2008) recommended to use semi-structured research questions, when conducting an exploratory research, so unexpected topics might bring up by the interviewees. Yin et al. 2003 also suggested to use, when possible, open questions to reduce the bias of the respondents. Thus, an interview protocol Appendix 2 based on the suggestion of the above-mentioned authors and Emans et al. 2004 was created. The protocol served as a checklist to the interviewer during the semi-structured interviews. It was also designed to ensure the reliability and validity of the cases research data. At Diagnoptics two interviews were performed: one with the quality assurance manager and the other with a software engineer, who had experience as Scrum Master and Developer.
At Brisp, the company’s owner was interviewed. He was the person who had introduced Scrum as software development methodology and he had five years experience as Product owner, Scrum Master and Developer.
Moreover, the interviews were recorded and the transcripts and the figures of the companies’ Scrum processes were sent back to the participants to be checked and validated as suggested by Yin et al. (2003). To maximize recollection and to easily fill the intervals in the data, the interviews transcripts were generated within two days from the dialogues with the company representatives, as suggested by Karlström et al. (2006). Furthermore, during the data analysis additional questions were sent to the participants to clarify concepts and finding that were superficially developed during the interviews.
3.3 Data analysis
Seaman et al. (1999) suggested that qualitative data analysis could lead to both hypothesis generating techniques and hypothesis confirmation techniques. The intent of this study is to generate hypotheses from the data and to compare it with the findings of the literature section. Runeson et al. (2009:151) suggested, “the researcher should try to be unbiased and open for whatever hypotheses are to be found in the data”.
The technique that was used to generate hypotheses was a constant comparison between the findings and the theories that have been summarized in the theoretical section (Seaman 1999).
derived from literature and the one suggested by the practitioners was performed. A table was created describing the differences between the Base Scrum (Sutherland et al. 2013) and the new extensions, in order to give a clear overview about the Scrum’s extensions and their purposes for developing medical device software.
4. Results
In this section the software development process used by the two companies is described and 4 figures are presented. The figures represent the detailed software development of the 2 cases and they were confirmed by the representatives of the companies
4.1 Scrum software development at Diagnoptics
Diagnoptics is a company based in Groningen, Netherland, specialized in production and development of class A medical devices. The company develops mostly embedded software that is built to match its unique hardware. For this reason, at the beginning of the project (Figure 1) a meeting is held between different departments within the company.
The Scrum project starts with a planning phase where: Product Owner, Mechanical/Electrical engineers, Quality Manager, Scrum Master, Developers and Marketing department identify the characteristics of the Medical Device project and the work and tasks are divided.
The Product Owner is the head of the R&D department and most of the time one Software engineer retains the roles of Scrum Master and Developer. In the company there are 3 software engineers and they all work on different projects.
The Product backlog is created from the collaboration between the Scrum Master and the Product Owner.
Firstly, the Scrum Master and the Quality Manager perform a Risk Analysis based on the functional and safety requirements identified in the Product Backlog. Risk analysis is the first step in the software development, since risk control measures need to be part of the software design. After the risk analysis the raw design of the software can be created so Test Planning can be determined.
The Quality Manager creates in parallel a quality management system in compliance to the ISO 13485. This activity is constantly managed and maintained during the project by the Quality Manager and it doesn’t relate to a specific phase of the software development.
Once the Product Backlog, Risk Analysis, Software Design, Test Planning and Quality Management System are created, the Scrum Team can plan the work to be done in the Sprint. Some times during the Sprint planning, the Scrum team is helped by other engineers of the company who can give input for effort estimation and priorities.
project can have access to it. The Sprint Backlog is managed and created in the cloud. The company uses a special project management tool called Redmine with which the Scrum Master manages the Scrum Board and monitors the project progress.
Figure 5. Planning phase at Diagnoptics
After these multi-planning phases, the Developer starts to develop and code the software (Figure 2). Every backlog item has to be verified with a test before it can be marked as “Done”. During each Sprint the items are subjects to testing. At the end of the Sprint, Product Owner, Scrum Master, and Quality Manager meet and perform Sprint retrospective and review. During this meeting, they also discuss if any requirements have been changed in the previous Sprint. If any late change in the requirement is made, then an additional Risk analysis is performed.
Traceability is maintained throughout the project. During the software-designing phase, it is determined which software module will fulfill which requirements. During the design phase, a test plan is also designed in which requirements and modules are linked to test cases. At the end of the project, the test plan is conducted and it will trace whether each requirement is fulfilled, by which test case and in which software module. Traceability is constantly maintained.
Figure 6. Scrum cycle at Diagnoptics 4.2 Scrum software development at Brisp.
At Brisp the software development starts with a meeting where (Figure 3) Customers, External Quality Assessor, Product Owner, Scrum Master, developers, Sale Department and Accounting Manager participate. In this meeting, the product’s characteristics are pinpointed. After this preliminary consultation, the Product Owner creates the product backlog, which contains safety and functional requirement.
The Scrum Team varies from 2 to 5 people depending on the size and complexity of the project. The Product Owner is also the Scrum Master and the company has decided that one of the Scrum Team Developers must fill these roles based on his/her experience.
Once the Product Backlog is created, the team creates a project plan where the project’s milestones are fixed.
Firstly, the Product owner/Scrum master creates the design of the software and then the same person performs the risk analysis. Brisp in contrast to Diagnoptics doesn’t include a quality management system in its project. However, it performs its own quality checks during the software developments. Moreover, the Scrum Master and the Quality Manager create a test-planning schedule that is incorporated in the milestone of the project plan.
the performance of the project and its related issues. Moreover, Customers and QA are allowed to communicate directly to the Scrum team by creating a Status alert in Asana’s dashboard. This helps the Scrum team to be more reactive to customers’ request for changes and it also serve as alarm to the QA team, which receives a message that some changes are required from the customer. The use of Asana and its free access to all the stakeholders improve the transparency of the project.
Figure 7. Planning phase at Brisp
At this point the Scrum Team starts developing and coding the software (Figure 4). Usually the code is created with the use of an open source platform called Github. With this cloud tool the company can code and traceability can be obtained thanks to special plug-ins that are present in the Github developer community. The company has also developed in-house frameworks that serve the developers to trace and record all the changes that are made during a Sprint. Thanks to the use of Github and in-house frameworks traceability is maintained in a continuous way throughout the project.
To ensure that the software being developed is safe and secure the code lines created are tested during each Sprint. The company uses its own automatic testing software as well as the support of a third party firm, which is specialized in safety testing.
The Scrum Team also participates to the Daily Stand-up, where the issues and the work to be done during the next 24 hours are discussed.
Figure 8. Scrum cycle at Brisp
5. Discussion
In this section the result derived from the two cases is compared with the theoretical background section. The Scrum’s extensions used by the two companies are identified and compared to understand if medical devices software developers need special ones.
5.1 Product Backlog and priority Items
Both cases presented create one Product Backlog. The ways the two companies create and manage the backlog confirm the model Safety oriented Scrum proposed by Guo and Hirschman et al. (2012) and the one of Hanssen et al. (2016), Safe Scrum. However, Safe Scrum suggests using two distinct Product Backlogs but it also accepts the possibility of using one (Hanssen et al. 2016). One Product Backlog is sufficient, but the most critical aspect for the developers is to include both safety and non-safety requirements, which is done by both cases.
5.2 Planning phase
Safe Scrum proposes to add a release planning once the Product Backlog is made. However, the authors don’t describe what activity should be included. For this reason, the findings of this research can add something to the “Safety oriented Scrum” model. In both cases, a release plan is performed. Nevertheless, there are some differences in the two cases.
At Diagnoptics, the release plan consists of 4 steps; on the other hand, at Brisp the release plan consists of 3 steps. The activity missing at Brisp is the creation of a Quality management system. Diagnoptics creates it because the ISO 13485 requires it. The quality manager is responsible for managing the quality management system and its activities. These activities are incorporated throughout the software development and not only to a particular phase. On the other hand, Brisp makes use of quality test to monitor the quality of its software during the Sprint.
At Diagnoptics, the project plan starts with a Risk analysis in line with the ISO 14971. The cooperation between Scrum Master and Quality Manager is necessary in order to conduct this activity in compliance to the ISO 149171. Scrum Master and Quality Manager have a deep knowledge regarding risk analysis and their strict cooperation enhances the perfect knowledge that is needed to succeed. Following the risk analysis, the Scrum Master proceeds with the creation of the software design.
In contrast, the first activity conducted at Brisp is the creation of the software design. The Product Owner/Scrum Master accomplishes this activity without the help of a quality manager.
When I asked the Brisp’s Owner “when do you create the software design”? The answer received was:
“As first activity of the release planning, we produce the software design. We are able to identify the risks after the software design is completed. Earlier is not possible”.
This statement is in line with the recommendations of Ge et al. (2010) Fitzgerald et al. (2013).
The Scrum Master of Diagnoptics explains that performing risk analysis after the software design might result in risk control measures, which cannot be applied without redesigning the product.
The Scrum Master supports this choice by stating:
“For our case however, we also need to take into account the hardware design. It is rather difficult and costly to do a hardware redesign. Therefore, we do a risk analysis before (or actually during) the design phase to be able to incorporate risk control measures into our design”.
“For example: a risk might arise when certain components of a device are not switched off within a certain amount of time (think of a heating component, laser, light source, etc.). Failing software might produce the risk of such a component not being switched off in time. In such a case we need to apply a hardware (electronic) measure as a risk control measure to reduce the risk to an acceptable level. Therefore, we need to perform a risk analysis during the software design for foreseeable risks to determine which measures need to be embedded into the design”.
5.3 Traceability and continuous testing
In both cases, the traceability requirement is maintained during the development process with the support of automatic tools, as described by Hanssen et al. (2016) & Fitzgerald et al. (2013), which make the developers’ work lighter.
The companies also use continuous testing during the Sprint (Hanssen et al. 2016 & Gulstuen et al. 2015). Figure 9 shows the second point where risk analysis is conducted.
Figure 9. Risk analysis after code testing Source: McGraw, page 82, 2004.
* Diagnoptics ** Brisp
where the increments to be marked as “Done” need to pass their respective tests. In both companies, the definition of “Done” includes the test that the software module, increment, must satisfy.
5.4 Risk management
Risk management is an important part of the software development and Scrum itself doesn’t have an explicit risk management plan. Many authors try to address traditional risk management practices to Scrum framework. Gold and Vassell (2015) research on how “to use the four component of Management of risk process (MoR) to balance software development” (Gold and Vassell, page 49, 2015). Their goal is to improve the risk management in scrum project. They state that risk management in scrum project is conducted in two ways, in a structured or unstructured manner. Based on their definition Brisp uses an unstructured approach where “ risk identification is mostly conducted by the managers and experienced team members of an organization, without any supporting practices; simply put, the Scrum Master and the team members have a discussion first based, on their knowledge of the Scrum process, and second, on their previous experiences (Gold and Vassell, page 51, 2015). At Brisp the Product Owner/Scrum Master is selected within the Scrum team based on his/her previous experience.
The structured approach consists of a “brainstorming meeting” in which various stakeholder participate. This happens at Diagnostics; the Scrum master and the Quality Manager conducts the risk analysis. Moreover, during the Sprint review the Product Owner sometimes is involved in the risk analysis.
The findings of our research confirm the suggestions of Gold and Vassel (2015), which recommended that the Scrum Master should be the person responsible for the risk management activities. In both cases, the Scrum Master is responsible for the risk management activities.
Medical device software developers need extensions for the successful implementation of Scrum framework in their software development projects. Through this research it was possible to identify the extensions that firms need and use, which are presented in table 5.
Table 5. Scrum’s extensions
A comparison between two firms that operate in safety critical environment helped us to conclude that the extensions needed by developers of health care application and medical device are the same, with the exception of the Quality management System. During the planning phase, the Quality manager must create a quality management System, which is required by the standard ISO 149171. This was the only major difference between the two cases, so I believe that any firm that produces software in regulated environment can use the extensions detected in this research. I hope that my research will help further researchers in the development of an extended Scrum framework that can work in any safety critical industry.
6.1 Further research
Scrum extension for medical device software developers
One product Backlog, which contains safety and non-safety requirements
If two items have the same priority, the first items to be developed is the one with the highest risk
Planning Phase must include: Risk analysis
Software design Test planning
Quality management system
Scrum master is responsible for risk management throughout the project
Quality Manage creates and manages the Quality management system and reviews the project documentations
Additional risk analysis must be performed if customer requirements change during the Sprint
McGraw (2004) describes the software development life cycle figure 10 and shows at what stages the risk analysis activities must be performed.
Figure 10. Software security best practices applied to various software artifacts Source: McGraw, page 82, 2004.
*Diagnoptics **Brisp
Risk analysis can be performed before or after the design. However, he doesn’t explain when it is more efficient to perform it or if it has to be performed in different stages based on the type of software developed. He also suggests that risk and security specialists should support designers and developers. “Putting the right people together for an analysis is important: consider the risk team very carefully. Knowledge and experience cannot be over emphasized because risk analysis is not a science, and broad knowledge of vulnerabilities, bugs, flaws, and threats is a critical success factor”(McGraw, G., page 80, 2004).
Further research should be conducted in order to understand when risk analysis should be performed in the cases of standalone medical device software and embedded medical device software. Moreover, it would be interesting researching what level of experience the Scrum team should have to be successful in performing this activity.
the finding of my research.
Akif and Majeed (2012) identify the barriers, which firms face when adopting agile in safety critical environment. Their findings are confirmatory to the study of McHugh et al. (2012). Moreover, Akif and Majeed (2012) propose solutions to resolve these issues. However, one issue unsolved regards the Backlog management practices. “Scrum provides insufficient guidance with respect to the structure of the backlog. The scrum management tools that are available in the market are either too complex (which in turn are more expensive) or too simple and are therefore not useful for the team. Due to this the teams that have recently adapted Scrum framework for their projects avoid large investments at initial level of framework implementation” (Akif and Majeed, page 2, 2012).
This issue needs to be researched further. When Diagnoptics implemented Scrum the Product Backlog was created in Microsoft Excel. The Scrum Master underlined that this way of managing the Product Backlog was inefficient, due to the manual work necessary to manage and maintain it. For this reason, the Scrum Master chose to switch to a dedicated tool called Redmine, which makes the Product Backlog management more agile. In addition, Brisp is now conducting a pivot period where its customers can have direct access to the Product Backlog in order to monitor the project progress and to facilitate the communication between the customer and the Scrum team. Customers are allowed to log-on in Asana, Brisp’s Product Backlog management tool, and create a status whenever they have a request. Surprisingly, Brisp experiences that in this way the company receives less remarks and questions; saving precious time, which can now be used to speed up the software development. For the above-mentioned reasons, I suggest further research regarding Product backlog management tools in order to identify the essential characteristics needed by the different type of organization. These kinds of tools can give a boost in customer involvement, which is one of the central Scrum’s objectives.
6.2 Limitation
Firstly, the findings of this research can be considered context dependent because they are based on two companies based in Groningen, Netherlands. To strengthen the results of this research a quantitative analysis must be performed in order to confirm if the extension identified are used by companies that operate in other European countries.
Secondly, it was not possible to answer the second sub-question.
• Is the support of an external Quality Assurance Team needed or is the Scrum Team capable of providing this function internally?
answer this sub question, I suggest future researchers to select development teams that are composed by a minimum of three people.
Reference:
https://ec.europa.eu/growth/single-market/european-standards/harmonised-standards/medical-devices_en
[2] Manifesto for Agile Software Development. (2001, February). http://agilemanifesto.org/
Akif, R., & Majeed, H. (2012). Issues and challenges in Scrum
implementation. International Journal of Scientific & Engineering Research, 3(8), 1-4.
Bjerke-Gulstuen, K., Larsen, E. W., Stålhane, T., & Dingsøyr, T. (2015, May). High Level Test Driven Development–Shift Left. In International Conference on Agile Software Development (pp. 239-247). Springer International Publishing.
Cardozo, E. S., Neto, J. B. F. A., Barza, A., França, A. C. C., & da Silva, F. Q. (2010, April). SCRUM and Productivity in Software Projects: A Systematic Literature Review. In EASE.
Cawley, O., Wang, X., & Richardson, I. (2010). Lean/agile software development methodologies in regulated environments–state of the art. In Lean Enterprise Software and Systems (pp. 31-36). Springer Berlin Heidelberg.
Censi, F., Mattei, E., Triventi, M., & Calcagnini, G. (2015). Regulatory frameworks for mobile medical applications. Expert review of medical devices, 12(3), 273-278.
Corbin J, Strauss C (2008) Basics of qualitative research, 3rd edn. Sage
Dybå, T., & Dingsøyr, T. (2008). Empirical studies of agile software development: A systematic review. Information and software technology, 50(9), 833-859.
Emans, B. J. M. (2004). Interviewing, theory, techniques and training. Leiden/Groningen: Stenfert Kroese.
Fitzgerald, B., Stol, K. J., O'Sullivan, R., & O'Brien, D. (2013, May). Scaling agile methods to regulated environments: An industry case study. In Software Engineering (ICSE), 2013 35th International Conference on (pp. 863-872). IEEE.
Fowler, M., & Highsmith, J. (2001). The agile manifesto. Software Development, 9(8), 28-35.
Gold, B., & Vassell, C. 2015, Using Risk Management to Balance Agile Methods.
Hanssen, G. K., Haugset, B., Stålhane, T., Myklebust, T., & Kulbrandstad, I. (2016, May). Quality Assurance in Scrum Applied to Safety Critical Software.
In International Conference on Agile Software Development (pp. 92-103). Springer International Publishing.
Hanssen, G. K., Kulbrandstad, I., & Haugset, B. (2015, May). Introducing SafeScrum. In Agile Processes in Software Engineering and Extreme Programming: 16th
International Conference, XP 2015, Helsinki, Finland, May 25-29, 2015, Proceedings (Vol. 212, p. 365). Springer.
Karlström D, Runeson P (2006) Integrating agile software development into stage-gate product development. Empir Softw Eng 11:203–225 doi:10.1007/s10664-006-6402-8
Klumper, M., & Vollebregt, E. (2010). The Regulation of Software for Medical Devices in Europe. Journal of Medical Device Regulation, 7(2), 5-13.
Lei, H., Ganjeizadeh, F., Jayachandran, P. K., & Ozcan, P. (2017). A statistical analysis of the effects of Scrum and Kanban on software development
projects. Robotics and Computer-Integrated Manufacturing, 43, 59-67.
Łukasiewicz, K., & Górski, J. (2016, September). AgileSafe-a method of introducing agile practices into safety-critical software development processes. In Computer Science and Information Systems (FedCSIS), 2016 Federated Conference on (pp. 1549-1552). IEEE.
McCaffery, F., Casey, V., Sivakumar, M. S., Coleman, G., Donnelly, P., & Burton, J. (2012). Medical device software traceability. In Software and Systems
Traceability (pp. 321-339). Springer London.
McCaffrey, F., Lepmets, M., & Clarke, P. (2015). Medical device software as a subsystem of an overall medical device.
McCaffery, F., Lepmets, M., Trektere, K., Özcan-Top, Ö., & Pikkarainen, M. Agile Medical Device Software Development.
McGraw, G. (2004). Software security. IEEE Security & Privacy, 2(2), 80-83.
challenges with plan-driven software development lifecycles. In Software
Engineering in Health Care (SEHC), 2013 5th International Workshop on (pp. 12-19). IEEE.
McHugh, M., McCaffery, F., & Casey, V. (2011, May). Standalone software as an active medical device. In International Conference on Software Process Improvement and Capability Determination (pp. 97-107). Springer Berlin Heidelberg.
McHugh, M., McCaffery, F., & Casey, V. (2012, May). Barriers to adopting agile practices when developing medical device software. In International Conference on Software Process Improvement and Capability Determination (pp. 141-147). Springer Berlin Heidelberg.
McHugh, M., McCaffery, F., & Coady, G. (2015). Adopting Agile Practices when Developing Medical Device Software.
McHugh, M., McCaffery, F., Casey, V., & Pikkarainen, M. (2012). Integrating agile practices with a medical device software development lifecycle.
Morsicano, R., & Shoemaker, B. Tutorial: Agile Methods in Regulated and Safety-Critical Environments. ShoeBar Associates.
Mwadulo, M. W., (2016, September). Suitability of Agile Methods for Safety-Critical Systems Development: A Survey of Literature. International Journal of Computer Applications Technology and Research Volume 5– Issue 7, 465 - 471, 2016, ISSN:- 2319–8656
Nidagundi, P., & Novickis, L. (2017). Introducing lean canvas model adaptation in the scrum software testing. Procedia Computer Science, 104, 97-103.
Opt, S. K, Sims, C. D. L. (2015). Scrum: Enhancing Student Team Organization and Collaboration. Communication Teacher, 29(1), 55-62.
Rafi, U., Mustafa, T., Iqbal, N., & Zafar, W. U. I. (2015). US-Scrum: A Methodology for Developing Software with Enhanced Correctness, Usability and
Security. International Journal of Scientific & Engineering Research, 6(9), 377.
Rasmussen, R., T. Hughes, J.R. Jenks, and J. Skach, Adopting Agile in an FDA Regulated Environment, in Agile Conference, 2009. AGILE '09. . 2009: Chicago, IL p. 151‐155.
and empirical study on success of risk management activity with respect to
scrum. Engineering Science and Technology: An International Journal (ESTIJ), 2(3), 467-473.
Regan, G., McCaffery, F., McDaid, K., & Flood, D. (2013). Medical device standards' requirements for traceability during the software development lifecycle and
implementation of a traceability assessment model. Computer Standards & Interfaces, 36(1), 3-9.
Robson C (2002) Real World Research. Blackwell, (2nd edition)
Rola, P., Kuchta, D., & Kopczyk, D. (2016). Conceptual model of working space for Agile (Scrum) project team. Journal Of Systems And Software, 118(2), 49-63. doi:10.1016/j.jss.2016.04.071
Rottier, P.A. and V. Rodrigues, Agile Development in a Medical Device Company, in Agile, 2008. AGILE '08. Conference. 2008.)
Runeson, P., & Höst, M. (2009). Guidelines for conducting and reporting case study research in software engineering. Empirical software engineering, 14(2), 131.
Rust, P., Flood, D., & McCaffery, F. (2016). Creation of an IEC 62304 compliant software development plan. Journal of Software: Evolution and Process, 28(11), 1005-1010.
Santos, I. C., Gazelle, G. S., Rocha, L. A., & Tavares, J. M. R. (2012). Medical device specificities: opportunities for a dedicated product development methodology. Expert review of medical devices, 9(3), 299-311.
Seaman C (1999) Qualitative methods in empirical studies of software engineering. IEEE Trans Softw Eng 25(4):557–572 see also Chapter 2 in Shull et al. (2008) Seventh International Conference on Software Engineering Advances
Stålhane, T., Myklebust, T., & Hanssen, G. K. (2012). The application of Safe Scrum to IEC 61508 certifiable software. In 11th International Probabilistic Safety
Assessment and Management Conference and the Annual European Safety and Reliability Conference (pp. 6052-6061).
Stavru, S. (2014). A critical examination of recent industrial surveys on agile method usage. Journal of Systems and Software, 94, 87-97.
269-276.
Sutherland, J., & Schwaber, K. (2013). The scrum guide. The Definitive Guide to Scrum: The Rules of the Game. Scrum. org.
Takahira, R. Y., Laraia, L. R., Dias, F. A., Abraham, S. Y., Nascimento, P. T., & Camargo, A. S. (2014, July). Scrum and Embedded Software development for the automotive industry. In Management of Engineering & Technology (PICMET), 2014 Portland International Conference on (pp. 2664-2672). IEEE.
Turk, D., France, R., & Rumpe, B. (2014). Assumptions underlying agile software development processes. arXiv preprint arXiv:1409.6610.
Yin RK (2003) Case study research. Design and methods, 3rd edn. London, Sage
Guo, Z., & Hirschmann, C. (2012). An Integrated Process for Developing Safety-critical Systems using Agile Development Methods. In ICSEA 2012 The Seventh International Conference on Software Engineering Advances.
Appendix 1
Safe Scrum requires creating two backlogs: one is dedicated to the functional requirements and the other to the safety requirements. The creators notice that the safety requirements usually tend to be stable in safety critical projects and only the functional ones are subject to frequent changes. Moreover, these requirements should be linked to each other in order to identify which safety requirement is affected by modification of the functional requirement.
The second extension proposed is that all processes need to be traceable.
“All decisions and changes throughout development must be documented, stored and made available to the assessor. The same goes for code reviews where all remarks and how they were resolved needs to be kept track of” (Hanssen et al. 2016, page 96). Fitzgerald et al. (page 869, 2013) coined a new term “living traceability’’ and suggest that traceability has to be recorded during all phases of medical devices software development. Nevertheless, Fitzgerald et al. (2013) suggest that links should automatically be established as developers check in code that realizes a certain task. This also helps the software developers to prove that the requirements specifications meet all stated customer and regulatory commitments.
In addition, McHugh et al. (2012) create a special tool, called “Hecho”, which facilitate the agile software developers to maintain traceability between the requirements and the different phases of the project development.
formally documented. The CIA has to be assessed during the Sprint review and consequently the product backlog needs to be updated. The Safe Scrum also requires adopting an agile practice called Test-Driven Development, which helps the Development Team to test the new code created during the Sprint. Using Safe Scrum would help the developers to produce documentation throughout the whole software development.
Finally, Safe Scrum proposes that a Quality Assurance Team needs to be designated to alleviate problems of quality assurance during software development. The QA team has specific tasks, which are: “check documentation coverage, check test coverage and check requirement task code traceability” (Hanssen et al. 2016, page 101).
Appendix 2
Arranging the interview
Explain the goal of this interview (I’m doing this interview in order to gain insight on how the company uses Scrum to manage the development of medical device software that must be compliant to the new amendment MDD (2007/47/EC) and its related standard IEC62304, ISO 13485 and ISO 14971) and offer them the end result.
Explain the end result of this research ad indicate that an audio recording of the interview is necessary.
State that probable interview time is between 1 to 1.5 hours.
Explain the structure of the interview, which is as follows: • Introduction
• Questions regarding general company information
• General questions regarding the use of Scrum with in the company.
Introduction phase • Introduce yourself.
• Explain the purpose of the interview.
• Again, before the interview starts permission should be asked to record the interview. Explain why this is necessary if needed.
Introduction questions (<5 minutes)
The following questions are related to the company you work for.
• You work for company X. Could you briefly explain what products the company produces and who are its customers? (These questions encompass the whole organization)
• Could you describe your role within the company? • How good is your knowledge of Agile and Scrum?
Explain the research question and how it comes to be as introduction to the next set of questions.
• What Scrum’s extensions do Medical Device Software developers need to ensure regulatory compliance with the European legislation?
General questions regarding the use of Scrum (10/15 min.)
• When did the company start to use Scrum to manage the development of Software?
• Can you tell me how many employees are involved in the software development project and what roles do they cover?
• Is Scrum successful for managing this type of project?
• Is any other party involved in the software development? (Ex. external stakeholders) what role do they have in the software development?
Questions regarding software development process and Scrum artifacts • Initial planning of the project.
• Who define the product requirements?
• Who does create and manage the Product Backlog? • And how long does it take?
• Does anyone support this task?
The Sprint
• Who participate to the Sprint planning? And why?
• Who create the Sprint Backlog? And who does prioritize the work? • How long does a Sprint last?
• What activities are performed during a Sprint?
• Does the Scrum team use daily standup meeting to organize the daily work? Who does participate to it?
• Sprint review: Can you describe who does participate to it and what activities are executed?
• Sprint retrospective: Can you describe who does participate to it and what activities are executed?
Documentation, traceability and testing
• How do you produce the documents required for certification? And who is responsible for it?
• When do you test the functionalities of the software? And the whole software? • Is the code developed using constant traceability?
• Can you describe the different phases of the product testing and how compliance is achieved?
Closing questions (5 minutes)
• Are there any points regarding the software development process that have not been covered in this interview but that you would have expected and would like to talk about?
• If I have any questions after transcribing the interview can I contact you?
End of the interview (5 minutes)
• Thank the interviewee for his or her time and the collected information
