This thesis reports is the final step of the “Computer Science and Engineering”
graduate program at Eindhoven University of Technology (TU/e). The thesis investigates “Product Software Quality” and was conducted at the Laboratory for Quality Software (LaQuSo), activity of the Faculty of Mathematics and Computer Science at TU/e.
LaQuSo aims to measure quantify and predict the quality of the designed software in order to increase the predictability of the development process. The focus of LaQuSo is on quality of software. The development process is not the subject of this study, but only the output of the development process.
This thesis project was related to LaQuSo certification services, where LaQuSo certifies software artifacts as an independent evaluator. The architecture of LaQuSo certification is presented on the figure below:
Figure 1 LaQuSo certification architecture
The project falls under “validation (empirical) methods, techniques and tools” in LaQuSo competences, presented on LaQuSo web site on following URL:
http://www.laquso.com/research/researchcertification.php.
Research Questions
This document reports on a study of: Product Software Quality, Quality models and Product Software.
In this project, we aim at researching the product software quality assessment, and argue the need of a quality model as a first step in the software quality assessment. A quality model is a set of quality characteristics that should be evaluated. We expect that the relative importance of the quality characteristics could be domain-dependent, where for some domain one characteristic may be very important, while for other domains the same characteristic may be irrelevant.
We address the following research questions:
• How do we measure product software quality?
Software quality in general is hard to define and therefore hard to measure.
• Is product software quality domain dependent?
We believe that different quality characteristics are important for different application domains. Our research should be able to endorse or reject this conjecture.
• How can we use the ISO/9126-1 quality model in different domains?
ISO/9126-1 is too general in order to give coverage for all application domains.
We expect that for some domains a reduced set of ISO/9126-1 can be used for assessing the quality. Using a reduced set of characteristics, the assessment and verification effort can also be reduced.
• What are the differences between product software quality and tailor-made software quality?
We want to compare which quality characteristics are important for the tailor-made software.
Our expectation is that for different domains we can define different domain-based quality models. These domain based quality models will be based on the ISO/9126-1 quality model, but we expect that they will not be identical to the ISO/9126-1 model.
Research Methods
During this project, we used the following research methods:
• Literature study
A literature study involves reviewing readily available scientific papers related to the research area. We conducted literature study in the areas of Quality, Software Quality Models and Product Software.
• Conducting a survey This method had several parts:
o Designing a questionnaire required for executing structured interviews.
We prepared a questionnaire and enclosed letters according to the theory of questionnaire design [Litkowski]. The complete version of the questionnaire is available in the Appendix A.
o Personal interviews with different stakeholders
We also used the web and library, scientific and industrial resources for interviewing [Litkowski]. These resources helped in conducting structured interviews.
o Analysis of the survey
The results of the interviews were further analysed. On the basis of the interviews, we tried to extract information which quality characteristics are important for the specific product software and for the specific application domain. Due to lack of insufficient feedback, our survey did not provide us with statistically significant results.
• Quality evaluation and construction of a quality model for real product software applications
This step will help in assessing relevance of the quality model on a real software application. As a starting point, we executed domain analysis in order to define which quality sub-characteristics are relevant for a specific product and to reduce the evaluation effort focusing on relevant characteristics only. To this end we departed from the ISO/IEC 9126 standard and made use of the methodology proposed in [Botella] for building ISO/IEC 9126-based quality models. The methodology consists of several steps and the basic idea is to derive metrics starting from ISO 9126 quality characteristics. In our work, we derive attributes for all relevantISO 9126 sub-characteristics and propose metrics for these attributes. As a guideline we used external metrics provided by [ISO/IEC 9126-2]: some metrics were literary taken from [ISO/IEC 9126-2], other were inspired by the [ISO/IEC 9126-2] metrics, while the third part were metrics not defined by the standard but related to the application domain of the product software.
Report Outline
The rest of this thesis is organized as follows. Chapter 2 explains the notion of product software and discusses the place of product software in the software market.
Chapter 3 focuses on quality, software quality, and quality models; the chapter describes related terms and previous research. Chapter 4 discusses the related research.
In Chapter 5, we present the survey results and the analysis of these results. Chapter 6 presents the evaluation procedure of four product software applications. Chapter 7 contains reflections and conclusion remarks.