University of Groningen
Continuous integration and delivery applied to large-scale software-intensive embedded
systems
Martensson, Torvald
IMPORTANT NOTE: You are advised to consult the publisher's version (publisher's PDF) if you wish to cite from it. Please check the document version below.
Document Version
Publisher's PDF, also known as Version of record
Publication date: 2019
Link to publication in University of Groningen/UMCG research database
Citation for published version (APA):
Martensson, T. (2019). Continuous integration and delivery applied to large-scale software-intensive embedded systems. University of Groningen.
Copyright
Other than for strictly personal use, it is not permitted to download or to forward/distribute the text or part of it without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license (like Creative Commons).
Take-down policy
If you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediately and investigate your claim.
Downloaded from the University of Groningen/UMCG research database (Pure): http://www.rug.nl/research/portal. For technical reasons the number of authors shown on this cover page is limited to 10 maximum.
Bibliography
─ Afzal, W., Ghazi, A.N., Itkonen, J., (...), Andrews, A. and Bhatti, K. (2015). An experiment on the effectiveness and efficiency of exploratory testing. Empirical Software Engineering 20(3), pp. 844-878.
─ Agresti, A. and Kateri, M. (2011). Categorical data analysis. Springer.
─ Ahmadighohandizi, F. and Systä, K. (2015) Icdo: integrated cloud-based develop-ment tool for devops, CEUR Workshop, pp. 76-90.
─ Andersson, H., Herzog, E. and Ölvander, J. (2013). Experience from model and soft-ware reuse in aircraft simulator product line engineering, Information and Softsoft-ware Technology, Volume 55 Issue 3, pp. 595-606.
─ Austel, P., Chen, H., Mikalsen, T., Rouvellou, I., Sharma, U., Silva-Lepe, I. and Subramanian, R. (2015). Continuous delivery of composite solutions: a case for col-laborative software defined paas environments. 2nd International Workshop on Soft-ware-Defined Ecosystems, pp. 3-6.
─ Bailey, S. and Handu D. (2012). Introduction to epidemiologic research methods in public health practice. Jones and Bartlett Publishers.
─ Bass, L., Holz, R., Rimba, P., Tran, A.B. and Zhu, L. (2015). Securing a deployment pipeline. 3rd International Workshop On Release Engineering. pp. 4-7.
─ Beck, K. (1999). Extreme programming explained: embrace change (1st edition). Addison-Wesley.
─ Beck, K. (2005). Extreme programming explained: embrace change (2nd edition). Addison-Wesley.
─ Bellomo, S., Ernst, N., Nord, R. and Kazman, R. (2014). Toward design decisions to enable deployability: empirical study of three projects reaching for the continuous delivery holy grail. 44th Annual IEEE/IFIP International Conference on Dependable Systems and Networks, DSN, pp. 702–707.
─ Bellomo, S., Nord, R. L. and Ozkaya, I. (2013). A study of enabling factors for rapid fielding combined practices to balance speed and stability. 35th International Con-ference on Software Engineering, ICSE, pp. 982–991.
─ Boehm, B. (1976), Software engineering, IEEE Transactions on Computers, Volume C-25 Issue 12, pp. 1226-1241.
─ Boehm, B. (1984). Software engineering economics. IEEE Transactions on Software Engineering 10 (1), pp. 4–21.
─ Boehm, B. (1988). A spiral model of software development and enhancement. Com-puter vol. 21 no. 5, pp. 61-72.
─ Boehm, B. W., Madachy, R. and Steece, B. (2000). Software cost estimation with Cocomo II with Cdrom. Prentice Hall PTR.
─ Booch, G. (1994). Object-oriented analysis and design with applications (2nd edi-tion). Addison-Wesley Longman.
─ Briand, L. C., Morasca, S. and Basili, V. R. (1996). Property-based software engi-neering measurement. IEEE Transactions on Software Engiengi-neering 22 (1), pp. 68– 86.
─ Brooks, F. P. (1975). The mythical man-month. Essays on Software Engineering. Addison-Wesley.
─ Brooks, G. (2008). Team pace keeping build times down, Agile Conference, IEEE, pp. 294–297.
─ Buchgeher, G., Klammer, C., Heider, W., Schuetz, M. and Huber, H. (2016). Im-proving testing in an enterprise soa with an architecture-based approach. 13th Work-ing IEEE/IFIP Conference on Software Architecture, WICSA, pp. 231–240. ─ Callanan M. and Spillane, A. (2016). Devops: making it easy to do the right thing.
IEEE Software, vol. 33, no. 3, pp. 53-59.
─ Chen, L. (2015a). Continuous delivery: huge benefits, but challenges too. IEEE Soft-ware, vol. 32, no. 2, pp. 50-54.
─ Chen, L. (2015b). Towards architecting for continuous delivery. 12th Working IEEE/IFIP Conference on Software Architecture, pp. 131-134.
─ Cohn, M. (2005). Agile estimation and planning. Addison-Wesley.
─ Collis, J. and Hussey, R. (2009). Business research: a practical guide for undergrad-uate and postgradundergrad-uate students (3rd edition). Palgrave Macmillan.
─ Conway, M. E. (1968). How do committees invent. Datamation 14 (4), pp. 28–31. ─ Cook, T. D., Campbell, D. T. and Day, A. (1979). Quasi-experimentation: design &
analysis issues for field settings. Vol. 351. Houghton Mifflin Boston.
─ Debois, P. (2012). Devops areas. http://www.jedi.be/blog/2012/05/12/codifying-devops-area-practices (accessed January 02, 2017).
─ De Gouw, S., Mauro, J., Nobakht, B. and Zavattaro, G. (2016). Declarative elasticity in abs. European Conference on Service-Oriented and Cloud Computing, pp. 118-134.
─ Del Rosso, C. (2006). Continuous evolution through software architecture evalua-tion: a case study. Journal of Software Maintenance and Evolution 18 (5), pp. 351-383.
─ Do Nascimento, L.H.O. and Machado, P.D.L. (2007). An experimental evaluation of approaches to feature testing in the mobile phone applications domain. Workshop on Domain-Specific Approaches to Software Test Automation - In conjunction with the 6th ESEC/FSE Joint Meeting, pp. 27-33.
─ Downs, J., Hoskins J. and Plimmer, B. (2010). Status communication in agile soft-ware teams: a case study. Fifth International Conference on Softsoft-ware Engineering Advances, ICSEA, pp. 82-87.
─ Duvall, P. (2007). Continuous integration, Addison Wesley.
─ Dösinger, S., Mordinyi R. and Biffl, S. (2012). Communicating continuous integra-tion servers for increasing effectiveness of automated testing. 27th IEEE/ACM In-ternational Conference on Automated Software Engineering, pp. 374–377.
─ Ebert, C., Gallardo, G. Hernantes, J. and Serrano, N. (2016). Devops. IEEE Soft-ware, vol. 33, no. 3, pp. 94-100.
─ Engblom, J. (2015). Virtual to the (near) end - using virtual platforms for continuous integration. Design Automation Conference, 2015-July, 7167385.
─ Erich, F., Amrit, C. and Daneva, M. (2014). Cooperation between information sys-tem development and operations: a literature review. 8th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, p. 69.
─ Fenton, N. E. and Neil, M. (2000). Software metrics: roadmap. Conference on the Future of Software Engineering, ACM, pp. 357–370.
─ Fitzgerald, B., Stol, K.-J., O’Sullivan, R. and O’Brien, D. (2013). Scaling agile methods to regulated environments: an industry case study. International Conference on Software Engineering, pp. 863-872.
─ Fitzgerald, B. and Stol, K.-J. (2014). Continuous software engineering and beyond: trends and challenges. 1st International Workshop on Rapid Continuous Software Engineering, pp. 1-9.
─ Fitzgerald, B. and Stol, K.-J. (2015). Continuous software engineering: a roadmap and agenda. Journal of Systems and Software 123, pp. 176-189.
─ Fowler, M. (2006). Continuous integration. http://www.martinfowler.com/arti-cles/continuousIntegration.html (accessed online February 12, 2016).
─ Fowler, M. (2013). Continuous delivery, http://martinfowler.com/bliki/Continu-ousDelivery.html (accessed January 02, 2017).
─ Frajtak, K., Bures, M. and Jelinek, I. (2016). Model-based testing and exploratory testing: Is synergy possible?. 6th International Conference on IT Convergence and Security, ICITCS 2016.
─ Frajtak, K., Bures, M. and Jelinek, I. (2017). Exploratory testing supported by auto-mated reengineering of model of the system under test. Cluster Computing 20(1), pp. 855-865.
─ Garousi, V. and Mäntylä, M.V. (2016a). A systematic literature review of literature reviews in software testing. Information and Software Technology 80, pp. 1339-1351.
─ Garousi, V. and Mäntylä, M.V. (2016b). When and what to automate in software testing? A multi-vocal literature review. Information and Software Technology 76, pp. 92-117.
─ Garousi, V. and Pfahl, D. (2016). When to automate software testing? A decision-support approach based on process simulation. Journal of Software: Evolution and Process 28(4), pp. 272-285.
─ Gebizli, C.Ş. and Sözer, H. (2016). Automated refinement of models for model-based testing using exploratory testing. Software Quality Journal, pp. 1-27.
─ Ghazi, A.N., Petersen, K. and Börstler, J. (2015). Heterogeneous systems testing techniques: an exploratory survey. Lecture Notes in Business Information Pro-cessing 200, pp. 67-85.
─ Gilb, T. (1976). Software metrics (Winthrop computer systems series). Winthrop. ─ Gmeiner, J., Ramler, R. and Haslinger, J. (2015). Automated testing in the
continu-ous delivery pipeline: a case study of an online company. IEEE 8th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2015.
─ Goodman D. and Elbaz, M. (2008). “It’s not the pants, it’s the people in the pants” learning from the the gap agile transformation what worked, how we did it, and what still puzzles us. Agile Conference AGILE’08, pp. 112-115.
─ Gotimer, G. and Stiehm, T. (2016). Devops advantages for testing: increasing quality through continuous delivery. CrossTalk Magazine, vol. May/Jun 2016, pp. 13-18.
─ Gouveia, N. (2016). Agile testing on an online betting application. Lecture Notes in Business Information Processing 251, pp. 193-200.
─ Grandy, H. and Benz, S. (2009). Specification based testing of automotive human machine interfaces. INFORMATIK 2009 - Im Focus das Leben, Beitrage der 39. Jahrestagung der Gesellschaft fur Informatik e.V. (GI), pp. 2720-2727.
─ Gregory, J. and Crispin, L. (2015). More agile testing. Addison Wesley. ─ Halstead, M. H. (1977). Elements of software science. Vol. 7. Elsevier. ─ Hendrickson, E. (2013). Explore it!. The Pragmatic Bookshelf.
─ Herzberg, F., Mausner B. and Snyderman, B. (1959). The motivation to work (2nd ed), John Wiley.
─ Honor, A. (2010). People over process over tools. http://dev2ops.org/2010/02/peo-ple-over-process-over-tools/ (accessed January 10, 2017).
─ Humble, J. (2010). Continuous delivery vs continuous deployment. http://jezhum-ble.com/2010/08/continuous-delivery-vs-continuous-deployment (accessed August 30, 2016).
─ Humble, J. (2014). The case for continuous delivery. https://www.thought-works.com/insights/blog/case-continuous-delivery (accessed online November 30, 2016).
─ Humble, J. and Farley, D. (2010). Continuous delivery: reliable software releases through build, test, and deployment automation. Addison Wesley.
─ Humble J. and Molesky J. (2011). Why enterprises must adopt devops to enable continuous delivery. Cutter IT Journal, vol. 24, no. 8, p. 6.
─ Itkonen, J. and Mäntylä, M.V. (2014). Are test cases needed? Replicated comparison between exploratory and test-case-based software testing. Empirical Software Engi-neering 19(2), pp. 303-342.
─ Itkonen, J., Mäntylä, M.V. and Lassenius, C. (2007). Defect detection efficiency: test case based vs. exploratory testing. 1st International Symposium on Empirical Software Engineering and Measurement, ESEM 2007, pp. 61-70.
─ Itkonen, J., Mäntylä, M.V. and Lassenius, C. (2009). How do testers do it? an ex-ploratory study on manual testing practices. 3rd International Symposium on Empir-ical Software Engineering and Measurement, ESEM 2009, pp. 494-497.
─ Itkonen, J., Mäntylä, M.V. and Lassenius, C. (2013). The role of the tester’s knowledge in exploratory software testing. IEEE Transactions on Software Engi-neering, 39(5), pp. 707-724.
─ Itkonen, J., Mäntyla, M.V. and Lassenius, C. (2016). Test better by exploring: har-nessing human skills and knowledge. IEEE Software 33(4), pp. 90-96.
─ Itkonen, J. and Rautiainen, K. (2005). Exploratory testing: a multiple case study. International Symposium on Empirical Software Engineering, ISESE 2005, pp. 84-93.
─ Jacobson, I., Booch, G. and Rumbaugh, J. (1999). The unified software development process. Addison Wesley.
─ Kaner, C. (1988). Testing Computer Software. TAB Books.
─ Kaner, C., Bach, J. and Pettichord, B. (2001). Lessons Learned in Software Testing. John Wiley & Sons.
─ Karlström, D. (2002). Introducing extreme programming-an experience report. 3rd International Conference on eXtreme Processing and Agile Processing Software En-gineering, XP2002.
─ Kitchenham, B. (2004). Procedures for performing systematic reviews. Keele UK Keele University 33, pp. 1–26.
─ Knauss, E., Staron, M., Meding, W., (...), Nilsson, A. and Castell, M. (2015a). Sup-porting continuous integration by code-churn based test Selection. 2nd International Workshop on Rapid Continuous Software Engineering, pp. 19-25.
─ Knauss, E., Andersson, A., Rybacki M. and Israelsson, E. (2015b). Research pre-view: supporting requirements feedback flows in iterative system development. 21st International Working Conference on Requirements Engineering: Foundation for Software Quality, REFSQ 2015, pp. 277-283.
─ Kohonen, T. and Somervuo, P. (1998). Self-organizing maps of symbol strings. Neu-rocomputing, vol. 21, no. 1, pp. 19-30.
─ Krusche S. and Alperowitz L. (2014). Introduction of continuous delivery in multi-customer project courses. 36th International Conference on Software Engineering, pp. 335-343.
─ Kuhn, A. (2013). On extracting unit tests from interactive live programming ses-sions. International Conference on Software Engineering, pp. 1241-1244.
─ Kumar, S. and Wallace, C. (2013). Guidance for exploratory testing through problem frames. Software Engineering Education Conference, pp. 284-288.
─ Labuschagne, A., Inozemtseva, L. and Holmes, R. (2017). Measuring the cost of regression testing in practice: a study of java projects using continuous integration. ACM SIGSOFT Symposium on the Foundations of Software Engineering, Part F130154, pp. 821-830.
─ Laghari, G., Murgia, A. and Demeyer, S. (2016). Fine-tuning spectrum based fault localisation with frequent method item sets. 31st IEEE/ACM International Confer-ence on Automated Software Engineering, pp. 274-285.
─ Larman, C. and Vodde, B. (2010). Practicies for scaling lean & agile development – large, multisite, and offshore product development with large-scale scrum. Addison Wesley.
─ Laukkanen E., Paasivaara M., Itkonen J., Lassenius C. and Arvonen T. (2017). To-wards continuous delivery by reducing the feature freeze period: a case study. IEEE/ACM 39th International Conference on Software Engineering: Software En-gineering in Practice Track, ICSE-SEIP 2017, pp. 23-32.
─ Leffingwell, D. (2011). Agile software requirements. Addison Wesley.
─ Lier, F., Lütkebohle I. and Wachsmuth, S. (2014). Towards automated execution and evaluation of simulated prototype hri experiments. ACM/IEEE International Conference on Human-Robot Interaction, pp. 230-231.
─ Lindbeck, A. and Snower, D. (1984). Involuntary unemployment as an insider-out-sider dilemma. Seminar Paper No. 309, Institute for International Economic Studies, University of Stockholm.
─ Liu, D., Zhu, H., Xu, C., (...), Green, M., Marshall, P. (2016). Cide: an integrated development environment for microservices. IEEE International Conference on Ser-vices Computing, pp. 808-812.
─ Liu, Y., Lu, Y. and Li, Y. (2014). An android-based approach for automatic unit test, IET Conference Publications.
─ Lwakatare, L.E., Kuvaja, P. and Oivo, M. (2015). Dimensions of devops. Interna-tional Conference on Agile Software Development, pp. 212-217.
─ Makondo, W., Nallanthighal, R., Mapanga, I. and Kadebu, P. (2016). Exploratory test oracle using multi-layer perceptron neural network. 2016 International Confer-ence on Advances in Computing, Communications and Informatics, ICACCI 2016, pp. 1166-1171.
─ Marijan, D. (2015). Multi-perspective regression test prioritization for time-con-strained environments. IEEE International Conference on Software Quality, Relia-bility and Security, pp. 157-162.
─ Marijan, D., Gotlieb, A. and Sen, S. (2013). Test case prioritization for continuous regression testing: an industrial case study. IEEE International Conference on Soft-ware Maintenance, ICSM, pp. 540-543.
─ Marijan D., Liaaen M., Gotlieb A., Sen, S. and Ieva C. (2017) Titan: test suite opti-mization for highly configurable software. 10th IEEE International Conference on Software Testing, Verification and Validation, ICST 2017, pp. 524-531.
─ McCabe, T. J. (1976). A complexity measure. IEEE Transactions on Software En-gineering (4), pp. 308–320.
─ Messina, A., Rizzo, R., Storniolo, P., Tripiciano, M. and Urso, A. (2016). The da-tabase-is-the-service pattern for microservice architectures. 7th International Con-ference on Information Technology in Bio- and Medical Informatics, ITBAM 2016, pp. 223-233.
─ Micallef, M., Porter, C. and Borg, A. (2016). Do exploratory testers need formal training? an investigation using hci techniques. IEEE International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2016, pp. 305-314.
─ Mihindukulasooriya, N., Rizzo, G., Troncy, R., Corcho, O. and García-Castro, R. (2016). A two-fold quality assurance approach for dynamic knowledge bases: the 3cixty use case, CEUR Workshop Proceedings.
─ Miller, A. (2008). A hundred days of continuous integration. Agile 2008 Conference. ─ Moss, C. (2013). Big visible testing. AGILE 2013, pp. 94-100.
─ Mossige, M., Gotlieb A. and Meling, H. (2013). Test generation for robotized paint systems using constraint programming in a continuous integration environment. IEEE 6th International Conference on Software Testing, Verification and Valida-tion, ICST 2013, pp. 489-490.
─ Mossige, M., Gotlieb A. and Meling, H. (2014a). Testing robotized paint system using constraint programming: an industrial case study. Lecture Notes in Computer Science, pp. 145-160.
─ Mossige, M., Gotlieb A. and Meling, H. (2014b). Using cp in automatic test gener-ation for ABB robotics’ paint control system. Lecture Notes in Computer Science, pp. 25-41.
─ Mossige, M., Gotlieb A. and Meling, H. (2015). Testing robot controllers using con-straint programming and continuous integration. Source of the Document Infor-mation and Software Technology 57 (1), pp. 169-185.
─ Mueller, E. (2010). A devops manifesto. https://theagileadmin.com/2010/10/15/a-devops-manifesto/ (accessed January 10, 2017).
─ Mueller, E. (2016). What is devops?. https://theagileadmin.com/what-is-devops/ (accessed January 02, 2017).
─ Mårtensson, T., Ståhl, D. and Bosch, J. (2016). Continuous integration applied to software-intensive embedded systems – problems and experiences. 17th Interna-tional Conference on Product-Focused Software Process Improvement, PROFES 2016, pp. 448-457.
─ Mårtensson, T., Hammarström P. and Bosch, J. (2017a). Continuous integration is not about build systems. 43rd Euromicro Conference on Software Engineering and Advanced Applications, SEAA 2017, pp. 1-9.
─ Mårtensson, T., Ståhl, D. and Bosch, J. (2017b). Continuous integration impedi-ments in large-scale industry projects. IEEE International Conference on Software Architecture (ICSA), pp. 169-178.
─ Mårtensson, T., Ståhl D. and Bosch, J. (2017c). The emfis model – enable more frequent integration of software. 43rd Euromicro Conference on Software Engineer-ing and Advanced Applications, SEAA 2017, pp. 10-17.
─ Mårtensson, T., Ståhl, D. and Bosch J. (2018). Enable more frequent integration of software in industry projects. Journal of Systems and Software 142, pp. 223-236. ─ Nagappan, N., Murphy, B. and Basili, V. (2008). The influence of organizational
structure on software quality: an empirical case study. 30th international conference on Software engineering, ACM, pp. 521–530.
─ Newman, S. (2015). Building microservices. O’Reilly Media.
─ Olsson, H. H., Alahyari, H. and Bosch, J. (2012). Climbing the “stairway to heaven” – a mulitiple-case study exploring barriers in the transition from agile development towards continuous deployment of software. 38th Euromicro Conference on Soft-ware Engineering and Advanced Applications, SEAA 2012, pp. 392-399.
─ Olsson, H.H. and Bosch, J. (2014). Towards agile and beyond: an empirical account on the challenges involved when advancing software development practices. Lecture Notes in Business Information Processing 179 LNBIP, pp. 327-335.
─ Olszewska M. and Waldén M. (2015). Devops meets formal modelling in high-criticality complex systems. 1st International Workshop on Quality-Aware DevOps, pp. 7-12.
─ Owen Rogers, R. (2004). Scaling continuous integration. Extreme Programming and Agile Processes in Software Engineering, pp. 68–76.
─ Penmetsa, J.R. (2016). Agile testing. Trends in Software Testing, pp. 19-33. ─ Perry, D. E., Staudenmayer, N. A. and Votta, L. G. (1994). People, organizations,
and process improvement. Software, IEEE 11 (4), pp. 36–45.
─ Petersen, K., Feldt, R., Mujtaba, S. and Mattsson, M. (2008). Systematic mapping studies in software engineering. 12th International Conference on Evaluation and Assessment in Software Engineering, vol. 17, no. 1.
─ Pfahl, D., Yin, H., Mäntylä, M.V. and Münch, J. (2014). How is exploratory testing used? a state-of-the-practice survey. International Symposium on Empirical Soft-ware Engineering and Measurement.
─ Pichler, J. and Ramler, R. (2008). How to test the intangible properties of graphical user interfaces?. 1st International Conference on Software Testing, Verification and Validation, ICST 2008, pp. 494-497.
─ Prakash, V. and Gopalakrishnan, S. (2011). Testing efficiency exploited: scripted versus exploratory testing. 3rd International Conference on Electronics Computer Technology, pp. 168-172.
─ Preuveneers, D., Heyman, T., Berbers, Y. and Joosen, W. (2016). Systematic scala-bility assessment for feature oriented multi-tenant services. Journal of Systems and Software 116, pp. 162-176.
─ Project Management Institute (2013). A Guide to the Project Management Body of Knowledge, PMBOK Guide (5th Edition).
─ Raappana, P., Saukkoriipi, S., Tervonen, I. and Mäntylä, M.V. (2016). The effect of team exploratory testing - experience report from f-secure. IEEE International Con-ference on Software Testing, Verification and Validation Workshops, ICSTW 2016, pp. 295-304.
─ Rashmi, N. and Suma, V. (2014). Defect detection efficiency of the combined ap-proach. Advances in Intelligent Systems and Computing, 249 VOLUME II, pp. 485-490.
─ Rathod, N. and Surve, A (2015). Test orchestration – a framework for continuous integration and continuous deployment. International Conference on Pervasive Computing: Advance Communication Technology and Application for Society, ICPC 2015.
─ Roberts, M. (2004). Enterprise continuous integration using binary dependencies. 5th International Conference on Extreme Programming and Agile Processes in Soft-ware Engineering, pp. 194-201.
─ Robson, C. (2002). Real world research (2nd edition). Blackwell.
─ Robson, C. and McCartan, K. (2016). Real world research (4th edition). Wiley. ─ Rodríguez, P., Haghighatkhah, A., Lwakatare, L.E., (...), Verner, J.M. and Oivo, M.
(2016). Continuous deployment of software intensive products and services: a sys-tematic mapping study. Journal of Systems and Software 123, pp. 263-291. ─ Rosenkranz, P., Wählisch, M., Baccelli, E. and Ortmann, L. (2015). A distributed
test system architecture for open-source iot software. Workshop on IoT Challenges in Mobile and Industrial Systems, pp. 43-48.
─ Royce, W. (1970). Managing the development of large software systems: concepts and techniques. WESCON Technical Papers, vol. 14.
─ Runeson, P. and Höst, M. (2009). Guidelines for conducting and reporting case study research in software engineering. Empirical Software Engineering 14 (2), pp. 131-164.
─ Saunders, M., Lewis, P. and Thornhill, A. (2009). Research methods for business students (5th edition). Pearson Education.
─ Schaefer, C.J. and Do, H. (2014). Model-based exploratory testing: a controlled ex-periment. IEEE 7th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2014, pp. 284-293.
─ Schaefer, C., Do, H. and Slator, B.M. (2013). Crushinator: a framework towards game-independent testing. 28th IEEE/ACM International Conference on Automated Software Engineering, ASE 2013, pp. 726-729.
─ Scholtes, I., Mavrodiev, P. and Schweitzer, F. (2016). From aristotle to ringelmann: a large-scale analysis of team productivity and coordination in open source software projects. Empirical Software Engineering 21 (2), pp. 642–683.
─ Sekitoleko, N., Evbota, F., Knauss, E., (...), Chaudron, M. and Olsson, H.H. (2014). Technical dependency challenges in large-scale agile software development. 15th International Conference on Agile Software Development, XP 2014, pp. 46-61. ─ Seth N. and Khare, R. (2015). Aci (automated continuous integration) using jenkins:
key for successful embedded software development. 2nd International Conference on Recent Advances in Engineering and Computational Sciences, RAECS 2015. ─ Shah, S.M.A., Gencel, C., Alvi, U.S. and Petersen, K. (2014a). Towards a hybrid
testing process unifying exploratory testing and scripted testing. Journal of software: Evolution and Process, 26(2), pp. 220-250.
─ Shah, S.M.A., Torchiano, M., Vetrò, A. and Morisio, M. (2014b). Exploratory test-ing as a source of technical debt. IT Professional 16(3), pp. 44-51.
─ Shah, S.M.A., Alvi, U.S., Gencel, C. and Petersen, K. (2014c). Comparing a hybrid testing process with scripted and exploratory testing: an experimental study with practitioners. Lecture Notes in Business Information Processing, 179 LNBIP, pp. 187-202.
─ Shahin, M. (2015). Architecting for devops and continuous deployment. 24th Aus-tralasian Software Engineering Conference, pp. 147-148.
─ Shoaib, L., Nadeem, A. and Akbar, A. (2009). An empirical evaluation of the influ-ence of human personality on exploratory software testing. IEEE 13th International Multitopic Conference.
─ Soni, M. (2015). End to end automation on cloud with build pipeline: the case for devops in insurance industry, continuous integration, continuous testing, and contin-uous delivery. International Conference on Cloud Computing in Emerging Markets, pp. 85-89.
─ Stigler, G. J. (1958). Economies of scale, the. JL & Econ. 1, 54.
─ Stolberg, S. (2009). Enabling agile testing through continuous integration. Agile 2009 Conference.
─ Sturdevant, K. F. (2007). Cruisin' and chillin': testing the java-based distributed ground data system "chill" with cruisecontrol. IEEE Aerospace Conference, pp. 1-8. ─ Ståhl, D. and Bosch, J. (2013). Experienced benefits of continuous integration in industry software product development: A case study. The 12th IASTED Interna-tional Conference on Software Engineering, pp. 736–743.
─ Ståhl, D. and Bosch, J. (2014a). Automated software integration flows in industry: a multiple-case study. 36th International Conference on Software Engineering, ACM, pp. 54–63.
─ Ståhl, D. and Bosch J. (2014b). Modeling continuous integration practice differences in industry software development. Journal of Systems and Software, vol. 87, pp. 48– 59.
─ Ståhl D. and Bosch J. (2016a). Industry application of continuous integration mod-eling: a multiple-case study. 38th International Conference on Software Engineering Companion, ACM, pp. 270–279.
─ Ståhl D., Hallén K. and Bosch J. (2016b). Achieving traceability in large scale con-tinuous integration and delivery: deployment, usage and validation of the eiffel framework. Empirical Software Engineering, pp. 1–29.
─ Ståhl, D., Mårtensson, T. and Bosch, J. (2017a). Continuous practices and devops: beyond the buzz, what does it all mean?. 43rd Euromicro Conference on Software Engineering and Advanced Applications, SEAA 2017, pp. 440–448.
─ Ståhl, D., Mårtensson, T. and Bosch, J. (2017b). The continuity of continuous inte-gration: correlations and consequences. Journal of Systems and Software 127, pp. 150-167.
─ Sunindyo, W.D., Moser, T., Winkler, D. and Biffl, S. (2010). Foundations for event-based process analysis in heterogeneous software engineering environments. 36th Euromicro Conference on Software Engineering and Advanced Applications, SEAA 2010, pp. 313–322.
─ Su, T., Lyle, J., Atzeni, A., (...), Ntanos, C. and Botsikas, C. (2013). Continuous integration for web-based software infrastructures: Lessons learned on the webinos project. 9th International Haifa Verification Conference, HVC 2013, pp. 145-150. ─ Suranto, B. (2015). Exploratory software testing in agile project. 2nd International
Conference on Computer, Communications, and Control Technology, pp. 280-283. ─ Sviridova, T., Stakhova, D. and Marikutsa, U. (2013). Exploratory testing:
manage-ment solution. 12th International Conference: The Experience of Designing and Ap-plication of CAD Systems in Microelectronics, CADSM 2013, pp. 361.
─ Syer, M.D., Shang, W., Jiang, Z.M. and Hassan, A.E. (2016). Continuous validation of performance test workloads. Automated Software Engineering, pp. 1-43. ─ Søvik, H. and Forfang, M. (2010). Tech challenges in a large-scale agile project.
Lecture Notes in Business Information Processing, 48 LNBIP, pp. 353-361. ─ Tan, C.-H. and Teo, H.-H. (2007). Training future software developers to acquire
agile development skills. Communications of the ACM 50 (12), pp. 97-98.
─ Thangiah, M. and Basri, S. (2016). A preliminary analysis of various testing tech-niques in agile development - a systematic literature review. 3rd International Con-ference on Computer and Information Sciences, ICCOINS 2016, pp. 600-605. ─ Trimble, J. and Webster, C. (2013). From traditional to lean, to agile development:
finding the optimal software engineering cycle, 46th Hawaii International Confer-ence on System SciConfer-ences, HICSS, pp. 4826-3833.
─ Tsai, W.-T., Shao, Q., Huang, Y. and Bai, X. (2010). Towards a scalable and robust multi-tenancy SaaS. 2nd Asia-Pacific Symposium on Internetware.
─ Vassallo, C., Zampetti, F., Romano, D., (...), Di Penta, M. and Zaidman, A. (2016). Continuous delivery practices in a large financial organization. IEEE International Conference on Software Maintenance and Evolution, ICSME 2016, pp. 519-528. ─ Virmani, M. (2015). Understanding devops & bridging the gap from continuous
in-tegration to continuous delivery. Fifth International Conference on Innovative Com-puting Technology, pp. 78–82.
─ Vost, S. (2015). Vehicle level continuous integration in the automotive industry. 10th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering, pp. 1026-1029. ─ Vöst, S. and Wagner, S. (2016). Trace-based test selection to support continuous integration in the automotive industry. International Workshop on Continuous Soft-ware Evolution and Delivery, CSED 2016, pp. 34-40.
─ Wedawatta, G., Ingirige, B. and Amaratunga D. (2011). Case study as a research strategy: investigating extreme weather resilience of construction SMEs in the UK. ARCOM doctoral workshop, International conference on building resilience. ─ Wettinger, J., Andrikopoulos, V. and Leymann F. (2015a). Enabling devops
collab-oration and continuous delivery using diverse application environments. OTM Con-federated International Conferences On the Move to Meaningful Internet Systems, pp. 348-358.
─ Wettinger, J., Breitenbücher, U. and Leymann, F. (2014a). Compensation-based vs. convergent deployment automation for services operated in the cloud. International Conference on Service-Oriented Computing, pp. 336-350.
─ Wettinger, J., Breitenbücher, U. and Leymann, F. (2014b). Devopslang – bridging the gap between development and operations. European Conference on Service-Ori-ented and Cloud Computing, pp 108-122.
─ Wettinger, J., Breitenbücher, U. and Leymann, F. (2015b). Dyn tail-dynamically tai-lored deployment engines for cloud applications. 8th IEEE International Conference on Cloud Computing, pp. 421-428.
─ Wettinger, J., Breitenbücher, U., Falkenthal, M. and Leymann, F. (2017). Collabo-rative gathering and continuous delivery of devops solutions through repositories, Computer Science - Research and Development, 32(3-4), pp. 281-290
─ Whittaker, J. (2010). Exploratory software testing. Addison Wesley.
─ Willis, J. (2010). What devops means to me. https://blog.chef.io/2010/07/16/what-devops-means-to-me/ (accessed January 10, 2017).
─ Woehrle, M., Lampka, K. and Thiele, L. (2009). Exploiting timed automata for con-formance testing of power measurements. Lecture Notes in Computer Science, pp. 275-290.
─ Woskowski, C. (2012). Applying industrial-strength testing techniques to critical care medical equipment. Computer Safety, Reliability, and Security, pp. 62-73. ─ Wright, S.A., and Druta, D. (2014). Open source and standards: the role of open
source in the dialogue between research and standardization. Globecom Workshops, GC Wkshps, pp. 650–655.
─ Xu, X., Zhu, L., Weber, I., Bass, L. and Sun, D. (2014). Pod-diagnosis: error diag-nosis of sporadic operations on cloud applications. 44th Annual IEEE/IFIP Interna-tional Conference on Dependable Systems and Networks, pp. 252-263.
─ Yüksel, H.M., Tüzün, E., Gelirli, E., Biyikli, E. and Baykal, B. (2009). Using con-tinuous integration and automated test techniques for a robust c4isr system. 24th International Symposium on Computer and Information Sciences, pp. 743-748.
─ Zampetti, F., Scalabrino, S., Oliveto, R., Canfora, G. and Di Penta, M. (2017). How open source projects use static code analysis tools in continuous integration pipe-lines. IEEE International Working Conference on Mining Software Repositories, pp. 334-344.
─ Zhu, L., Bass, L. and Champlin-Scharff, G. (2016). Devops and its practices. IEEE Software, vol. 33, no. 3, pp. 32-34.
─ Zhu, L., Xu, D., Tran, A.B., (...), Weber, I. and Dwarakanathan, S. (2015). Achieving reliable high-frequency releases in cloud environments. IEEE Software, vol. 32, no. 2, pp. 73-80.
