VU Research Portal

VU Research Portal

Safety for mobile robotic system

Bozhinoski, Darko; Di Ruscio, Davide; Malavolta, Ivano; Pelliccione, Patrizio; Crnkovic,

Ivica

published in

Journal of Systems and Software

2019

DOI (link to publisher)

10.1016/j.jss.2019.02.021

document version

Publisher's PDF, also known as Version of record

document license

Article 25fa Dutch Copyright Act

Link to publication in VU Research Portal

citation for published version (APA)

Bozhinoski, D., Di Ruscio, D., Malavolta, I., Pelliccione, P., & Crnkovic, I. (2019). Safety for mobile robotic

system: A systematic mapping study from a software engineering perspective. Journal of Systems and Software,

151, 150-179. https://doi.org/10.1016/j.jss.2019.02.021

General rights

Copyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights. • Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain

• You may freely distribute the URL identifying the publication in the public portal ?

Take down policy

If you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediately and investigate your claim.

E-mail address:

vuresearchportal.ub@vu.nl

software

engineering

perspective

Darko

Bozhinoski

_,

_Davide

_Di

_Ruscio

_,

_Ivano

_Malavolta

_,

_Patrizio

_Pelliccione

_,

Ivica

Crnkovic

a IRIDIA, Université Libre de Bruxelles, Belgium b University of L’Aquila, L’Aquila, Italy

c Faculty of Sciences, Vrije Universiteit Amsterdam, De Boelelaan 1081a, Amsterdam, 1081 HV, the Netherlands d Department of Computer Science and Engineering, Chalmers University of Technology, Sweden

e Department of Computer Science and Engineering, University of Gothenburg, Sweden

a

r

t

i

c

l

e

i

n

f

o

Article history:

Received 18 September 2017 Revised 5 February 2019 Accepted 8 February 2019 Available online 11 February 2019

Keywords:

Software

Safety for mobile robots Systematic mapping study

a

b

s

t

r

a

c

t

Robotic research is making huge progress. However, existing solutions are facing a number of challenges preventing them from being used in our everyday tasks: (i) robots operate in unknown environments, (ii) robots collaborate with each other and even with humans, and (iii) robots shall never injure peo- ple or create damages. Researchers are targeting those challenges from various perspectives, producing a fragmented research landscape.

We aim at providing a comprehensive and replicable picture of the state of the art from a software engineering perspective on existing solutions aiming at managing safety for mobile robotic systems. We apply the systematic mapping methodology on an initial set of 1274 potentially relevant research papers, we selected 58 primary studies and analyzed them according to a systematically-defined classification framework.

This work contributes with (i) a classificationframework for methods or techniques for managing safety when dealing with the software of mobile robotic systems (MSRs), (ii) a map of current software methods or techniques for software safety for MRSs, (iii) an elaboration on emergingchallengesand im-plications for future research, and (iv) a replicationpackage for independent replication and verification of this study. Our results confirm that generally existing solutions are not yet ready to be used in everyday life. There is the need of turn-key solutions ready to deal with all the challenges mentioned above.

© 2019 Elsevier Inc. All rights reserved.

1. Introduction

Robots are increasingly used in industry butalso for tasks of oureverydaylife.Inarecentbook,RiseoftheRobots(Ford, 2016 ), MartinForddiscussesthe transitioninrobotics fromspecial pur-pose robots, built to operate in highly controlled environments on a specific task, to general purpose robots that can operate in a heterogeneous environment, intermixed with humans, and perform a broad spectrum of tasks. Smart robots equipped with sensors and intelligent software promise to bring a new indus-trial revolution. According to Industrie 4.0 (Kagermann et al., 2013 ), we are in the middle of the 4th industrial revolution that is based on autonomous and smart Cyber Physical Systems

∗ _{Corresponding author.}

E-mail addresses: darko.bozhinoski@ulb.ac.be (D. Bozhinoski),

davide.diruscio@univaq.it (D. Di Ruscio), i.malavolta@vu.nl (I. Malavolta),

patrizio.pelliccione@gu.se (P. Pelliccione), ivica@chalmers.se (I. Crnkovic).

(CPSs)(Kagermann et al., 2013 ),abletocooperatewitheachother andhumansinasafe,autonomous,andreliablemanner.The mar-ket forindustrialrobotics isexpectedto riseata Compound An-nual GrowthRate (CAGR) of 11,5%annually through 2021 andto reach $48.9 billion by 2021 (UAV, 2018 ). The total smart robots market is expected to reach USD 7.85 billion by 2020, at an es-timatedCAGRof19.22%between2015and2020.

In this paper we focus on Mobile Robotic Systems (MRSs). This class of robots opens new long-term ambitions and busi-ness opportunities.Commercial drone revenue in Europein 2017 wasaround $188million,almostdoubletheamountthanin2015 which was around $98 million (Dro, 2018 ). Moreover, the total globalUnmannedAerialVehicle(UAV)marketisexpectedtogrow from$20.71billion in2018to reach$52.30billionby 2025(UAV, 2016; 2018 ).Inanearfuture,therewillbe theneedof customer-specific MRS solutions fora specific domain, such as: Homeland Security (e.g. coastal surveillance), Environmental Protection (e.g.

https://doi.org/10.1016/j.jss.2019.02.021

emissionmonitoringandcontrol),ProtectionofCritical Infrastruc-ture(e.g.monitoringwaterandgaspipelines).

However, MRSs pose also important challenges: they need to be able tooperate inuncontrollable andunknown environments, which are often shared with humans, and often they will be requiredto collaborateeach other,andevenwithhumans, to ac-complishcomplexmissions.Becauseofthesechallenges,these sys-tems are both safety and mission critical. Safety criticality is an aspect of MRSs wherefailure or malfunction of the system may cause injure to people or severe damage to equipment/property, while mission criticality is another aspect of MRSs where a fail-ure ormalfunction maylead to an unacceptable loss of mission goals. Although robotic research has made huge progress in the lastdecades,theaforementionedfunctionalitiesandexisting solu-tionsseemtobenot-yet-readytobeusedineverydaylife,andin uncontrollable andunknown environments oftensharedwith hu-mans(Mitka et al., 2012 ),whichwillbeshownaspartofthe con-clusionofthisstudy.

The goal ofthisstudyisto identify, classify, andevaluatethe state of the arton safety forMRSs in termsof technical charac-teristics,potentialforindustrialadoption,andtheirchallengesand implicationsforfutureresearchonsafetyforMRSs.Thestudy ex-clusivelyfocusesonsoftwareaspects.

In order to target our goal, we apply a well-established methodologyfromthemedicalandSoftwareEngineeringresearch communities called systematic mapping (Petersen et al., 2015; Kitchenham and Charters, 2007 ). The aim of a systematic map-pingstudyistoprovidean unbiased,objectiveandsystematic ap-proach to answer a set of research questions aboutthe state of the artandresearchgaps ona giventopic. Amappingstudy fol-lowsawell-definedandreplicableprincipledprocess forboththe search and selection of relevant studies, and the collected data andresultssynthesistendtobemorequantitativeandqualitative

(Wohlin et al., 2012, Section 4.4) .Throughoursystematicmapping process, we selected 58 primary studies among 1274 potentially relevantstudiesfittingatbestthreeresearchquestionswe identi-fied(seeSection 3.1 ).Then,wedefinedaclassificationframework composed of more than 50 different parameters for comparing state-of-the-art approaches, andwe applied it to the 58selected studies. Finally, we analysed anddiscussed the obtaineddata for each parameterofthe classificationframework andhowit fitsin theresearchlandscapeaboutsafetyforMRSs.

Themaincontributionsofthisstudyare:

• areusablecomparisonframeworkforunderstanding,classifying, andcomparingmethodsortechniquesforsafetyforMRSs;

• asystematicreviewofcurrentmethodsortechniquesforsafety forMRSs,usefulforbothresearchersandpractitioners;

• adiscussionofemergingresearchchallengesandimplicationsfor futureresearchonsafetyforMRSs;

• areplicationpackagecontainingdetailedreports,rawdata,and analysisscriptsforenablingindependentreplicationand verifi-cationofthisstudy.

Tothebestofourknowledge,thispaperpresentsthefirst sys-tematicinvestigation intothestate oftheartonsafetyforMRSs. The results ofthis studyprovide a complete,comprehensive and replicable pictureofthe state oftheartofresearch on safetyfor MRSs,helpingresearchersandpractitioners infinding characteris-tics, limitations, andchallenges of currentresearch on safety for mobile roboticsystems.Themain emergingchallengesand impli-cationsforfutureresearchonsafetyforMRSsareshowninTable 1 .

Article outline. The article is organized as follows. In

Section 2 we provide background notions forsetting the context of our study by clarifying and discussing (i) mobile robotic sys-tems,(ii)safetyformobileroboticsystems,and(iii)existing stud-ies onsafetyforMRSs.Section 3 describesindetails theresearch

methodology we followed for designing, conducting, and docu-menting the study.Data demographics is presentedin Section 4 , followedby adescriptionof theobtainedresultsinSections 5 –7 . WepresentlimitationsandthreatstovalidityinSection 8 .Related works are discussed in Section 9 ,whereas Section 10 closes the articlewithfinalremarks.

2. Background

2.1. Mobileroboticsystems

Robotshavebeensuccessfullydeployedinindustrytoimprove productivity and perform dangerous, tedious, or repetitive tasks (Siciliano and Khatib, 2008 ). In the literature, a variety of def-initions exists defining the term “robot” (Robots, 2014; Oxford dictionary, 2014; Harris, 2014 ). All of them share the following concept:arobotisanintelligentdevicewithacertaindegreeof au-tonomy that contains sensors, control systems, manipulators, power supplies and software all working together to perform the required tasks.Underthisperspective,amobile robotrepresentsarobotic systemconsisting of a SW/HW platform carried around by loco-motive elementsandable to perform tasks in differentcontexts. Thekindoflocomotionthattherobotisabletoperformis primar-ilydecidedupontheenvironment(aquatic,aerialorterrestrial)in which the robot will be operating (Garcia et al., 2007 ). Mobility givesrobotsenhancedoperativecapabilities,butatthesametime increasescomplexityandbringsadditionalsafetychallenges.

Inordertoreducethehumaninvolvementinscenariosthatare characterizedbyrepetitiveanddangeroustasks(eg.natural catas-trophes,nuclearpowerplantdecommissioning,extra-planetary ex-ploration, or less dangerous activities, such as delivery services, surveillance,andenvironmentalmonitoring), innovative technolo-gies and approachesrepresented by mobile robotics are seen as particularly suitable for aiding in the process of replacement of thehuman beingswithrobotic systems.Thatwilllead to a soci-ety where mobile robotswill operate in a dynamicenvironment and perform the necessary tasks in these scenarios. But, if we want mobile robots to be widely accepted and adopted among the general public, it is fundamental to carefully consider safety aspects.

2.2.SafetyforMRSs

C5) Research community on software engineering and robotics : even though there is a growing interest, the community of software engineering for robotic systems is still not consolidated.

I5) The challenge for the research community is to promote a shift towards well-defined engineering approaches able to stimulate component supply-chains and significantly impact the robotics marketplace.

aspects(e.g.softwareengineering,controltheory,mechatronics),if themajorcontributionistowards softwareengineeringprinciples andpractises, we become inclusive andwe are considering it in ourstudy.Thiswaywe positionourpaperto helpresearchers in identifyingdesigntoolsandmethodologiesforsoftwareformobile robotsthatfollowsafetystandards.

To address the increasing complexity and the needs of the variegated nuances of mobile robots, the robotics and automa-tion industry are working towards the establishment of new international safety standards through the International Organi-zation for Standardization (ISO) for robots and robot systems integration(Safety Standards, 2014 ). The current developed stan-dardsvarymuchastheydependontheparticularapplication do-mains where the considered robotic systems are employed. The domainof personal care andagriculture is expandingrapidly. As a result, the ISO13482 standard for Safety requirements for per-sonal care robots and the ISO18497 standard for Safety of highly automated agricultural machines have been developed. Another reallyimportantstandardisISO15066,whichfocusesonthe col-laboration betweenpeople androbots.It specifies safety require-mentsforcollaborative industrialrobotsystemsandsupplements the requirements and guidance on collaborative industrial robot operation. Commercialisation and adoption of mobile robots in dynamic environments will only occur if the safety aspects are consideredand incorporatedasfirst classelementsin thedesign ofthe system. Establishingtheguidelines andstandardsto regu-late a safeuse of these innovative technologies is the means to increasetheir trustworthiness andthereby their appreciation and use,notonlyintheresearchandbusinesssectors,butalsointhe privatesocialsphere.Certificationbodiesshouldassuresafety cer-tification that relies on a complete understanding of the system. However,formobilerobotsthat operateindynamicenvironments itis quite challenging to consider all variants of theoverall sys-temdueto their adaptivebehaviour (Skrzypietz, 2012 ). Recently, researchershaveputtheirfocus onthepotential forusingrobots toaidhumansoutsidestrictlyindustrialenvironments,inmore un-structured and dynamic ones (Ogorodnikova, 2009 ). The authors ofNakabo et al. (2009) developedasafetymodulethat integrates safety functions required for robots to work side by side with humans; it is compliant with international safety standards and Japanese law. It is strongly recommended to revise safety prop-ertiesforMRSs inother application domains that willcomplyto identifiedinternationalsafetystandards.

Finally,asoftodaywedidnotfindanyevidencethatcouldhelp usinassessingtheimpactofexistingresearch onsafetyinmobile robots. With this studywe aim athelping researchers and prac-titionersin identifyingthecharacteristics, challenges,andgaps of currentresearchonthistopic,itsfuturepotential,andits applica-bilityinpracticeinthecontextofreal-worldroboticprojects.

3. Studydesign

Fig. 1 showstheoverviewoftheprocesswefollowedfor carry-ing out this study. The overall process can be divided into three main phases, which are the classical ones for systematic map-pingstudies(Kitchenham and Charters, 2007; Wohlin et al., 2012 ): planning,conducting, and documenting. Inthe following we will gothrougheachphaseoftheprocess,highlightingitsmain activi-tiesandproducedartifacts.

Planning. It is the first phase ofour study andit aims at (i) establishing the need forperforming a mapping study on safety forMRSs; indeed,asdiscussedalsoin Section 9 , secondary stud-ies exist on topics relatedto robotics safety like mechanicaland controller design (Tadele et al., 2014 ) and human-robot interac-tion (Goodrich and Schultz, 2007; Vasic and Billard, 2013; Alami et al., 2006 ),butnoneofthemtakesintoconsiderationsafetyfrom asoftwareengineeringpointofview;(ii)identifyingthemain re-search questions (see Section 3.1 ); and (iii) defining the review protocoldetailing eachstepofthewholestudy.Theoutputofthe planningphaseisawell-definedreviewprotocol.Inorderto miti-gatepotentialthreatstovalidity,ourreviewprotocolhasbeen cir-culatedtoexternalexpertsforindependentreviewandwerefined itaccordingtotheirfeedback.1

Conducting.Inthisphasewecarriedouteachstepoftheabove mentioned review protocol. More specifically, we performed the followingactivities:

• Conduct search: in this activity we applied a search string to well-known academic search databases (see Section 3.2 ). The outputofthisactivityisacomprehensivelistofallthe candi-datestudiesresultingfromthesearch.

• Screeningofallstudies:candidateentrieshasbeenfilteredin or-dertoobtainthefinal listofprimary studiestobe considered inlateractivitiesofthestudy.Thebasisfortheselectionof pri-marystudiesistheinclusionandexclusioncriteriadescribedin

Section 3.2 .

• Classification framework definition: we created a classification frameworkto comparethe selectedprimary studies. The clas-sificationframework hasbeendesignedtocollect datafor an-swering the research questions of this study (Wohlin et al., 2012 ) andincludes categoriessuch asthe level ofabstraction inwhichsafetyismanaged,compliancetostandards,thescope andcardinalityofhazards,etc.Thisactivitywillbedescribedin moredetailsinSection 3.3 .

1 We thank Richard Torkar (University of Gothenburh, Sweden) and Wasif Afzal

Fig. 1. Overview of the whole mapping process.

Table 2

Goal of this research. Purpose Analyse

Issue The characteristics and potential for industrial adoption

Object Of existing approaches for safety for MRSs

Viewpoint From a researcher’s and practitioner’s point of view.

• Dataextraction:inthisactivityweanalysedeachprimarystudy, andwe filled the data extractionform withthe extracted in-formation.Filled formshave beencollectedandaggregatedin ordertobereadytobeanalyzedduringthenextactivity.More detailsaboutthisactivitywillbepresentedinSection 3.4 .

• Datasynthesis:thisactivityfocussedonacomprehensive sum-maryandanalysisofthedataextractedinthepreviousactivity. The maingoalof thisactivityisto elaborate onthe extracted datainordertoaddresseachresearchquestionofourresearch. ThedetailsaboutthisactivityareinSection 3.5 .

Documenting.Themainactivitiesperformedinthisphase con-sistof(i)athoroughelaborationonthedataextractedinthe pre-vious phase withthemain aimofsetting theobtainedresultsin their context, (ii) theanalysisof possiblethreats tovalidity, spe-ciallytheonesidentifiedduringthedefinitionofthereview proto-col(in thisactivityalsonewthreatstovalidity mayemerge), and (iii) the writingof a final report describing in details the design andresultsofthisresearch.

3.1. Goalandresearchquestions

We formulate the goal of this research by using the Goal-Question-Metric perspectives (i.e., purpose, issue, object, view-point (Basili et al., 1994 )).Table 2 showsthe resultofthe above mentionedformulation.

The goal presented above can be refined into the following mainresearchquestions.

-RQ1:HowdoexistingapproachesaddresssafetyforMRSs? Ob-jective:toidentifyandclassifyexistingapproachesforsafety

inMRSs in order to build (i) a solid foundation for classi-fyingexisting(andfuture) research onsafetyforMRSsand (ii)anunderstandingofcurrentresearchgapsinthefieldof safetyforMRSs.

- RQ2: What is the potential for industrial adoption of existing approachesforsafetyforMRSs?Objective:toassess howand ifthe current state of the art on safetyfor MRSs is ready tobetransferredandadoptedinindustry.Hereweconsider criteriasuchastherigorandprecisionoftheapplied valida-tionstrategies (e.g.,in-the-labexperiment, industrial appli-cation),therealismandscaleoftheperformedevaluations, etc.

-RQ3:Whatarethemainemergingchallengesforfutureresearch on safetyfor mobilerobotics systems? Objective:to put into contexttheresults ofRQ1andRQ2 inorderto identify the main challengeswhich will be faced by future researchers onsafetyforMRSs.

Answeringthose research questions willprovide a solid foun-dation forunderstanding thestate of theart onsafety forMRSs, together withits research gaps andfuture challenges. The above listed research questions will drive the whole systematic review methodology, with a special influence on the primary studies search process,the dataextractionprocess, andthe dataanalysis process.

3.2.Searchandselection

Fig. 2. The search and selection process of this research.

give abrief descriptionofeach stage of oursearch andselection process.

Stage 1. In this stage we performed automatic searches on electronic databases. In order to cover as much as possible rel-evant literature, four of the largest and most complete scientific databases were chosen as the sources of primary studies of this stage, namely: IEEE Xplore Digital Library, ACM Digital Library, SpringerLink, andScienceDirect. The selection ofthese electronic databases is guided by (i) their high accessibility, (ii) their abil-itytoexportsearchresultstowell-defined,computation-amenable formats,and(iii)because they havebeenrecognized asbeingan efficient means to conduct systematic literature reviews in soft-wareengineering(Brereton et al., 2007; Dyba et al., 2007 ).

Tocreatethesearchstring,we breakdown ourresearch ques-tions intoindividual facets (population, intervention, comparison, outcomes, context - PICOC) asdiscussed in Keele (2007) . In our study,thePICOCelementsthatweidentifiedareasfollows:

• Population:mobileroboticsystems;

• Intervention: approachesthataddresssafetyinmobile robotic systems;

• Comparison:notapplicable;

• Outcomes: the classification framework populated with the identifiedprimarystudies;

• Context:academic peer-reviewedpublicationswithasoftware engineeringperspective.

Then we drawup alistof synonyms,abbreviations, and alter-native spellings, which combined by logical ANDs and ORs gave thesearch string. Moreover, itis importantto highlight that this studyfocusesonsoftwareaspects.Thisdoesnotmeanthatsafety inroboticsis onlya softwareaspect, butthisisthefocus ofthis studyandthe focusdefinestheboundary ofthestudyitself.The obtainedsearch string is given below andit has been tested by executingpilotsearchesonIEEEXploreDigitalLibrary.

(mobile OR groundORwater OR fly∗ ORsail∗ OR unmanned

ORselfORautonomous)AND(robot∗ ORvehicle∗)AND(safe∗

ORfaultORfailure)ANDsoftware

Forthesakeofconsistency,thesearchstringshasbeenapplied toanidenticalsetofmetadatavalues(i.e.,title,abstractand

key-words)fromallelectronicdatabases.Thisstageresultedinatotal numberof1274potentiallyrelevantstudies.

Stage2. Themain goal ofthisstage is toconsider all the se-lected studies andfilter them according to a set of well-defined inclusion andexclusion criteria. As suggestedin Kitchenham and Charters (2007) ,wedecidedtheselectioncriteriaofthisstudy dur-ingitsprotocoldefinition,sotoreducethelikelihoodofbias.Inthe followingweprovideinclusionandexclusioncriteriaofourstudy. Inthiscontext,astudywillbeselectedasaprimarystudyifitwill satisfy allinclusioncriteria,anditwillbe discardedifitwillmet

anyexclusioncriterion.

I1) StudiesproposinganapproachforsafetyforanMRS.2

I2) Studiesfocussingon safetyinMRSsfromasoftware engineer-ingperspective(e.g.,nocontroltheoryormechatronicsstudies, nostudiesfocussingonhardware,etc.).

I3) Studies providing some kind of evaluation of the proposed methodology(e.g., via a casestudy, a survey,experiment, ex-ploitationinindustry,formalanalysis,exampleusage). I4) Studiessubjecttopeerreview(Wohlin et al., 2012 )(e.g.,journal

papers,paperspublishedaspartofconferenceproceedingswill beconsidered,whereaswhitepaperswillbediscarded). I5) StudieswritteninEnglishlanguageandavailableinfull-text. E1) Studiesexclusively focussingonsafetyfor industrialandother

immobilerobots.

E2) Secondary studies (e.g., systematic literature reviews, sur-veys)(Wohlin et al., 2012 ).

E3) Studiesintheformoftutorialpapers,shortpapers,poster pa-pers, editorials,becausethey do not provideenough informa-tion.

Inordertoreducebias,theselectioncriteriaofthisstudyhave beendecidedduringthereviewprotocoldefinition(meaningthat theyhavebeencheckedbythetwoexternalreviewers).

Inthisstage,eachpotentiallyrelevantstudyhasbeenanalysed inthreephases. Firstlyithasbeenanalysed byconsidering its ti-tle,keywords,andabstract;secondly,iftheanalysisdidnotresult inacleardecision,alsoitsintroductionandconclusionshavebeen

2 In the context of this research an approach can be considered as an organized

analysed;finally,weperformedacomprehensivethirdmanualstep inwhichwe read thefulltext ofallconsidered studies(title, ab-stract, keywords, all sections and appendices, ifany) in order to take the final decision about its inclusion in our set of primary studies. Tworesearchershave beeninvolved duringthose phases andathirdresearcherhasbeeninvolvedinordertosolveconflicts andtakeconvergetowardsthefinaldecisions,whileavoidendless discussions(Zhang and Babar, 2013 ).

In thisstage,it isfundamentalto selectpapersobjectively.To thisend,assuggestedbyWohlin et al. (2012) ,tworesearchers in-dependentlyassessed arandomsampleofstudies,then the inter-researcheragreement hasbeenmeasured usingthe CohenKappa statistic; we obtained a Cohen Kappa statistic of 0.80, which is a goodindicationoftheobjectivenessoftheperformedselection. Thisstageresultedinatotalnumberof51relevantstudies.

Stage3.Inthisstageallstudiesfromthefirststagehavebeen combinedtogether intoasingleset.Duplicatedentrieshavebeen identified and merged by matching them by

title

authors

year

publication

Stage4.Asrecommended inguidelinesforsystematicstudies, we extendedthecoverageoftheprevious stagesby complement-ing thepreviouslydescribedautomatic searchwithasnowballing activity.Themaingoalofthisstageistoenlargethesetofrelevant studies byconsideringeach studyselectedintheprevious stages, andfocussingonthosepaperscitedbyit.Moretechnically,we per-formedaclosedrecursivebackwardandforwardsnowballingactivity (Wohlin, 2014 ). From a practical point ofview,we went through each selectedstudyandweincludedalso therelevantstudies ei-thercitedbyorcitingit(basedonGoogleScholar(Wohlin, 2014 )). The start set for the snowballing activity was composed of the 35studiesselectedinstage 3.Then, weconsideredeach paperin the start setand appliedthe same selectioncriteria discussedin stage 2 to each papereither citedby or citing it.If apaper was included,snowballingwasappliediterativelyuntilnonewpapers havebeenfound.Duplicateswereremovedateachiterationofthe snowballingactivity.

Thisstage largelyincreasedthenumberofpotentiallyrelevant studies, bringing it to 61. As a possible explanation of this fact, we noticed that researchersused avery heterogeneous terminol-ogy whenwritingthe title, abstract,andkeywords oftheir stud-ies; this fact maynegatively impactour automatic search, which may havemissed some potentially relevantstudies. We included thesnowballingactivityinordertomitigatethispotentialthreatto validity.Asafurtherconfirmation,thestudyreportedinJalali and Wohlin (2012) empiricallyobservedthatsimilarpatternsand con-clusions are identified when using automatic search and snow-balling,especiallywhentheyareusedincombination.

Stage5.Thisstagehasbeenperformedinparallelwiththedata extractionactivity.Basically,theideaisthatwhenreadingastudy in details forextracting its information, researchers could recog-nize that it wasout of scope, and so it hasbeen excluded. This stage led usto thefinalizedset of58 primary studies ofour re-search,whichiscomprisedof58entries.

3.3. Classificationframeworkdefinition

One of the main contributions in our study is the classifica-tion framework, which consists ofparameters that we identified as part of the protocol. We consider that thesenewly identified parameterscanbereusedinfuturestudiestohelpauthorsofnew methodsandtechniquestocomparetheircontributiontoexisting ones. The different categories ofour classification framework are describedinmoredetailsinthefollowingsubsections.The classifi-cationframeworkiscomposedofthreefacets,eachonededicated to one ofthe RQ1 and RQ2research questions (see Section 3.1 ).

RQ3 doesnot havea dedicated facet in the classification frame-worksinceitisorthogonaltoRQ1andRQ2anditaimsatputting theirresultsinthecontextoffutureemergingchallengesonsafety for MRSs. The classification framework also contains publication metadata(e.g.,publicationvenues,authors,etc.), whichhavebeen collectedfordemographicspurposes(seeSection 4 ).

3.3.1. HowsafetyforMRSsismanaged(RQ1)

SinceresearchquestionRQ1isatthecoreofourresearch, the creationofitscorresponding facet intheclassificationframework demandsadetailedanalysisofthecontentsofeachprimarystudy. Inlight ofthis,we followeda systematicprocess called keyword-ing(Petersen et al., 2008 ) forbuildingthisfacet ofour classifica-tionframework.Keywordingaims atreducing thetimeneededin developingaclassificationframeworkandensuresthatittakesthe consideredstudiesintoaccount(Petersen et al., 2008 ).

AsshowninFig. 3 ,keywordingisdoneintwosteps:

1.Collectkeywordsandconcepts:wecollectedkeywords and con-cepts by reading the abstract of each primary study. When all primarystudies havebeenanalysed, allkeywordsand con-ceptshavebeencombinedtogethertoclearlyidentifythe con-text, nature,andcontributionoftheapproach.Assuggestedin

Petersen et al. (2008) , when the abstract of a primary study was not informative enough, then we analysed also its intro-duction and conclusion sections.Considering that the authors of theprimary studies mayusedifferentterms forsame con-ceptsandsame termsfordifferentconcepts, inthisphasewe keptallkeywordsandconceptstoensureconsistencyand com-patibility.Theoutputofthisstageisthesetofkeywordsasthey havebeenusedineachprimarystudy.

2.Clusterkeywordsandformcategories:whenkeywordsand con-cepts have been collected, then we performed a clustering operation on them in order to have a set of representative clustersofkeywords.Weidentifiedtheclustersbyapplyingthe open cardsortingtechnique(Spencer, 2009 )tocategorize key-wordsintorelevantgroups.Morespecifically,weconsideredall thekeywordsandconceptscollectedinthepreviousphaseand iteratively groupedthem together until asaturation of all the concepts hasbeenachievedandall primarystudies were ana-lyzed. Inordertominimize bias,thisstephasbeenperformed by two researchers andthe resultshave beendouble-checked by the other two researchers.The output of thisstage is the classification framework containingall the identified clusters, each ofthemrepresentingaspecificaspectofsafetyforMRSs. The specific categoriesemergingfromthe keywordingprocess aredescribedinSection 5 .

Moreover, we collected alsodata related to the main research contribution and application field independence of each primary study.The categories forresearch contributions are derived from

Petersen et al. (2008) and include valuessuch as“method”, “ar-chitecture”, “tool”; they are discussed in details in Section 5.1 . Forwhat concerns applicationfield independence, while piloting this study we noticed that in the discussion of related work of some papers authors were referring to both domain-specific ap-proaches and generic ones; based on this, we decided to cate-gorize our primary studies about whether they are independent withrespect toanyapplicationfield (e.g.,abstractapproaches or-thogonaltoanyapplicationfield)ornot(e.g.,approachesthatare specificallytailoredtoself-driving cars,agriculture,environmental monitoring).

Sincethisresearchquestionisofkeyimportanceforthissurvey, wemadeapre-studyinordertoclassifyexistingworksonsafety mechanisms.The pre-studyconsistsinanalysingthreerecent sur-veys on MRS safety from 2017, namely Guiochet et al. (2017) ,

Fig. 3. Overview of the keywording process.

Table 3

Classification parameters proposed by other secondary studies.

Survey Parameters Description

A Survey of Methods for Safe Human-Robot Interaction ( Lasota et al., 2017 )

Reactive Safety If it is reactive (not performing any planning)

Proactive Safety If it is proactive (producing plans to address specific safety-related issues)

Proactive Safety with prediction If it can anticipate the actions and movements of the rest of the team of mobile robots or people

Psychological safety If it takes consideration of psychological factors Safety-critical advanced robots:

A survey ( Guiochet et al., 2017 )

Fault prevention If it prevents the occurrence or introduction of faults, including techniques coming from system engineering and good practices from system designing

Fault removal If it reduces the number and severity of faults

Fault forecasting If it estimates the present number, the future incidence, and the likely consequences of faults.

Fault tolerance If it avoids service failures in the presence of faults using redundancy, error detections Robot Collisions: A Survey on

Detection,Isolation, and Identification ( Haddadin et al., 2017 )

Precollision If it discusses collision avoidance strategy

Detection If it has ability to understand if a system collision occurred Isolation If it understands the impact of the collision

Identification If it understands the impact of the collision

Classification If it has capability to understand the nature of the collision

Reaction If it provides strategies for the system to react purposefully to a collision event Post-collision If it discusses strategies how the robot will proceed after a safe state has been reached

theparameters they haveused intheir classification schemaand weusedonour primarystudies.Foreach oftheprimary studies, we collectedin a spreadsheet a record for each parameter. Each cellintherecordrepresentsabooleanvaluethatgiveinformation iftheprimary studyisaddressing aparticularaspect represented bytheparameterextractedfromthesurveys.

AllthreesurveysaresecondarystudiesthataddressMRSsafety from different domain, having different perspective and conclu-sions. Lasota et al. (2017) focuses on classification schema for methodsforsafehuman-robotinteraction,Guiochet et al. (2017) is a survey on dependability techniques used for increasing safety in MRS addressing large scope of application domains and

Haddadin et al. (2017) reviews and evaluates model-based al-gorithms forreal-time collision detection, isolation, and identifi-cation focusing on control strategies for safe robot reaction. As we see all the surveys address safety from a different perspec-tive.We extractedall the parametersthey have usedin all three surveys and we used this classification schema on our primary studies.Foreachoftheprimarystudies,wecollectedina spread-sheeta record foreach parameter. Eachcell in therecord repre-sentsa booleanvalue thatgivesinformationiftheprimary study isaddressingaparticularaspectrepresentedbytheparameter ex-tracted fromthe surveys. All parameters have been described in

Table 3 .

3.3.2. Potentialforindustrialadoption(RQ2)

Toanswer thisresearchquestionwe performedan analysisof qualitativedata.Toperformtheanalysiswe usedthealready pre-sented keywording method, and then we analysed and summa-rized the potentials forindustrial adoption that have been high-lightedinthepapers.Theparametersthatweconsideredare:

• applied research method: here we distinguished between ap-proaches validated in a controlled setting (or in the lab) and approachesevaluatedinreal-world(industrial)contexts;

• validation/evaluationstrategies:hereweextractedthestrategies applied for assessing the proposed approaches (e.g., real de-ployment, simulation-based, proof of concept), independently ofwhethertheyare performedinthecontext ofvalidationor evaluationresearch;

• technology readiness level (TRL): it has been proposed by the Horizon 2020 European Commission for the 2014/2015 work program3_{, the TRL is a metric for measuring the maturity of}

agiventechnology;

• rigor and industrial relevance: we measured the precision, ex-actnessandrealismoftheevaluationofeachprimarystudyby

3_{http://ec.europa.eu/research/participants/data/ref/h2020/wp/2014 _ 2015/}

applyingtherigorandindustrialrelevancemetricsproposedby

Ivarsson and Gorschek (2011) ;

• industryinvolvement:whethereachprimarystudyhasbeen car-riedout onlyby academics,practitioners (ora mixtherof) for understandinghowresearchersandpractitionerscollaborateon safetyforMRSs.

3.3.3. EmergingchallengesforFutureResearch(RQ3)

Toanswerthisresearchquestionwefollowedasimilarstrategy totheoneusedforRQ2.Webasicallyanalyzedalltheprimary pa-perswith theaimof collecting all thechallengesthat havebeen highlighted in such papers, andthen we summarized theresults thatemerged.

3.4. Dataextraction

Asalreadysaid,theclassificationframework isthebaseofthe dataextractionform, i.e.,awell-structuredformtostorethedata extractedfromeach primary study.Foreach ofthesestudies, we collectedinaspreadsheetarecordwiththeextractedinformation forsubsequentanalysis.As suggestedinWohlin et al. (2012) ,the data extractionform (andthus also the classificationframework) hasbeenindependentlypilotedonasampleofprimarystudiesby tworesearchers,anditeratively refinedaccordingly.Oncethedata extractionformwassetup,weconsideredeachprimarystudyand itscorrespondingdataextractionformhasbeenfilledwiththe ex-tracteddata.

In order to validate our data extraction strategy, 10 primary studieshavebeenrandomlyselectedandtworesearcherschecked whether the results were consistent, independently from the re-searcherperformingtheextraction.Inthiscontext,each disagree-menthasbeendiscussedandresolved,withtheinterventionofa thirdresearcher,whennecessary.

3.5. Datasynthesis

This activity involved collating andsummarising the data ex-tractedfromtheprimary studies(Kitchenham and Charters, 2007, Section 6.5) withthemaingoalofproducingtheactualmapof cur-rentresearch on safetyforMRSs. Whenpossible, inthisresearch weappliedbothquantitativeandqualitativeanalysismethods, de-pendingonthenatureofeachspecificparameterofthe classifica-tionframework.

Foreachparameter ofthe classificationframeworkwe divided our quantitative analysis on two main steps: (i) we counted the numberofprimarystudiesfallinginrelevantcategoriesinthe con-textofthespecificparameterand(ii)weaggregatedandvisualized the extractedinformation to better clarifysimilarities and differ-encesbetweentheprimarystudies.

Forwhatconcernstheanalysisofqualitativedata,weusedthe alreadypresentedkeywordingmethodforidentifyingalsothe pos-siblevaluesofeachparameteroftheclassificationframework,and thenwe analysedandsummarizedthetrendsandcollected infor-mationinaquantitativemanner.

Finally,we carried out anarrative synthesisof theresults ob-tained both quantitatively andqualitatively; thisstep allowed us to (i)perform an evidence-basedinterpretationof themain find-ingscoming fromtheprevious analyses and(ii)extract themain challengesandimplicationsforfutureresearch.Narrativesynthesis refers to a commonlyused methodto synthesize research inthe context ofsystematicreviewswhere atextual narrativesummary (i.e.,byusingwordsandtext)isadoptedtoexplainthe character-isticsofprimary studies(Popay et al., 2006 ), alongsideorinstead of a statistical analysis (Petticrew et al., 2009; Cruzes and Dyb, 2011 ).Inthecontextofourstudy,foreachparameterofour clas-sification frameworkwe firstlysummarizeditfroma quantitative

perspective(i.e.,statisticalsummary)andthenwe complemented such quantitative analysisby applying the generalframework for narrative synthesis proposed in Popay et al. (2006) , namely: (i) wedevelopedatheoryaboutthespecificvaluesoftheparameter bytabulatingtheresultsanditerativelyperformingcontent analy-sissessions, (ii)wedeveloped apreliminarysynthesis offindings basedon thequantitative analysis, (iii)we explored potential re-lationshipsinthe data (i.e.,horizontalanalysis), (iv)we assessed therobustnessofthesynthesisby criticallyreflecting onthe syn-thesisprocessandcheckingtheobtainedsynthesiswithauthorsof primarystudies.

4. Demographics

This research considers a set of 58 primary studies, each of thempublishedindifferentyearsandvenues.Fig. 4 showsthe dis-tributionoftheprimarystudiesovertheyearsandbythetypeof venuewheretheyhavebeenpublished.4_{Theobtaineddataclearly}

shows a growing trend in terms of publication intensity, with mostofthestudiespublishedintheveryrecentyears;specifically, 46studiesover58havebeenpublishedfrom2009to2016 (with anaverageofmorethan5publicationsperyear),where17studies havebeenpublishedonlyin2015and2016.Ifwelookatthe pub-licationnumbersbefore2009wehaveadroptolessthanone pub-licationper year.Theseresults area confirmation ofthegrowing scientific interest on safety for mobile robotic systems, specially inthelast years.The motivationsbehindsucha publicationtrend canbemanifoldincludingthegrowinginterestaboutautonomous vehicles5 _{andthe increasingfunding opportunitiesfordeveloping}

roboticsystemstobeemployedbothinindustrialandindomestic contexts.6

More on a historical perspective, the first studyon safety for mobile robotic systems (P11) has been published in the Applied Intelligenceinternationaljournalin1992.InP11 theauthors pro-posedanautomateddiagnosticmethodforkeepinganautonomous underwater vehicle operational for several weeks without hu-manintervention.The approachwasbasedona distributed fault-tolerantcontrol systemaimingatmanagingunpredictedfaultsby preservingits overallperformance level.The approach makes the assumptionthatthenormalbehaviourofeachcomponentis avail-ableatdesigntime.

Wealsoclassifiedtheprimarystudiesby(i)typeofpublication and(ii)targetedpublication venues.As showninFig. 4 ,themost commonpublicationtypeisconferencepaper(34/58),followedby journalpapers (16/58), workshoppapers (7/58), andfinally book chapters(1/58).

InTable 4 wereportthepublication venuesthathostedmore than two publications (the last row of the table is an aggre-gate ofall the publication venueswithtwo or lesspublications). What strikes the eye is the extreme fragmentation of the tar-geted publication venues(43 unique venues for58 publications). Nevertheless,we can observethat themosttargeted venues(i.e., theonestargetedbyatleasttwoprimarystudies,seeTable 4 )are quitehomogenousanddedicatedtorobotics,autonomoussystems, automation, andhigh-assurance systems. It is important to note that with Table 4 we are not aiming atestablishing which pub-licationvenue isthemostrelatedto safetyforMRSs; indeed,the sizeandfrequencyofconferencesandjournalsmayinfluencethe numbersreportedinthetable(e.g.,ayearlyconferencehas poten-tiallymoresafety-relatedpublicationsw.r.t.abiannualconference).

4 Our search activity covers the research studies published until January 2017,

thus we potentially have only partial data for 2016.

5_{https://www.gartner.com/smarterwithgartner/the- road- to- connected-}

autonomous-cars/ .

Fig. 4. Distribution of primary studies over the years - results.

Table 4

Targeted publication venues.

Nevertheless, given their focus on aspects related to safety for MRSs,wecanconsiderthevenuesreportedinTable 4 asgood can-didatesforfuturepublicationsonthisarea.

Inthefollowingwepresenttheresultsofthisstudyfor answer-ingourresearchquestions(seeSection 3.1 ).Foreachparameterof ourclassificationframework wereportboth quantitativedataand aninterpretationoftheobtainedresults.

5. Howsafetyismanaged(RQ1)

Thissectionaimsatidentifyingandclassifyingexisting method-ologiesthataddresssafetyinmobileroboticsystems.

In Section 3.3.1 we explained that inorder to provide a clas-sification framework we performed keywording that produces as outputtheformationofcategoriesoftheclassificationframework. Keyworkingisa standard techniqueand moreinformation might be found in Section 3.3.1 and in Petersen et al. (2008) . Roughly speaking, we collected all keywords across all papers and we groupthemtogetherintomeaningfulgroups.Theresultinggroups arethen clustered intoattributes andvalues(with different

pos-sible levels of hierarchy). The data extraction form is available atBozhinoski et al. (2016) .Fig. 5 showsagraphicalandtree-based representationofthecategoriesintheclassificationframework. It isimportanttohighlightthat thecategoriesthathavebeen iden-tified for safety management from the analysis of our primary studies through keywording and by following the process de-scribed in Section 3.3.1 .What emerges fromthis classification is that, fordesigning a solution forsafetymanagement we need to consideralsootheraspects,likethenatureofhazards,the charac-teristics ofthe system, whethermodels are used ornot, andthe involvedstandards,ifany.

According to the classification framework and the summary of the categories in Fig. 5 , research question RQ1 has been de-composed intomore detailedsubquestions.Therefore,we discuss about:

Fig. 5. How safety is managed.

Fig. 6. Safety management - results.

• systemcharacteristics:thefeaturesofthesystemssupportedby the proposed approach (e.g., cooperative versus local adapta-tion,thetypeofrobots,theircardinality,etc.);

• models:itisaboutthemodels7_{ofthesystemandtheirfeatures}

(i.e.,whethertheproposed approachisbasedonmodel-based techniques,thepurposesoftheusedmodels);

7 It is important to remark that in this paper, with the term model we refer to

specifications defining the different software aspects of the system being developed (e.g., requirement, component, and deployment specifications). Thus we do not refer to other kinds of models like 3D, mathematical, and physical ones that are consid- ered by the robotic community.

• standards: the standards to which the proposed approach is compliant(e.g.,IEC61508,ISO10218);

• hazards: aboutthe characteristics of the hazards managedby the approach (i.e., whether they are unexpected, their scope andcardinality).

Inadditiontothat,byfollowingwhatdiscussedinSection 3.3.1 , in the highlights of RQ1 (end of this section) we classified the primary studies with respect to parameters of other sec-ondarystudiesthat wediscovered inapre-study,asdescribedin

Fig. 7. Types of research contribution (a) and application field independence (b) - results.

Table 5

Types of research contribution (adapted from Petersen et al., 2008 ). Research contribution Description

Model Presents information, representations, and abstractions to be used in safety for MRSs.

Method Presents general concepts and working procedures to address specific concerns about safety for MRSs. Architecture Presents the fundamental concepts or properties of an

MRS embodied in its elements, relationships, and in the principles of its design and evolution ISO (2011) . Metric Presents specific indexes and measures to assess

certain properties of safety for MRSs. Tool Presents any kind of developed tool or prototype

related to safety for MRSs.

5.1.Safetymanagement-researchcontributions

In order to characterize where researchersare focussing their efforts, we extracted themain research contributionof each pri-marystudy.Categories ofresearchcontributions arederived from

Petersen et al. (2008) ,andcanbe oneormoreofthealternatives showninTable 5 .

The results of our analysis are shown in Fig. 7 a. It does not comeasa surprisethat themain contribution ofthemajority of primary studies is a method to address specific concerns about safetyfor MRSs (43/58); thisresult doesnot come as a surprise since our inclusion criterion I1 is explicitly dealingwith studies proposing eithera method ora technique forsafety. The second mostrecurrentresearch contributionisarchitecture(21/58);those studies present the fundamental concepts or properties related to the safety of an MRS by reasoning on its elements, relation-ships,andintheprinciplesofitsdesignandevolution(ISO, 2011 ). This result is interesting since it confirms that safety has been treatedas a system-level property by researchers, andthat con-sideringsafety at a higher level of abstractionis a valuable and effective strategy for attacking the problem. Other studies con-tributewiththeinformation,representations,andabstractionsfor safetyofMRSs (model,11/58), anddeveloped toolsorprototypes forsafetyofMRS(tools,9/58).Asafinalconsideration,noprimary studyhasasmaincontributionmetrics,indexes,ormeasuresto as-sess certain propertiesofsafety ofMRSs. By followingold adage thatwhat gets measuredgets managed,workingon safety-specific metricsforMRSscanbeanaddedvalueforthefieldandsurelyan interestingresearchgaptobefilledbyfutureresearch.

5.2.Safetymanagement-applicationfieldsindependence

As showninFig. 7 b,almostall theprimarystudiesaregeneric

withrespectto anyapplicationfield. Thismeansthatthose

stud-ies are kindoforthogonal and can be appliedto some extent to differenttypes ofrobots,tasksto be performed, operational con-texts, etc. Forexample, the authors ofP9 achieved generality by applyingthewell-known abstractionandautomationprinciplesof the Model-Driven Engineering paradigm (MDE, (Schmidt, 2006 )). By quoting their own words, their approach directly enables an implementation-independent reuse of the safety-related part of a robotcontrollerbetweendifferentreleases,sincetheRuBaSS declara-tiondoesnot needto changewhentheunderlyingsoftwarechanges (exceptthatnamessharedbetweenRuBaSSrulesandcomponent in-terfacesmustbekeptconsistent).Moreover,theinfrastructurecanbe reused in a range of products: the code generator can be directly reused whereaslow-level interfacesto sensors and actuators willbe specifictoeachrobot.Safety-relatedcustomisationfortheproductsis thus mainly achievedat the higherlevel, using the safety language

(P9).

Application-specificapproacheshavebeenproposedin8primary studies (namely, P7, P9, P11, P39, P42, P50, P54, and P56), with application fields ranging from health to domestic or industrial robotics.

It isimportantto knowthat applicationfield independenceis strongly related to the level of abstraction of a given approach. Specifically,ahigherlevelofabstractioncanresultinahigher po-tential for reuse across domains, thanks to the abstraction from the low-level details and constraints of a specific domain. Also, if an approach is independent froma specific domain, then po-tentiallyitmaybe usedbyawider community,leadingtohigher potential forcross-fertilization acrossdisciplines(e.g.,an obstacle avoidance algorithm for planetary exploration may be used and adaptedforterrestrialexploration),orevenmorebugsdiscovered (and potentially fixed) inthe tool supporting the approach. Nev-ertheless,having an approach specifically tailoredto a given do-main(e.g.,exploratoryrobotsinwildareas)allowsengineerstobe morespecializedwhensolvingdomain-specificissues(e.g.,howto managetheinteraction withwildanimals),potentially raisingthe chancesofindustrialadoptionintheshortterm.

5.3. Safetymanagement-worldknowledge

It is important to identify the knowledge ofthe robotof the environmentinwhichtherobotwilloperate.Whenwedealwith multi-robots,thevariousrobotsmightsharetheknowledgeabout theenvironmentindifferentways.Webelievetheseareimportant aspectsthatshouldbetakenintoaccountforhavingrobotsableto performeverydaytasksinenvironmentsthat,increasingly,willbe uncontrollableandonlypartiallyknown.

the mission)is localto eachrobot, withoutmechanisms toshare knowledgebetweendifferentrobots.2approacheshavea central-izedworldknowledge,meaningthattheknowledgeoftheoverall system is maintained by a centralized entity. 8 approacheshave cooperativeworldknowledgeandthismeansthatthereare mech-anismstoshareknowledgebetweendifferentrobotsthattakepart inthemission.

It is important to note that only two approaches with local knowledgeinvolvemulti-robots,namelyP43andP51.Thisexplains why we havea majorityof approachesthat rely on local knowl-edge. In general, we might say that having a centralized world knowledge inmulti-robot systems mighthamperthe adoptionof decentralizedalgorithmsfor(re)planning,issuesresolution,andso on.

Managingthe uncertaintyof the environmentwhere the con-sidered robot has to operate is an orthogonal aspect, which is cross-cutting to those previously mentioned. Even though hav-ing the availability of a complete model of the environment represents the ideal situation, in practice only partial and lim-ited world models are possibly available and consequently, spe-cialized techniques are needed to permit robots to work with uncertainworldknowledge.Forinstance,inPapp et al. (2008) au-thors propose an approach for modeling cooperative intelligent vehicles by means ofmodeling constructs enablingthe specifica-tion ofuncertaintydegreesforattributes ofthemodeled objects. In Gheta et al. (2010) authors propose an approach to support world modeling forautonomoussystems. The maincharacteristic oftheproposed techniqueisthat “itmodels uncertaintiesby prob-abilities, which are handled by a Bayesian framework including in-stantiation, deletion and update procedures”. Recently, a novel ap-proach has been proposed to deal with uncertainty of software models, byfocusing onmeasurement uncertainty,andconfidence (Burgueño et al., 12018 ). However, dealing with uncertainty is a very challenging problemand an in-depth treatment of it is be-yond thescopeofthissection, whichismorefocusedontheway worldknowledgeismanaged(e.g.,locallyorinacooperative man-ner)andnotonitscontent.

5.4. Safetymanagement-mechanism

Concerningthisparameterwedonot listthedifferent mecha-nisms,butwecategorizethemaslocal,centralizedorcooperative. A mechanismis localifit is conceivedto work on singlerobots, without anycooperation,centralized ifthereis an entity manag-ingthesafetyaspectofthesystem,orcooperativeifsafety mecha-nismsinvolveacooperationbetweendifferentrobots.Asshownin

Fig. 6 b,mostoftheapproaches(46/58)adoptlocalsafety mecha-nisms,i.e.safetymechanismsthatareconceivedtoworkonsingle robots, without anycooperation. This is expectedsince, as high-lighted in Section 5.3 , most of these approachesfocus on single robots. The exceptions are P43 and P51 that deal with multiple robots even though they have local safety mechanisms, and P54 that hasbothlocal andcentralized safetymechanisms.Ascan be seen inthe figureonly 1approach hasa centralized safety man-agement mechanism. Instead, 8 approaches rely on cooperative safetymechanisms,meaningthatsafetymechanismsinvolvea co-operation between differentrobots.Finally, 4 approachesprovide noinformationaboutthisaspect.

5.5. Safetymanagement-abstractionlevels

Whendevelopingcomplexsystems,abstractionisakeyconcept to master complexity.Insoftware engineering, the systemsto be developedareanalyzedatdifferentlevelsofcomplexityby focus-ing on a few issues andaspects ata time. As shown in Fig. 6 c,

Table 6

Safety management - abstraction levels.

Level(s) Number of studies Requirements 3

Requirements + Low-level design 8 Architecture 9 Architecture + Low-level design 3 Architecture + Implementation 1 Low-level design 28 Low-level design + Implementation 3 Implementation 3

the abstraction level of the safety management spans from re-quirementtill implementation.A requirements value means that safetyisconsideredwhen eliciting/specifyingthe requirementsof the system (e.g., generic safety rules written in a non-technical way).Architecturemeansthatsafetyisconsideredatthe architec-turallevel(e.g.,they talkaboutarchitecturaltactics, styles, archi-tectural patterns, system infrastructure, communication topology, etc.).Low-leveldesign meansthat safetyisconsidered atthe de-signlevel (e.g., design patterns, design models, etc.). Finally, im-plementationmeans that safetyisconsidered atthesource code, programminglevel.

Themajorityof theapproachesworksatthe designlevelthat seems to be the most appropriate level to reason anddeal with safety management. The design level is followed by the archi-tecture level.In fact, as shown inTable 6 , 7 approaches address safety at the implementation level and among them only 3 ap-proachesexclusivelyaddresssafetyattheimplementationlevel,3 approachesaddresssafetyalsoatthedesignlevel,and1atthe ar-chitecturelevel. Thistestifiesthat itmightbe difficulttomanage safetydirectlyattheimplementationlevelanditismoreprofitable todealwithitatmoreabstractlevels.

5.6.Safetymanagement-separationofconcerns

AsshowninFig. 6 d,forthemajorityoftheapproaches(35/58), themanagement ofsafety-specificissues (e.g.,safetyrules)isnot keptseparatedfromthefunctionalmanagementoftherobots(e.g., themission).Keepingaseparationofconcernsmeansforinstance that the approach prescribes a special layer formanaging safety, whichis totallyseparatedfrom therestof thesystem. Managing complexmissions requiresaclearseparationofconcernsbetween safetyand other aspects of the system. We consider that safety-specific objectives should be separated fromthe rest of the sys-tembecausethenatureofthesafetyobjectivesisdifferenttothe other objectives (e.g., missionobjectives). Safetyis considered as a first class concern in MRSs which means that MRS should al-ways satisfy the safety objectives, while the other concerns (e.g. missionconcerns)canbepartiallysatisfied.Thatwayasafety en-gineercan focus ondefinition ofsafety-specific mechanismsthat aregenericandindependentfromthefunctionalbehaviourofthe system,while,forexample,an operatorcan focusonthe mission functionalspecification.

5.7.Safetymanagement-platformindependentspecification

5.8.Safetymanagement-additionalpropertytypes

As showninFig. 6 f,mostoftheapproachesdeal with proper-tiesthataredifferentfromsafety.Infact,40approachesdealwith additionalproperties,onlyone approachisexclusivelyfocusedon safety,P3,and17papersdonotprovideinformation.Table 7 shows the additional properties and adds a reference to primary stud-iesthat are addressing the specific properties.There is abig va-riety of additional properties that are addressed by the primary studies -22 different additional propertiesconsidered by the 40 primary studies that consider additional properties. Performance is the most addressed property, followed by functional correct-ness. The motivations behind the interest on performance when managingsafetyofMRSs can be manifold, includingthe need of improvingthenon-functionalpropertiesofthesoftwareand hard-warecomponents that areinvolved whenreacting tounexpected events.Similarly, functional correctness is an additional property tobe addressed forexamplewhen developing monitorsthat can detectconditionsthat maylead tofailuresandthusneed totake correctiveactions.

5.9.Systemcharacteristics-openness

In the context ofthis study,by open systems we meanthose systems that allow forentrance and exitof entities during mis-sion execution (Bucchiarone et al., 2013 ). Openness can improve the dynamicity of the MRS, for example by allowing to let new robotswithbetterornewfunctionalities(ornewhumanactors)to getintotheMSRortolet robotsthathavecompleted theirtasks toexittheMSR.AsshowninFig. 8 a, mostofthe approachesare unabletodealwithopensystems(only 5approaches,namelyP2, P22,P48,P49,P53,are abletodealwithopen systems).This im-pliesthatmostoftheapproachesthathavebeenproposedarenot ableto manage safetyoncethe systemevolvesin termsof addi-tionor removalofrobots and/or other typesof agents,including humans.Thisisindeedan interestingresearchdirectionsince sys-temsofthenearfuturewillbenecessarilycharacterisedby

open-ness,anditisoftenimpossibletoassessatdesigntimetheexact boundariesandtopologyofthesystem.

5.10. Systemcharacteristics-contextawareness

Ascanbe seeninFig. 8 b,mostoftheapproaches(41/58)deal with systems (including the robots) that are able to understand some key properties aboutthe operational context of the robots (e.g.,presenceofobstacles,existence ofotherrobots,etc.).10 out of58approachesdonotprovideinformation.Contextawarenessis anotherimportant characteristicto enabletheadoption ofrobots inreal lifescenarios, whereoftenthe operationalenvironment is (partially)unknownanduncontrollable.

5.11. Systemcharacteristics-adaptiveness

Fig. 8. System characteristics - results.

conjunctionwithcontextawarenesssinceawarenessofthecontext isarequiredcapabilityinordertosupportadaptiveness.

5.12. Systemcharacteristics-scenetype

This parameter aims to show how much of the safety ap-proaches are tailored for specific scene types and how much of them are independent fromthe type of scenewhere the MRSis performing its mission. Fig. 8 d describes the ability of the sys-tem to work indoor (21/58), outdoor (16/58), or independent of thescene(9/58).Someapproachesprovidenoinformationinthis concern(13/58).Pleasenoticethatwecategorisedan approachas independentonlyiftheapproachexplicitlymentionsaboutits in-dependence ability. In conclusion, the majority of the safety ap-proaches are tailoredto systems that performin aspecific scene type (indoor or outdoor) instead of having a more generalized safetyapproach.

5.13. Systemcharacteristics-heterogeneousrobots

Anotherpeculiarsystemcharacteristicisthecapabilityof man-aging teams consisting of robots of different types (e.g., robots forgrabbingobjects,forvideostreaming,sensinganddiscovering relevant information). According to Fig. 8 e most of the analyzed systems (46/58) do not have the capability of managing het-erogeneous robots. Only 10 systems provide users with such a functionality, whereas 2analyzed systems donot provide aclear statementaboutthat.Hence, mostsafetyapproachesthat are ad-dressingteamofrobotsarefocusedonhomogeneousrobots.

5.14. Systemcharacteristics-cardinalityofrobots

Missions can be executed by one or more robots.Indeed the management of different robots introduce additional challenges mainly related totheir collaborationand coordination.As shown in Fig. 8 fmostof theanalyzed systems(45/58)support missions

performedby a single robot (e.g., self-driving car), while few of themdeal withthemanagement ofmultiplerobots.Hence, main focus on safety approaches have been single robots. Researchers shouldconsider proposingsolutions that willaddress safetyona teamlevel.

5.15.Systemcharacteristics-typeofrobots

Thisparameter can havevalues in the set {TERRAIN, UNDER-WATER,AERIAL,ACQUATIC, GENERIC}.Ifthe authorsofaprimary studyexplicitly claim that their proposed approach isspecific to a type of robots (e.g., UAVs), then we set the value of this pa-rameterto thefamilyof thespecific type ofrobot(e.g., AERIAL); if the authors of a primary study claim that their proposed ap-proachisindependentofthetypeofrobots,thevalue ofthis pa-rameter has been set to GENERIC. In order to manage different kinds of missions it is preferable that the used systemprovides users with functionalities that are robot independent. According tothe performedanalysis, 7out of58analysed systemsare spe-cific to terrain robots (see Fig. 8 g), 2 specifically conceived for aerialrobots,and1forunderwaterrobots.Mostofthesystemare generic(47/58)andpaperP40doesnotprovideanydetailsabout thesupportedrobottypes.

5.16.Systemcharacteristics-platform

Fig. 9. Model-based specifications and standards - results.

theanalysis,we counted17differentplatforms inadditionto ad-hocones. In Table 8 we show the most used platforms (at least twooccurrences). ROSisone ofthemostusedplatforms (13/58), eventhoughthemajorityoftheanalyzedprimarystudiespropose their ad-hoc technologies (20/58).Such numbers are justified by the need of abstraction layers taming the complexity of writing softwareforroboticsystems. EventhoughROS wasexplicitly de-signedwithsucha goal,ad-hoc platformsare alsoemployede.g., toovercomelimitationsofROS(e.g.,scalabilityandreliability)that mightbecriticalforsomeapplicationdomains.

5.17. Models-model-basedspecification

Engineering mobile robotic systems has to take into account severalaspects that mightgo fromrequirementelicitation tothe specificationofhardware characteristics. Consequently,the adop-tionofmodel-basedtechniques canhelp developers inmanaging the different aspects by increasing abstraction and enabling au-tomation.Manyapproachesmake useofmodels (42/58asshown inFig. 9 a)forvariouspurposes,e.g.,tosupportthespecificationof missions,safetyconstraints,hardwareinvariants, etc. Only10 ap-proachesdonotmakeuseofmodelsfordevelopingandusingthe roboticsystemsathand.

5.18.Models-purposeofthespecifiedmodels

Bycontinuingthediscussionrelatedtothepreviouspoint,the adoption of models can be done for different purposes. Most of theconsideredapproaches(35/42asshowninFig. 9 b)adopt mod-elsforanalysispurposes(e.g.,feasibilityassessment,mission exe-cutiontimeprediction,etc.). Someofthem (7/42)usemodelsfor

generating the code of the modeled systems or to apply model-to-model transformations(7/42) targetingmodels that are in the form, which is more convenient for the particular analysis task. Someoftheanalyzedsystems(5/42)usemodelsatrun-timee.g., tosupport theexecutionofthe missionwhileit isexecuted. The papers in the Other category are P15, P18, P29, P37, P41. In P15 modelsareusedtosupporttherun-timeanddynamicadaptation ofsystemsduetounforeseenenvironmentchanges.Adaptive sys-temsareconsideredalsoinP18andP29thatproposetheadoption ofmodelstodealwithfaulttolerantaspectsofthesystemsbeing developed.FaultmanagementisalsothemaintopicofP37,which adoptsmodelsforspecifyingsystemsconsistingofmultiplemobile robots.P41proposestheadoptionofmodelsforsupportingthe de-velopmentofautonomoussystems,whichhavetobeself-healing.

5.19. Standards-compliantstandards

Mobileroboticsystemsareverycomplexastestifiedalsobythe number of standardsthat are considered when developing them (see Table 9 ). According to Fig. 9 c10/58 approaches are compli-ant to standardsthat specifically target safetyaspects. As shown inTable 9 ,each approachcan adoptmorethanone standard de-pending on the peculiar aspects of the system being developed. Forinstance,P35 andP42makeuseof4standardseach.The for-mer,proposesanapproachtodevelopsafecontrolsystemsandas asuchitreferstothefollowingstandards:

• IEC61508 – Functional Safety of Electrical/Electronic/ ProgrammableElectronicSafety-relatedSystems;

Table 9

Standards with compliant approaches.

• ISO13855 – Safety of machinery - Positioning of safeguards with respect to the approach speeds of parts of the human body;

• ANSI/RIAR15.06– IndustrialRobotsandRobotSystems-Safety Requirements.

InP42authorsproposeanapproachtoverifythecorrectnessof visionpipelinesinagriculturalsettingswiththeaimofimproving thesafetyofthesystemsbeingdeveloped.Theproposedapproach considersthefollowingstandards:

• ISO13482 – Robots and robotic devices- Safety requirements forpersonalcarerobots;

• ISO25119– Tractorsandmachineryforagricultureandforestry Safety-relatedpartsofcontrolsystems;

• ISO18497 – Agricultural machinery and tractors – Safety of highlyautomatedagriculturalmachines;

• IEC61496 – Safety of machinery - Electro-sensitive protective equipment.

As it is possible to notice, the standards that are referred by theexistingapproachesvarymuchdependontheparticular appli-cationdomainswheretheconsideredroboticsystemswillbe em-ployed.

5.20. Hazards-unexpectedenvironmenthazards

Inordertoemploymobileroboticsystemsinrealcontexts,itis importantthat theyhavethecapabilityofreactingtounexpected environment threats, such as the presence of unpredicted obsta-cles,thepresenceofhumansintheoperatingarea,etc.Wedefine

hazardasan atomicevent, situation,and/orobjectthat bringsan unavoidabledangerorriskinmobileroboticsystems.Hazardscan have a variety of forms (ex. an internal fault of a robot, an un-wanted human behavior, an unexpected situation - dynamic ob-stacle,anemergentbehaviourraisedfromthecooperationandthe coordinationoftherobotsandmuchmoreothersituationscoming internallyfromthesystemorexternallyfromtheenvironment).As shownin Fig. 10 a, the majority of the analyzed systems (29/58) implementsuch acapability. The primary studiesP3, P4, P8, and P38do not give explicit informationaboutthat. In particular, P3 proposes an approach to support the diagnosis of complex sys-tems.P4discusses alltheconcepts thathavetobetakeninto ac-count whendesigning autonomoussystems by touching different peculiaraspects likecommunication, control, andnavigation.The focus of P8 is supporting testing activities when developing the controlsoftwareforautonomoussystems.Withtheaimof improv-ing the quality of the software of robotic systems, P38proposes anapproachtomanagefaults ofcomponentsbasedontheOPRoS platform.

5.21.Hazards-scope

When considering unexpected environment hazards, systems canbe distinguishedwithrespect totheir capabilityofmanaging threatsimpactingordueto asinglerobot(44/58asaccordingto