ContentslistsavailableatScienceDirect
The
Journal
of
Systems
and
Software
jo u r n al h om e p a g e :w w w . e l s e v i e r . c o m / l o c a t e / j s s
Empirical
research
methodologies
and
studies
in
Requirements
Engineering:
How
far
did
we
come?
Maya
Daneva
a,∗,
Daniela
Damian
b,
Alessandro
Marchetto
c,
Oscar
Pastor
daUniversityofTwente,TheNetherlands
bUniversityofVictoria,Canada
cIndependentResearcher,Italy
dTechnicalUniversityofValencia,Spain
a
r
t
i
c
l
e
i
n
f
o
Articlehistory:
Received17June2014
Accepted18June2014
Availableonline30June2014
a
b
s
t
r
a
c
t
SincetheinceptionoftheREconferenceseries(1992),bothresearchersandpractitionersintheRE com-munityhaveacknowledgedthesignificanceofempiricalevaluationasaninstrumenttogainknowledge aboutvariousaspectsofREphenomenaandthevalidityofourresearchresults.Asignificantnumberof empiricalstudieshavebeenconductedinthesearchforknowledgeaboutREproblemsaswellasevidence ofsuccessfulandlesssuccessfulapplicationofproposedsolutions.Thiseditorialpresentstheprogress empiricalREresearchhasmadesince1992.BasedonasearchintheScopusdigitallibrary,wereport fromananalysisofpeer-reviewedsystematicliteraturereviewsandmappingstudiestoshowcasemajor areasofREresearchthatusemethodsfromtheEmpiricalSoftwareEngineeringparadigm.Wesummarize priorempiricalresearchinREandintroducethecontributorstothisspecialissueonempiricalresearch methodologiesandstudiesinRE.
©2014ElsevierInc.Allrightsreserved.
1. Introduction
RequirementsEngineering(RE)hasbecomeawell-established disciplinewhereawiderangeofapproaches,techniquesandtools havebeenproposed. Systematicattempts toevaluateand com-paretheusefulness,effectivenessandusabilityofsuchproposals resultedinagrowingattentiontomethodsforempirical assess-ment.EmpiricalSoftwareEngineering(ESE)aimsatapplyingthe empirical research methodologies to the software engineering field.Itaimsatstudyingandproposingqualitativeandquantitative methodstocollectandanalyzeevidencethathelpsevaluating soft-wareengineeringapproaches,techniquesandtools.Experiments, surveys,casestudies,actionresearchstudies,hence,become indis-pensableandvaluablemethodstoverifythattheproposedresearch ideasandresultsconformwiththerealityofsoftwareengineering; theybecomeindispensableinassessingtheirvalue,costand bene-fitsinparticularoperationalcontexts.TheobjectiveofthisSpecial Issueistoincreasetheawareness ofthevalueofandtheneed forcross-fertilizationofESEmethodsandRE.Morespecifically,we intendtofosteradiscussionamongresearchersandpractitioners
∗ Correspondingauthor.Tel.:+31534892889.
E-mailaddress:m.daneva@utwente.nl(M.Daneva).
onwhatarethebetterways(1)tostrengthenthe methodologi-calbaseofREresearchand(2)toleverageempiricalevaluation approachestoexploreandconsolidatethemultidisciplinarynature ofindustry-relevantREresearch.
Historically,aworkshopseriesfocusedspecificallyon empir-ical methods in RE was created in 2011 in the form of the InternationalWorkshoponEmpiricalRequirementsEngineering (EmpiRE),and onearticle fromits2012editionis publishedin this special issue.The EmpiRE workshop series buildsupon an earlier workshop series onComparative Evaluation in Require-ments Engineering (CERE) (2003–2007), and which aimed at setting up systematic empirical evaluation practices that serve thepurposeofcomparingREmethods,processesand
technolo-gies (Gervasi et al., 2004). The workshop triggered a number
of important conversations on pragmatic topics that directly respondedtotheurgentneedforformulatingcriteriathatmake comparisonoftheeffectivenessofvariousREresearchoutcomes possible.Most notably,thetopics included(1) what evaluation criteria empirical REresearchers shouldconsider for usewhen comparingREmethods,processesandtechnologies(e.g.CERE’06
http://www.di.unipi.it/CERE06/program.html),and(2)what
crite-ria tousetoevaluatethe researchoutputof suchcomparative efforts(e.g.Wieringaetal.,2005; Easterbrook,2007a).Perhaps, themostlastingcontributionoftheCEREworkshopseriesisthe http://dx.doi.org/10.1016/j.jss.2014.06.035
classificationofREresearchpapers (Wieringaetal.,2005,2006;
Wieringa,2005;Easterbrook,2007a,b)whichtothesedayisused
bytheREcommunitytoguidetheworkofthePCmemberswhen reviewingpaperssubmittedtotheannualREandREFSQ confer-ences.
Thiseditorialnotonlyintroducesthespecialissuepapers,but alsoissetouttoprovideareflectiononpastempiricalresearch in REand challenges lying ahead. In particular, it is important forustounderstandwhatempiricalREevidencehasbeen accu-mulatedover time and onwhich RE sub-topics(e.g.elicitation methods, modeling techniques) or application domains. In the RE field, the first empirical study has been published in 1983
(Davis,1983),nineyearsbeforetheinitiationoftheREconference
series.Given the importanceof empirical researchand evalua-tion,itisperhapssurprisingthatrelativelylittleefforthasbeen donetowardthecross-fertilizationofESEandREinasystematic way.
Therestofthiseditorialisorganizedasfollows:InSection2
we providebackground onthe possiblefactors that fueledthe increasedawarenessofempiricalREinthepastfewyears.InSection
3,wefirstdescribethegrowthofempiricalREpublications,whichis basedonasearchforempiricalREpublicationsintheScopusdigital library.Also,wesummarizeobservationsfrompublishedmapping studiesandsystematicliteraturereviewsinREavailableinScopus. Weexaminedthetopicscovered,theprimarystudiesbeingused intheprocessofevaluatingevidence,thetheoriesbeingusedin thosestudies,andthewaygeneralizationquestionsweretreated. Wesummarizetheresearchprogress,discusssomechallengesfor thefuture,andfinallypresentthepapersmakingupthisSpecial Issue.
2. TheincreasingawarenessofESEprinciplesinREresearch
IntheRE communitythere is a consensusthatcomparative evaluation of RE research efforts play a crucial role in (1) the growthofREasascientificdisciplineand(2)technologytransfer. Muchempiricalpublicationoutputwasproducedbythe commu-nityandabroadvarietyofempiricalresearchmethodsfromthe evidence-basedsoftwareengineeringparadigm(Kitchenhametal., 2004) have been used and reflected upon: systematic reviews, experiments, case studies, focus group studies, grounded the-orystudies,action research.TheREcommunity seemstowork really hard onidentifyingtheappropriate research methodolo-giestostudyREphenomenaandtheevaluationcriteriasuitable to judge RE research efforts and output (Gervasi et al., 2004;
Wieringa et al., 2005). We outline below some of the main
reasons for the growth of publications on empirical research inRE.
First,thesoftwareindustryandtheITconsultingsectorin gen-eralareincreasinglymoreawareofboththeexpensesassociated withpoorrequirementsandtheimportanceofadoptingor adapt-inggoodREpractices. Detailedmarketdataanalysesconducted over the years point to RE as the most expensivepart of any systemsdeliveryproject,regardlessofcountryororganizational settings.Similarly,researchoncriticalsuccessfactorsand produc-tivityinsoftwaredevelopmenthasbeenadvancing(e.g.Shauland
Tauber,2013;WagnerandRuhe,2008),providingclearevidence
ofthequalityofrequirementsasafactorthatcanmakeorbreak a project. Thisfuels themotivationof REresearchers to evalu-atetheextenttowhich theirproposedtechniquesaddvalueto businessesandhelpchangetheoddsforprojectoutcomes.To com-panies,thismeansgettingmoreconsciousaboutactivelysearching forpracticesthatworkedinotherorganizationsandattempting toemulatetheseorganizations’successesbasedonevaluationof existingevidence.
Second, the recent developments in large scale outsourcing worldwidehighlighttheneedofexplicitlyspecifyingrequirements veryearlyinthesystemsdeliverycycleaspartofsetting outsourc-ingcontracts(Damian,2007;DamianandMoitra, 2006;Daneva etal.,2013).Often,REbecomesa pre-projectonitsownthatis pricedseparatelyandisaimedexplicitlyat‘gettingrequirements right’.
Third,researchfundingagenciesinEuropeandNorthAmerica callincreasinglyforindustry-universitycollaborationproject pro-posalsonmulidisciplinaryresearchinwhichREplaysanimportant role,e.g.inthedevelopmentofsystemsinthedomainsofsmart cities,internetofthings,healthcareandonlinegaming.Project pro-posalsarerequired toincludeplansfor empiricalevaluation of existingREmethods,processesandtechnologiesforthepurpose ofunderstandingtheirpossiblefittotheproblemtodeterminethe requirementsforsystemsintheseapplicationdomains.
Forth,astheREfieldhasbeenmaturingforthepasttwodecades, theawarenessofexplicitandsystematicdocumentationof empiri-calresearchdesignshasbeengrowing,too.Thereisanagreementin theREcommunitytodaythatthemoreexplicitlyaresearchdesign isdescribed,theeasieritisforthereadersofempiricalREpapers toevaluatethegeneralizabilityof theresearchbeingpublished, e.g.theextenttowhichreadersmightexpecttoobservepublished resultsinothersimilarbutdifferentsettings.
3. ReviewofpastempiricalresearchinRE
3.1. GrowthofempiricalREpublicationsandgeographic distribution
Fig.1illustratesthegrowthofempiricalREpublicationsbased ona searchof literaturesourcesavailable intheScopus digital library.1ThesearchwasdoneonMay13,2014andyielded2218 researchpaperspublishedbetweenJanuary1,1983andDecember 31, 2013. For the purposeof getting indicative information on thepointsdiscussedinthiseditorialaswellastoprovide exam-ples, wechose touseScopusbecause it tracksa largenumber ofjournalsandconferencesincomputerscienceandinformation systemsresearch,whilegivingustheadvantageinfacilitatinga sin-glesearchqueryaccessitemsfromabroadvarietyofpublishers, unlikeSpringerwhenusingSpringerLink,orElsevierwhenusing ScienceDirect.
Thissearchresultedin628journalpapers,19bookchapters, and1590conferencepapers.Fig.1showsthegrowthofpublished empiricalREstudiesbetween1983andendof2013.
Usingtheresultsofoursearch,weexaminedthedistributionof thesepapersacrossthedifferentcountries.Fig.2presentsthe num-berofarticlespercountry.Weobservethatwhilenearly30%(773 outof2218)ofthepapersoriginatedintheUnitedStates,United KingdomandCanada,theaffiliationsoftheauthorsofempiricalRE researchpublicationsarelocatedinatotalof40countries.Among theEuropeancountries,authorsfromGermanyproducedthe high-est number of empirical RE papers (232). The other European countriestowhichahighnumberofempiricalREauthorsaffiliated whilepublishingempiricalpapersareItaly,Sweden,Austriaand theNetherlands.Authorsfromthesecountriesproduced papers ranginginvolumebetween106and128,ineachcountry.Outside NorthAmericaandEurope,Brazilstandsoutasacountry contribut-ingahighnumberofempiricalREpapers(95).Wealsoobserve thatempiricalREresearchactivitytakesplaceinAsiancountries,
1Weusedthefollowingsearchstring:“requirementsengineering”AND(“action
research”OR“groundedtheory”OR“focusgroup”OR(empiricalAND(studyOR
evaluationORassessmentORanalysis))OR“fieldstudy”OR“qualitativestudy”OR
0 50 100 150 200 250 1983 1985 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013
Number of publicaon
s
Fig.1.NumberofempiricalREstudypublishedsince1983.
Table1
Conferences.
Conferencevenue Numberofempirical
REpublications RE 198 REFSQ 55 ICSE 47 APSEC 28 EmpiRE 24 EuromicroSEAA 24 SAC 19 ESEM 17 HICCS 10 RCIS 8 COMSAC 8 EDOC 5
suchasSaudiArabia,Pakistan,SouthKoreaandTaiwan.Thisisa verypositive andencouragingdevelopment,given thefactthat thesecountriesareunder-representedintheproceedingsofthe REconferenceseries(thepremiermeetingpointoftheRE com-munity).Wenotethatexamplesofempiricalstudiescarriedoutby authorsfromthesecountriesarepublishedhoweverintheSpringer RequirementsEngineeringJournal(whichisthepremierjournalin theREcommunity).
Furthermore,welookedatthevenues,bothconferenceseries
(Table 1) and journals (Table 2) where most research papers
havebeenpublished.Table1confirms theintuitiveassumption thattheRequirementsEngineering(RE)conference (requirements-enginnering.org)isthemostpopularconferenceoutlet,followed bytheREFSQseries(www.refsq.org).Clearly,thisisunsurprising giventhefactthatbothvenuesareregardedbytheREcommunity asthepremiumdestinationsforREresearchingeneral,and empiri-calRE,inparticular.Furthermore,initsthreeeditions(2011–2013) theEmpiRE2workshoppublishedatotalof24empiricalREstudies.
2Atthetimeofwritingthiseditorial,theEmpiRE2014workshopisevaluating
18otherempiricalREstudies.
Next, we observethat quite a few empirical RE papers are publishedingeneralSEconferences.Thehighestnumberof pub-lications (see Table 1) are at the International Conference on Software Engineering (ICSE), at the Asia-Pacific Software Engi-neeringConference(APSEC),atEuromicroConferenceonSoftware EngineeringandAdvancedApplications(SEAA),and attheACM SymposiumforAppliedComputing(SAC).Alargenumberofpapers however arespread overan extremelylargenumber ofevents, someofwhicharepopularwithcommunities,suchasthe Infor-mationSystemsResearchcommunity,Human-MediaInteraction, CloudComputing,WirelessSensorsandCellularNetworks.Most oftheseeventshavepublishedonlyoneempiricalREstudythatis usuallyrelatedtothecentralthemeoftheevent.
Intermsofjournalpapers,theRequirementsEngineering
Jour-naltopsthelist(see Table2), followedbythejournalsthatare
well-knownforpublishingESEresearch:theJournalofInformation andSoftwareTechnology,theTransactionsofSoftware Engineer-ing,theJournalofSystemsandSoftware,andIEEESoftware.Next tothesewell-knownESEoutlets,quiteafewempiricalREpapers (16)havebeenpublishedintheEuropeanJournalofInformation Systems. It issomewhat surprisingthat theEmpiricalSoftware EngineeringJournalsofarpublished14empiricalREpapers,which ranksitlowerintermsofpopularityintheREcommunitycompared tootherjournals.
Moreover,Table2indicatesthatempiricalREpapersappeared injournalsfocusedonspecificSEsub-areas,e.g.theSoftware Qual-ityJournal,theJournalofSoftwarePracticeand Experience,the JournalofSoftwareMaintenanceandEvolution,theInternational JournalonSoftwareToolsforTechnologyTransfer,theExpert Sys-temsJournalandtheJournalofExpertSystemswithApplications, whichsuggeststhatotherSEcommunitiesalsopayattentionto empiricalREresearch.Wealsoobservethatanumberofpapers arespreadover awiderangeofjournals,suchastheJournalof InformationandManagement, InformationSystems,theJournal of EnterpriseInformationSystems, theJournal of Human Com-puterStudies,theHealthInformaticsJournalandtheJournalof BiomedicalInformaticsIEEEIntelligentSystems,Journalof Enter-priseTransformation.Thisagain,isagainahintthatempiricalRE researchdrawstheattentionofothercommunitiesbeyondSE.
367 263 232 173 156 128 116 113 106 95 88 77 65 40 36 35 33 32 30 29 29 27 24 22 18 18 17 17 16 15 12 12 10 10 9 9 8 8 7 7 400 350 300 250 200 150 100 50 0 USA UK Germany Canada China Italy Sweden Austria Netherlands Brazil Spain Japan France Australia India Finland Soth Korea Malta Norway Belgium Portugal Taiwan Sweden Pakistan Ireland HongKong Denemark United Emirates Israel Argenna Greece Turkey NewZealand Chile Singapoor SaudiArabia Poland Mexico Columbia SouthAfrica
Number of publicaon
s
Fig.2.NumberofempiricalREstudypercountry.
3.2. Topicsandtheoreticalperspectives
Thissectionnarrowsdownthediscussiontoprovideasnapshot ofstate-of-theartinevaluation ofevidenceproducedin empir-icalREstudiesasreportedin systematicliteraturereviews and mapping studies in RE. Generally, one of the goals of a map-pingstudiesandsystematicreviewsis todescribeandevaluate state-ofthe-art research byusing empirical evidence produced in primary studies (Kitchenham, 2006). As our goalis to indi-cateprogresstodateinempiricalREanduseofmethodologies, we consideredreviewing thesetwo kinds of reviews asa use-fulwaytogainsomeindicativepointsforreflectionontrendsin thearea.Forthepurposeofthiseditorial,wesearchedScopusfor thesetwokindsofreviews.Oursearchyielded7mapping stud-iesand49systematicreviews.Foreachone,wechecked(i)the topicscovered,(ii)whetherthestudyfocusesonaREartifactor aREprocess,(iii)whetherthestudyaddressesaparticular orga-nizational/developmentcontext,and(iv)theapplicationdomain in which RE is applied. The next sub-section summarizes our observations.
3.2.1. Topics
Table3presentsthetopicsthat havebeencovered by
map-pingstudiesandsystematicreviewspublisheduntilMay2014.We found38distinctivetopics.Thetopicwiththehighestnumberof studiesare:requirementsprioritization(4studies),requirements specification (4 studies), security RE (4 studies) and creativity techniquesforRE(3studies).Table3alsoshowsthatthe follow-ingtopicsareaddressedintwostudieseach:userparticipation, stakeholderidentification,goal-orientedRE,andrequirements elic-itationwhileallothertopics wereaddressedin onlyonestudy each.
WefoundthreeSRsthatdealtwithREartifacts:releaseplanning models(Svahnbergetal.,2010),causesofrequirementschanges (Banoet al.,2012), andrequirementserrors (Waliaand Carver, 2009).Regardingdevelopmentcontexts,wefoundthefollowing: distributed(2reviews–Eblingetal.,2009;Laietal.,2012), model-drivenRE(Loniewskietal.,2010),developmentofsoftwareproduct lines(Avesetal.,2010),andagile(Rachevaetal.,2009). Concern-ingdomains,fivedomainshavebeenaddressedsofar:multi-agent systems in two reviews (Blanes et al., 2009a,b), self-adaptive
Table2
Journalsandmagazines.
Journal/magazine Numberof
empiricalRE publications
RequirementsEngineeringJournal 66
InformationandSoftwareTechnology 39
TransactionsofSoftwareEngineering 38
JournalofSystemsandSoftware 36
IEEESoftware 20
EuropeanJournalofInformationSystems 16
EmpiricalSoftwareEngineeringJournal 14
JournalofSoftwareEngineeringandKnowledge Engineering
13 CommunicationsinComputers&InformationScience 11
AutomatedSoftwareEngineering 10
IEICEtransactionsonInformation&Systems 10
JournalofUniversalComputerScience 8
InformationSystems 8
IEEELatinAmericaTransactions 8
JournalofUniversalComputerScience 8
ACMTransactionsonSoftwareEngineeringand Methodology
7
InternationalJournalonHuman-ComputerStudies 7
SoftwarePracticeandExperience 6
JournalofSoftwareMaintenance 6
SoftwareMaintenanceandEvolution 6
SoftwareQualityJournal 6
IETJournalofSoftware 5
JisuanjiXuebao/ChineseJournalofComputers 5
InformationSystemsJournal 4
Information&Management 4
JournalofComputerInformationSystems 4
EnterpriseInformationSystems 4
InternationalJournalonSoftwareToolsforTechnology Transfer
4 JisuanjiYanjiuyuFazhan/ComputerResearchand
Development
4 InternationalJournalofInformationSystemModeling
andDesign
3
InformationSystemsande-BusinessManagement 3
ExpertSystemswithApplications 3
Knowledge-BasedSystems 3
JournalofIndustrialManagementDataandSystems 3
AppliedSoftComputingJournal 2
InformationSystemsFrontiers 2
InteractingwithComputers 2
ExpertSystems 2
JournalofResearchandPracticeinInformation Technology
2
JournalofBiomedicalInformatics 2
InnovationsinSystemsandSoftwareEngineering 2
HealthInformaticsJournal 2
InternationalJournalofSoftwareEngineeringandits Applications
2
InternationalReviewonComputersandSoftware 2
MISQuarterly 2
systems(Yangetal.,2014),cloudsystemsintworeviews(Mellado
etal.,2010;IankoulovaandDaneva,2012),service-oriented
sys-tems(Tekaetal.,2012),andvirtualrealitysystems(Santosetal., 2013).
Wealsofoundquiteafewreviews(5)thatdealtwiththeuseof supporttechnologyforRE:wikies(Laietal.,2012),recommender systems(Mohebzadaetal.,2012),requirementsmanagementtools (Reiner,2009), knowledge sharing platforms(Silaber and Breu, 2014) and technology transfer decision support (Ivarsson and
Gorschek,2009).However,inourexperience,itseemsthereare
onlytwoSRsthatinvestigatedevidenceconcerningapossible rela-tionship between useof a RE supporttool/techniques and the impacta tool/technique makeson its environment of use. The reviewofSillaberandBreu(2014)focusedonunderstandingthe evidencethat existed regardingthe impactofa particular type of infrastructure (namely knowledge sharing platforms) onRE
processesinaparticularcontext(namely,distributed),whilethe oneofAbrahaoetal.(2009)evaluatedtheevidenceontheuseof usabilitytechniquesforagent-basedsystems.Last,oneSR concern-ingtheadoptionofempiricalREina specificgeographicregion (Brazil)hasbeenpublished(Oliveiraetal.,2013).
3.2.2. ComparisonwithtwoREroadmappublications
In theREliterature, there aretwo publications that present REroadmaps(ChengandAttlee,2007;NuseibehandEasterbrook, 2000).Weusedthesepaperstocompareandcontrastthetopics
fromTable3(treatedinthesystematicreviewsandthemapping
studiesinRE)andthosetopicsputforwardintheroadmaps.This allowedustoseehowresearchevolvedovertimeandwhattopics remainedstillunder-researchedintermsofaggregatedevidence.
Table4presentsthosetopicsfromthetworoadmapsthathave
beenaddresseduntilnowandthosethatwerenotaddressedso far.TherightmostcolumninTable4suggeststhat6areascould possiblybenefitifweaggregatetheavailableempiricalresearch resultsproducedinprimarystudiesoftheareas:(1)requirements scaling,(2)REforself-managementsystems,(3)effectofsystem environmentonRE,(4)impactofREresearchonindustrypractice, (5)requirementsnegotiation,(6)conflictresolution.
3.2.3. Theoreticalperspectives
TheREasa disciplinegrewoutofmulti-facetedinteractions betweenpractitionersfromthecorporateworldandscholarsfrom SoftwareEngineering(SE)andInformationSystemsResearch(ISR) schools.Asamatteroffact,manyREtextbooksgrewoutof expe-riencesaccumulatedbypractitioners inlargecorporateprojects
(Lauesen,2002;Young,2001;RobinsonandRobinson,2012)and
manyothers grewoutofthecollectivelearningandwisdomof collaboratingREconsultingpractitionersandscholars(e.g. Alexan-derandBujic,2006;PohlandRupp,2011;MaidenandAlexander, 2007).Moreover,REscholarsbelongingtoSEresearchschoolswere instrumentaltoestablishingexperiments-focusedresearch prac-ticesandquantitativereasoninginempiricalREresearch.Atthe same time, ISR scholarshelped establishthe notionof REas a sociallyconstructedactivityandactivelysoughttoaddressthe var-iousconcernsofdeployingREtechniquesinreal-lifeorganizational settings.Theyalsoputforwardqualitativeresearchdesignpractices suchascase studytechniques(basedone.g.groundedtheories, focusgroups)thatfirstoriginatedinsocialsciences.Thisvarietyof backgrounds(industrypractitionersandscholarsfromSEandISR) addeduptothebreadthanddepthoftheconversationon empir-icalREresearchovertheyears,whichisreflectedinthevarietyof theoreticalperspectivesdeployedtothearea.
Toillustratethe varietyof theoreticallenses usedin empir-icalRE,welookedattheprimary studiesincludedinthethree mostrecentsystematicreviewsinREthatarepublishedin
jour-nals(Methetal.,2013;AbeleinandPaech,2013;Yueetal.,2011)
and thatprovideanexact countandreferencesoftheinvolved studies.Table5illustratesthepenetrationoftheoriesofother dis-ciplinesintoempiricalRE.Wenotethatthisapproachhasofcourse limitations(i.e.themostrecentreviewshaveastheirtopicsuser involvementwhichgenerallylendsitselftoinvestigationbyusing theoriesfrom social sciences); however,it serves thesole pur-posetoprovideageneralideaoftherangeoftheoreticallenses thatempiricalREresearchdesignsconsidered.InTable5,each col-umnindicatesthetheoriesthatwereusedbytheprimarystudies includedineachreview.
Aswewantedtoseethemotivationsforchoosingtheoriesand howthesewereusedinempiricalREdesigns,wecheckedthe pri-marystudiesincludedineachreview.Thefirstone(Methetal., 2013)included36studies,thesecond(AbeleinandPaech,2013)– 58studies,andthethird(Yueetal.,2011)–20studies,respectively. Thecellsthataremarkedwithastar(*)indicatethattheauthors
Table3
REtopicaddressedinsystematicliteraturereviewsandmappingstudies.
REtechniques/practices/frameworks/approachesstudied Numberof
reviews Requirementsprioritizationtechniques(HerrmannandDaneva,2008;Pitangueiraetal.,2013;Rinkelevicetal.,2013;PergherandRossi,2013) 4
Requirementsspecificationtechniques/notations(Tekaetal.,2012;DaSilvaandBenitti,2011;Condori-Fernandezetal.,2009;Amyotand
Mussbacher,2011)
4
Securityrequirements(IankoulovaandDaneva,2012;Souagetal.,2012;Mellado,2009;Goudarzietal.,2013) 4
CreativitytechniquesforRE(Sahaetal.,2012;Lemosetal.,2012;Nguyen,2009) 3
Userparticipationandclientinvolvement(BanoandZowghi,2013;AbeleinandPaech,2013) 2
Stakeholderidentification(PachecoandGarcia,2012;CarlaandIvan,2008) 2
Goal-orientedREframeworks(Ghanavatietal.,2011;Horkoffetal.,2014) 2
Requirementstriageandselection(Khurumetal.,2012) 2
Requirementselicitationtechniques(Davisetal.,2006;Ouhbietal.,2013) 2
Requirementsanalysistechniques(Yueetal.,2011;Aguilaretal.,2010) 2
Automatedrequirementselicitationtechniques(Methetal.,2013) 1
Requirementsmodelingandanalysistechniquesforself-adaptivesystems(Yangetal.,2014) 1
Tracingtechniques(Torkaretal.,2012) 1
GenerationofrequirementsspecificationfromSEmodels(NicolásandToval,2009) 1
Technologytransferdecisionsupportapproaches(Ivarsonetal.,2009) 1
Model-drivenREtechniques(Loniewskietal.,2010) 1
Transformationalapproachesbetweenuserrequirementsmodelsandanalysismodels(Yueetal.,2011) 1
Requirementsmanagementtools(Reiner2009) 1
UseofknowledgesharingplatformsforRE(SilaberandBreu,2014) 1
Knowledgecreation(Schneideretal.,2013) 1
Requirements-basedsoftwaretestingtechniques 1
Requirementsevolutionapproaches(Zhangetal.,2012) 1
RE-specificwikisfordistributedcontext(Laietal.,2012) 1
REtechniquesfordistributeddevelopmentprojects(Eblingetal.,2009) 1
Dataqualityrequirements(Guerra-Garciaetal.,2010) 1
Approachesfordeterminingbusinessvalueofrequirements(Rachevaetal.,2009) 1
Qualityrequirementsmanagementtechniques(Svenssonetal.,2010) 1
REtechniquesforproductderivation(Rabiser,2010) 1
Requirementsreusetechniques(DeAzambujaetal.,2009) 1
RiskandsafeguardpracticesinglobalRE(Lopezetal.,2009) 1
Techniquesforaligningrequirementsandtesting(Barmietal.,2011) 1
Usabilityrequirementselicitation(Orme ˜noandPanach,2013) 1
Practicesforconstructionhigh-qualityrequirementsmodels(El-AttarandMiller,2012) 1
RecommendersystemsforRE(Mohebzada,2012) 1
REtechniquesforsoftwareproductlines(Avesetal.,2010) 1
REeducation(Ouhbietal.,2013) 1
Causesofrequirementschange(Banoetal.,2012) 1
Releaseplanningmodels(Svahnbergetal.,2012) 1
Requirementserrors(WaliaandCarver,2013) 1
ofaprimarystudythatusedtherespectivetheory,motivatedtheir choiceforthistheory.
Last,thenumber inbrackets aftereach theoryindicatesthe numberof primarystudiesthat usedtherespectivetheory.For example,inthereviewofAbeleinandPaech(2013),nineprimary studiesusedthetheoreticalperspectiveofparticipatorydesignand motivatedwhytheychoseit.
Wenote thatin thesethree reviewsnotall primarystudies explicitlystatedthetheorytheyused.Also,someprimarystudies wereexploratoryinnatureandusedaqualitativeresearchmethod thatassumesnotheoryasastartingpointfortheempiricalresearch undertaken.
WethinkthatTable5suggestsapositivedevelopmentasRE ismultidisciplinaryandunderstandingitnecessarilyimpliesthe applicationofmultipleperspectives.Thetableindicatesthat the-oreticalperspectivesthatoriginatedintheComputerScienceand SoftwareEngineeringfieldaremostofthetimemotivatedwhen areputintouseinempiricalREstudies.However,our unsystem-aticchecksuggests thattheoriesthat are borrowedfromother disciplines(e.g.sociology,organizational behavior)weremostly referredto,withlittleornoexplicitdiscussiononhowthe con-ceptsofthetheoryweretranslatedintotheempiricalREsetting. Whileinsomecasesthismightbejustifiable,voicesfromtheESE communitywereraisedthattheriskofsuchapracticemightbe
Table4
TopicsinSRandmappingstudiesandthetworoadmappapers.
Roadmap Year TopicsaddresseduntilnowinSRs TopicsthatarenotaddresseduntilnowinSRs
Cheng,Attlee 2007 •Securityrequirements •Requirementsscaling
•GlobalRE •REforself-managementsystems
•DistributedRE •EffectofsystemenvironmentonRE
•REeducation •ImpactofREresearchonindustrypractice
Nuseibeh, Easterbrook
2000 •Techniquesforformallymodelingandanalyzing propertiesoftheenvironment
•Requirementsnegotiation
•Requirementselicitation •Conflictresolution
•Non-functionalrequirements •Reuseofmodels
Table5
Examplesoftheoriesusedinthreesystematicreviews.
Methetal.(2013);36studies AbeleinandPaech(2013);58studies Yueetal.(2011);20studies
Corpus-basedfrequencyprofiling* Technologyacceptancemodel Transformationalparadigm*
Signalprocessing* Usabilitymodel*(2) Object-orientedparadigm*(5)
Domainontology*(4) Organizationaltheory Linguisticanalysis*(2)
Symbolicandconnectionistparadigm Collaborationengineeringtheory Patterns*
Transformationalparadigm*(3) Technologydefusingtheory Relatedtriad
Probabilisticclassificationtheory* Marketingdiffusiontheory TwinPeaksmodel
Designrecoveryprocess Participatorydesign*(9) Metamodellingparadigm
Objectorientedparadigm*(4) Culturalprobestheory Formalgrammar
Datamining Empiricallearning Datatypetheory*
Organizationallearning* Actiontheory
Datastructures* Userconfigurationtheory
Problem-solutiondomainreasoning(2) User-centricdesignparadigm(3)
Case-basedreasoning Stakeholdertheory
Similarityanalysis*(2) Referenceframeworkforsoftwareprocessimprovement
Apprenticeshipmulti-strategylearningtheory Mediarichnesstheory(2)
Learningtheory Socio-technicaltheoreticallens
Unsupervisedclustering Contextualdesign
Classificationtheory Two-levelgrammar Patterntheory Graphtheory
significant.Asindicatedin(Simetal.,2001),approachesand theo-riesfromotherdisciplinescanrarelybeappliedwholesalewithout firststudyingtheirunderlyingassumptions.Ifwedonot explic-itlystatehowtheassumptionsofatheorymatchthesettingsto whichitisapplied,thenwemightcommitcriticalerrorsorposea seriousthreattovalidityoftheresultsobtained.Morediscussion, therefore,ontheassumptionsbehindchoosingandusingatheory fromanotherdiscipline,forthepurposeofREresearchwouldhelp readersofempiricalREpapersbetterunderstandandevaluatethe validityoftheresults.
3.3. Thisspecialissue
Forthisspecialissue,wecalledforcontributionsthat(1) evalu-atetechniquesfromESEforsuitabilityandinclusioninREstudies, or (2) address RE problems and solutions in new domains by deployingempiricalresearchmethods.Ourcallforpaperstothis special issue brought 17 submissions. All were subjected to a systematicmultiple-stagesreviewprocesswiththeengaged par-ticipationofatleastthreereviewers.Fivehigh-qualitypaperswere selectedforinclusioninthisissue.
Thepaper“Softwareproductmanagement–Anindustry evalu-ation”byChristofEbertandSjaakBrinkkemperfocusesonproduct managementasakeydriverforREandreportsonanfieldstudy withpractitionersfromfifteenorganizationsworldwide comple-mentedwithanindustrysurvey,concerningtheroleofproduct managersanditsrelationshiptoprojectsuccess.Thekeyfindingis thatincreasinginstitutionalizationofaconsistentandempowered productmanagement roleleadstoanimprovedsuccessrateof projectsin terms ofschedulepredictability,quality and project duration.
The paperof R.J. Wieringa, “Empiricalresearch methods for technologyvalidation:Scalinguptopractice”answersthe ques-tionofhowtogeneralizefromempiricalREvalidationresearchto futurepractice.TheauthorelaboratesonfourapproachesthatRE researchersmightconsiderforuseinempiricalresearchdesigns thataimatsimulatingfuturepracticaluseofREtechnology.These approaches areexpert opinions,single-case mechanism experi-ments,technicalactionresearchandstatisticaldifference-making experiments.
ThepaperofSamiJantunenandDonalGausse“Usingagrounded theoryapproachforexploringsoftwareproductmanagement chal-lenges”, demonstrates the use of a grounded theory research
methodinanexploratorystudyaboutmarket-drivenREpractices andchallenges.Theauthor’stheorizingeffortisfocusedon devel-opingatheoryproposalthatusedthenotionsofdesignproblems andparadigmshiftstoexplainwhythecompaniesparticipatingin thestudyexperiencedspecifichechallengesinmarket-drivenRE. ThepaperofJohanHoorn,“Stakeholderlogisticsofan interac-tivesystem”reportsonanempiricalstudythatusesthetheoretical constructsoftheStakeholderLogisticsformulatehypothesesabout therelationshipsbetweenfourvariables:Usability,Efficiencyand EffectivenessandSatisfaction.Usingempiricaldatafrom profes-sionalusersandprojectsinthebankingandheathcaresector,the authorcarriesoutahypothesestestingexercise.Theauthor’s find-ingsindicatethatEfficiencyandEffectivenessseemmoreimportant thanusabilityinexplainingwhystakeholdersaresatisfiedwitha systemofnot.
ThepaperofFabioMassacci,FedericaPaci,LeMinhSangTrana, and Alessandra Tedeschi, “Assessing a requirements evolution approach:Empiricalstudiesintheairtrafficmanagementdomain” reportsontheempiricalevaluationoftheeffectivenessofanovel approachformodelingandreasoningaboutrequirements evolu-tion.Part ofthestudyresponds tothequestionaboutwhether theeffectivenessdependsontheuser’slevelofknowledgeofthe approachandoftheapplicationdomain.Theauthorspresentthree quantitativestudiesinwhichhypothesesaretestedondata col-lectedfrom three differentgroups of participantsthat differin termsofexposuretotheapproachandtotheapplicationdomain.
4. Conclusions
Thiseditorialshowsthatempiricalapproachestothestudyof REphenomenahaveaccompaniestheREdisciplinesinceitsvery beginningandhavereceivedmuchattentionoverthepast20years, addinguptoabroadvarietyintermsofresearchmethodsdeployed, theoretical lenses and contextual settings. The RE community demonstratedremarkablecommitmenttodeployingtheoriesfrom otherdisciplinesinthedesignofempiricalREresearch. Aggregat-ingevidencefromempiricalREstudiesandgeneralizingknowledge claimshavebeenhoweverabumpyroad.Whileasteadyincrease inthenumberofempiricalstudieseachyearisapositive devel-opment,yettoabsorbthefullbenefitsoftheproducedempirical studies,empiricalresearchactivitiesneedtobecoupledwith reflec-tionsontheuseofthedeployedtheoriesandpossiblyembedthe learningandtheevidencefromtheREstudiesintotheprocessof
extendingtheexistingtheoriesfromotherdisciplinesorcreating newtheoriesspecifictotheareaofRE.
Compared to the research agendas outlined in the two RE roadmappublications(ChengandAttlee,2007;Nuseibehand East-erbrook,2000),wefoundthat:
1.TheREsub-areasforwhichempiricalREeffortswerefocused onaggregatingevidenceare:requirementselicitation, specifi-cation,prioritizationandtracingandaswellas‘user-front-end’ sub-areas as e.g. userinvolvement and stakeholder analysis. Techniquesfortheseareashave beenevaluatedbymeansof usingthesystematicliteraturereviewtechniques(Kitchenham, 2006).
2.Aggregating evidence about requirements validation tech-niques, requirements negotiation techniques (e.g. conflict resolution),requirementsscalingandREforsystemsofsystems stillremains tobedone. Effortstoconsolidateknowledge in thesesub-areascouldbealineforfutureresearch.
3.MostREempirical claim aggregation effortsfocused on pro-viding better answers to old questions (e.g. effectiveness of techniquesspecificto asub-area suchaselicitation, prioriti-zation). There is also a recent trend in evaluating evidence pertainingtousingREtechniquesinspecificnewcontextsof development(suchasdistributed).However,veryfewreviews indicatedanyfocusonnewapplicationdomainssuchasservice systems,internet-of-thingssystems,gameandhealth-care sys-tems. As theimportance of those is growing, we think that evaluatingtheevidenceprovidedbyempiricalREresearchin theseareasisaworthwhileendeavor.
4.Whiletheinterestinexplorationofusingtheoriesfromother disciplinesseemoverwhelming andleadstointenseresearch activity,verylittleseemstohavebeendoneonsharing experi-encesonhowtoborrowatheoryfromanotherdisciplineand putitinproductiveuseforthepurposeofempiricalREresearch. Neitherthereseemtobeanyefforttoaggregatethecollective learningabouttheuseoftheories.Reflectionontheoryuse there-foreisarelevantandworthwhileactivityforthefuture.
Acknowledgements
We appreciate very much the time and effort of those of EmpiRE’12 Program Committee members who also served as reviewers to this Special Issue: Dan Berry, Nelly Condori-Fernandez, Daniela Cruzes, Oscar Dieste, Joerg Doerr, Andrea Herrmann, Eric Knauss, Olga Ormanjieva,Anna Perini, Norbert Seyff.We arealsoindebtedtothefollowingexternalreviewers: NorahPower,RichardBerntssonSvenson,SergioEspa ˜na,Dietmar Pfahl,VirginiaLealFranqueira,NourAli,LuigiBuglione,Mauricio Aguiar,KlaasSikkel,MarijoKauppinen,KaiPetersen,HelenSharp, JuneVerner,NilsBredeMoe,SivaDorairaj,FelixGarcia,Marcela Genero,KalleLyytinen,SabrinaMarczak,PeteSawyer,Stefan Wag-ner,MariaTeresaBaldassarre,RaimundasMatulevicius,Miroslaw Staron,VeraWerneck.
OurgratitudegoestotheJSSEditorinChiefHansvanVlietforhis promptresponseandadviceonaveryshortnoticewhiledealing withallpracticalaspectsintheprocessofpreparingthisspecial issue.
References
Abelein,U.,Paech,B.,2013.Understandingtheinfluenceofuserparticipationand involvementonsystemsuccess–asystematicmappingstudy.Empir.Softw. Eng.,1–54.
Barmi,Z.A.,Ebrahimi,A.H.,Feldt,R.,2011.Alignmentofrequirementsspecification andtesting:asystematicmappingstudy.ICSTW,476–485.
Blanes,D.,Insfran,E.,Abrahão,S.,2009a.Requirementsengineeringinthe develop-mentofmulti-agentsystems:asystematicreview.IDEAL,510–517.
Blanes,D.,Insfran,E.,Abrahão,S.,2009b.Reviewingtheuseofrequirements engi-neeringtechniquesinthedevelopmentofmulti-agentsystems.IWANN(2), 134–137.
Carla,P.,Ivan,G.,2008.Stakeholderidentificationmethodsinsoftware require-ments:empiricalfindingsderivedfromasystematicreview.ICSEA,472–477.
Condori-Fernandez,N.,Daneva,M.,Sikkel,K.,Wieringa,R.,Dieste,O.,Pastor,O.,2009.
Asystematicmappingstudyonempiricalevaluationofsoftwarerequirements specificationstechniques.ESEM,502–505.
DaSilva,R.C.,Benitti,F.B.V.,2011.Writingstandardsrequirements:asystematic literaturemapping.In:14thWorkshoponRequirementsEngineering,WER, pp.259–272.
Damian,D.,2007,March/April.Stakeholdersinglobalrequirementsengineering: lessonslearnedfrompractice.IEEESoftw.
Damian,D.,Moitra,D.,2006,September/October.Globalsoftwaredevelopment: howfarhavewecome?GuestEditors’introduction,specialissueonglobal softwaredevelopment.IEEESoftw.23(5).
Daneva,M.,vanderVeen,E.,Amrit,C.,Ghaisas,S.,Sikkel,K.,Kumar,R.,Ajmeri, N.,Ramteerthkar,U.,Wieringa,R.J.,2013.Agilerequirementsprioritizationin large-scaleoutsourcedsystemprojects:anempiricalstudy.J.Syst.Softw.86(5), 1333–1353.
Davis,J.,1983.TransferofautomatedRequirementsengineeringtooltechnology –acasestudy.In:IEEEComputerSocietyWorkshoponSoftwareEngineering TechnologyTransfer,MiamiBeach,FL,USA,ISBN:0818604689.
Davis,A.,Dieste,O.,Hickey,A.,Juristo,N.,Moreno,A.M.,2006.Effectivenessof requirementselicitationtechniques:empiricalresultsderivedfromasystematic review.RE,176–185.
DeAzambuja,F.B.,Bastos,R.M.,Bacelo,A.P.T.,2009.Systematicreviewof require-mentsreuse.SEKE,67–72.
Easterbrook,S.,2007a.TesttheTheory,nottheTool,FifthWorkshoponCERE2007, inconjunctionwithRE’07.
Easterbrook, S., 2007b. Empirical Research Methods in Requirements Engi-neering, Tutorial at RE’07. http://www.cs.toronto.edu/∼sme/presentations/ re07tutorial-vPrint.pdf
Ebling,T.,Audy,J.L.N.,Prikladnicki,R.,2009.ASystematicLiteratureReviewof
RequirementsEngineeringinDistributedSoftwareDevelopmentEnvironments. PaperpresentedattheICEIS2009–11thInternationalConferenceonEnterprise InformationSystems,Proceedings,ISAS,pp.363–366.
El-Attar,M.,Miller,J.,2012.Constructinghighqualityusecasemodels:asystematic
reviewofcurrentpractices.Requir.Eng.17(3),187–201.
Gervasi,V.,Zowghi,D.,Easterbrook,S.,Sim,S.E.,2004.Reportonthefirst
interna-tionalworkshoponcomparativeevaluationinrequirementsengineering.ACM SIGSOFTSoftw.Eng.Notes29(2),1–4.
Ghanavati,S.,Amyot,D.,Peyton,L.,2011.Asystematicreviewofgoal-oriented
requirements management frameworks for business process compliance. RELAW,25–34.
Goudarzi,S.,Abdullah,A.H.,Mandala,S.,Soleymani,S.,RezazadehBaee,M.A.,
Anisi,M.H.,Aliyu,M.,2013.Asystematicreviewofsecurityinvehicularad
hocnetwork.In:2ndSymposiumonWirelessSensorsandCellularNetworks
(WSCN’13),Jeddah,SaudiArabiahttp://wscn2013.com/assets/16.pdf
Herrmann,A.,Daneva,M.,2008.Requirementsprioritizationbasedonbenefitand
costprediction:anagendaforfutureresearch.RE,125–134.
Horkoff,J.,Li,T.,Li,F.L.,Pimentel,J.,Salnitri,M.,Cardoso,E.,Giorgini,P.,Mylopoulos,
J.,2014.TakingGoalModelsDownstream:ASystematicRoadmap,RCIS’14.
Iankoulova,I.,Daneva,M.,2012.Cloudcomputingsecurityrequirements:a
system-aticreview.RCIS,1–7.
Ivarsson,M.,Gorschek,T.,2009.Technologytransferdecisionsupportin
require-mentsengineeringresearch:asystematicreviewofREj.Requir.Eng.14(3), 155–175.
Khurum, M., Uppalapati, N., Veeramachaneni, R.C., 2012. Software
require-mentstriageandselection:state-of-the-artandstate-of-practice.APSEC1, 416–421.
Kitchenham,K.A.,Dybå,T.,Jørgensen,M.,2004.Evidence-basedsoftware
engineer-ing.ICSE,273–281.
Lauesen,S.,2002.SoftwareRequirements–StylesandTechniques.Addison-Wesley.
Lemos,J.,Alves,C.,Duboc,L.,Rodrigues,G.N.,2012.Asystematicmappingstudyon
creativityinrequirementsengineering.SAC,1083–1088.
Loniewski,G.,Insfran,E.,Abrahão,S.,2010.Asystematicreviewoftheuseof
requirementsengineeringtechniquesinmodel-drivendevelopment.MoDELS (2),213–227.
Mellado,D.,Blanco,C.,Sánchez,L.E.,Fernández-Medina,E.,2010.Asystematic
reviewofsecurityrequirementsengineering.Comput.Stand.Interfaces32(4), 153–165.
Meth,H.,Brhel,M.,Maedche,A.,2013.Thestateoftheartinautomatedrequirements
elicitation.Inform.Softw.Technol.55(10),1695–1709.
Mohebzada, J.G., Ruhe,G.,Eberlein, A., 2012. SystematicMapping of
Recom-mendationSystemsforRequirements Engineering.PaperPresented atthe 2012InternationalConferenceonSoftwareandSystemProcess,ICSSP2012– Proceedings,pp.200–209.
Nicolás,J.,Toval,A.,2009.Onthegenerationofrequirementsspecificationsfrom
softwareengineeringmodels:asystematicliteraturereview.Inform.Softw. Technol.51(9),1291–1307.
Oliveira,K.,Pimentel,J.,Santos,E.,Dermeval,E.,Guedes,G.,Souza,C.,Soares,M.,
Castro,J.,Alencar,F.,Silva,C.,2013.25YearsofRequirementsEngineeringin
Brazil:ASystematicMapping.WER’13.
Orme ˜no,Y.I.,Panach,J.I.,2013.Mappingstudyaboutusabilityrequirements
Ouhbi,S.,Idri,A.,Fernández-Alemán,J.L.,Toval,A.,2013.Requirementsengineering education:asystematicmappingstudy.Requir.Eng.,1–20.
Pacheco, C., Garcia, I., 2012. A systematic literature review of stakeholder
identification methods in requirements elicitation. J. Syst. Softw. 85 (9), 2171–2181.
Pergher,M.,Rossi,B.,2013.RequirementsPrioritizationinSoftwareEngineering:
ASystematicMappingStudy.PaperPresentedatthe20133rdInternational WorkshoponEmpiricalRequirementsEngineering,EmpiRE2013–Proceedings, pp.40–44.
Pitangueira,A.M.,Maciel,R.S.P.,DeOliveiraBarros,M.,Andrade,A.S.,2013.A
sys-tematicreviewofsoftwarerequirementsselectionandprioritizationusingSBSE approaches.Lect.NotesComput.Sci.
Pohl,K.,Rupp,C.,2011.RequirementsEngineeringFundamentals:AStudyGuidefor
theCertifiedProfessionalforRequirementsEngineeringExam.RockyNook.
Racheva,Z.,Daneva,M.,Sikkel,K.,2009.Valuecreationbyagileprojects:
method-ologyormystery?PROFES,141–155.
Robinson,S.,Robinson,J.,2012. Mastering theRequirementsProcess:Getting
RequirementsRight.Addson-Wesley.
Saha,S.K.,Selvi,M.,Büyükcan,G.,Mohymen,M.,2012.ASystematicReviewon
Cre-ativityTechniquesforRequirementsEngineering.PaperPresentedatthe2012 InternationalConferenceonInformatics,ElectronicsandVision,ICIEV2012, pp.34–39.
Shaul,L.,Tauber,D.,2013.Criticalsuccessfactorsinenterpriseresourceplanning
systems:reviewofthelastdecade.ACMComput.Surv.45(4),paper55.
Souag,A.,Salinesi, C.,Wattiau, I.,2012.Ontologies forsecurityrequirements:
a literature surveyand classification.In: 2ndInternational Workshopon
Information Systems Security Engineering WISSE’12 in Conjunction with
the24thInternationalConferenceonAdvancedInformationSystems
Engi-neering(CAiSE’12),CAiSE,http://hal.inria.fr/docs/00/70/92/29/PDF/Ontologies
forSecurityRequirementsliteraturesurvey.pdf
Svahnberg,M.,Gorschek,T.,Feldt,R.,Torkar,R.,Saleem,S.B.,Shafique,M.U.,2010.A
systematicreviewonstrategicreleaseplanningmodels.Inform.Softw.Technol. 52(3),237–248.
Svensson,R.B.,Höst,M.,Regnell,B.,2010.Managingqualityrequirements:A
sys-tematicreview.In:36thEUROMICROSEAA,pp.261–268.
Teka,A.Y.,Condori-Fernandez,N.,Sapkota,B.,2012.Asystematicliteraturereview
onservicedescriptionmethods.REFSQ,239–255.
Torkar,R.,TonyGorschek,T.,RobertFeldt,R.,Svahnberg,M.,Raja,U.A.,Kamran,K.,
2012.Requirementstraceability:asystematicreviewandindustrycasestudy.
IJSoftw.Eng.Knowl.Eng.22(3),385–434.
Wagner,S.,Ruhe,M.,2008.ASystematicReviewofProductivityFactorsinSoftware
Development.SoftwareProductivityAnalysisandCostEstimation.TUM.
Wieringa,R.,2005.Requirementsresearchers:arewereallydoingresearch?Requir.
Eng.10(4),304–306.
Wieringa,R.,Maiden,N.,Mead,N.,Rolland,C.,2005.AClassificationofREPapers:Are
WeResearchingorDesigningRETechniques?ThirdWorkshoponCERE2005, inConjunctionwithRE’05.
Wieringa,R.,Maiden,N.,Mead,N.,Rolland,C.,2006.Requirementsengineering
paperclassificationandevaluationcriteria:aproposalandadiscussion.Requir. Eng.11(1),102–107.
Yang,Z.,Li,Z.,Chen,Y.,Jin,Z.,2014.Asystematicliteraturereviewofrequirements
modelingandanalysisforself-adaptivesystems.REFSQ’14.
Young,R.,2001.EffectiveRequirementsPractices.Addison-Wesley.
Furtherreading
Achimugu,P.,Selamat,A.,Ibrahim,R.,Mahrin,N.,2014.Asystematicliterature
reviewofsoftwarerequirementsprioritizationresearch.Inform.Softw.Technol. 56(6),568–585.
Alexander,I.,Maiden,F.N.,2004.Scenarios,Stories,UseCases:ThroughtheSystems
DevelopmentLife-Cycle.Wiley.
Appan,R.,Browne,G.J.,2012.Theimpactofanalyst-inducedmisinformationonthe
requirementselicitationprocess.MISQuart.36(1),85–106.
Hassine,J.,2014.Earlymodelingandvalidationoftimedsystemrequirementsusing
TimedUseCaseMaps.Requir.Eng.,1–31.
Jiang,L.,Eberlein,A.,Far,B.H.,2008.Acasestudyvalidationofaknowledge-based
approachfortheselectionofrequirementsengineeringtechniques.Requir.Eng. 13(2),117–146.
Li,J.,Zhang,H.,Zhu,L.,Jeffery,R.,Wang, Q.,Li,M.,2012. Preliminaryresults
ofa systematic review onrequirementsevolution. IET Semin.Digest (1), 12–21.
Orme ˜no,Y.I.,Panach,J.I.,Pastor,O.,2012.UsabilityRequirementsElicitation:An
OverviewofaMappingStudy.PaperPresentedattheACMInternational Con-ferenceProceedingSeries.
Rin¸k¸eviˇcs,K.,Torkar,R.,2013.Equalityincumulativevoting:asystematicreview
withanimprovementproposal.Inform.Softw.Technol.55(2),267–287. MayaDaneva,PhD,isAssistantProfessorintheInformationSystemsandSoftware
EngineeringgroupattheUniversityofTwente,theNetherlands.Herkeyresearch
interestsareempiricalsoftwareengineering,requirementsengineeringforlarge
systems,requirements-basedprojectestimation,andqualitativeresearchmethods,
suchasgroundedtheoryandfocusgroups.Mayahasastronginternationalexposure
havingspenttwoyearsofhercareerinGermanyattheUniversityofSaarbruecken
andintheIDSScheer,and9yearsasabusinessprocessanalystatTELUS
Corpora-tion,Canada’ssecondlargesttelecommunicationcompany.Shehasbeenaleading
memberofseveralindustry-universityresearchprojectsandservesastheliaisonto
theindustrymembersoftheDutchSoftwareMeasurementAssociation(NESMA).
MayaservesastheUniversityofTwente’srepresentativetoISERN,theInternational
EmpiricalSoftwareEngineeringResearchNetwork.Shehaspublishedmorethan15
empiricalREstudieswiththeREandREFSQconferencesandwiththeESEM
sym-posium.Since2012,MayahasbeenservingasthegeneralchairoftheEmpiricalRE
(EmpiRE)workshopatRE.
DanielaDamianisaProfessorofSoftwareEngineeringinUniversityofVictoria’s
DepartmentofComputerScience,wheresheleadsresearchintheSoftware
Engi-neeringGlobalinterActionLaboratory(SEGAL,segal.uvic.ca).Herresearchinterests
includeSoftwareEngineering, RequirementsEngineering, Computer-Supported
CooperativeWorkandEmpiricalSoftwareEngineering.Herrecentworkhas
stud-iedthedevelopers’socio-technicalcoordinationinlarge,geographicallydistributed
softwareprojects,aswellasstakeholdermanagementinlargesoftwareecosystems.
Daniela’sresearchmethodologiesinvolveextensivefieldworkandinsitustudiesof
softwareteamsthroughcollaborationswithindustrialpartnerssuchasIBM,
Gen-eralMotors,SiemensandDell.Danielahasservedontheprogramcommitteeboards
ofseveralsoftwareengineeringconferences,wastheprogramco-chairfortheFirst
InternationalConferenceonGlobalSoftwareEngineering(ICGSE06),andaguest
editoroftheIEEESoftwareSpecialIssueonGlobalSoftwareEngineering(2006).
SheiscurrentlytheCo-ChairfortheSoftwareEngineeringinSocietyTrackatICSE
2015andservingontheeditorialboardsofTransactionsonSoftwareEngineering,
theJournalofRequirementsEngineering,istheRequirementsEngineeringArea
Edi-torfortheJournalofEmpiricalSoftwareEngineering,andtheHumanAspectsArea
EditorfortheJournalofSoftwareandSystems.
AlessandroMarchettoiscurrentlyanindependentresearcherworkinginthefield
ofSoftwareEngineering.HereceivedhisPhDdegreeinSoftwareEngineeringfrom
theUniversityofMilano,Italyin2007.From2006tilltheendof2012hewas
aresearcherattheCenterforInformationTechnology(CIT)oftheBrunoKessler
FoundationinTrento,Italy,workingwiththeSoftwareEngineeringgroup.His
pri-maryresearchinterestsconcernSoftwareEngineeringand,inparticular,include
quality,verificationandtestingofSoftwareSystemsandofInternet-basedsystems.
Hepublishedmorethan70papersinprimaryinternationalconferencesand
jour-nals.Heregularlyreviewspapersforinternationalconferences(e.g.ICSM,CSMR,
WCRE)andjournals(e.g.STTT,JSS,IET).Hecollaboratedtotheorganizationof
morethanteninternationalscientificevents(e.g.SSBSE2012,SCAM2012,EmpiRE
2011–2012–2013,WSE2008–2012).
OscarPastorisFullProfessorandDirectorofthe“CentrodeInvestigaciónen
Méto-dosdeProduccióndeSoftware(PROS)”attheUniversidadPolitécnicadeValencia
(Spain).HereceivedhisPh.D.in1992.HewasaresearcheratHPLabs,Bristol,UK.He
haspublishedmorethan200researchpapersinconferenceproceedings,journals
andbooks,receivednumerousresearchgrantsfrompublicinstitutionsandprivate
industry,andbeenkeynotespeakeratseveralconferencesandworkshops.Chair
oftheERSteeringCommittee(2009–2010),andmemberoftheSCofconferences
asCAiSE,ER,ICWE,ESEM,CIbSEandRCIS,hisresearchactivitiesfocuson
concep-tualmodeling,webengineering,requirementsengineering,informationsystems,
model-basedsoftwareproduction,andempiricalsoftwareengineering.Hecreated
theobject-oriented,formalspecificationlanguageOASISandthecorresponding
softwareproductionmethodOO-METHOD.Heledtheresearchanddevelopment
underlyingCARETechnologiesthatwasformedin1996.CARETechnologieshas
createdanadvancedMDA-basedConceptualModelCompilercalledIntegranova,a
toolthatproducesafinalsoftwareproductstartingfromaconceptualschemathat
representssystemrequirements.Heiscurrentlyleadingamultidisciplinaryproject
linkingInformationSystemsandBioinformaticsnotions,orientedtodesigningand
implementingtoolsforConceptualModeling-basedinterpretationoftheHuman
