Amsterdam University of Applied Sciences
The interpretation of forensic conclusions by criminal justice professionals: The same evidence interpreted differently
van Straalen, Elmarije K. ; de Poot, Christianne J. ; Malsch, Marijke; Elffers, Henk DOI
10.1016/j.forsciint.2020.110331 Publication date
2020
Document Version Final published version Published in
Forensic Science International License
CC BY
Link to publication
Citation for published version (APA):
van Straalen, E. K., de Poot, C. J., Malsch, M., & Elffers, H. (2020). The interpretation of forensic conclusions by criminal justice professionals: The same evidence interpreted differently. Forensic Science International, 313, 1-11. [110331].
https://doi.org/10.1016/j.forsciint.2020.110331
General rights
It is not permitted to download or to forward/distribute the text or part of it without the consent of the author(s) and/or copyright holder(s), other than for strictly personal, individual use, unless the work is under an open content license (like Creative Commons).
Disclaimer/Complaints regulations
If you believe that digital publication of certain material infringes any of your rights or (privacy) interests, please let the Library know, stating your reasons. In case of a legitimate complaint, the Library will make the material inaccessible and/or remove it from the website. Please contact the library:
https://www.amsterdamuas.com/library/contact/questions, or send a letter to: University Library (Library of the University of Amsterdam and Amsterdam University of Applied Sciences), Secretariat, Singel 425, 1012 WP Amsterdam, The Netherlands. You will be contacted as soon as possible.
Download date:26 Nov 2021
The interpretation of forensic conclusions by criminal justice professionals: The same evidence interpreted differently
ElmarijeK.van Straalena,b,*,Christianne J.de Poota,b,c,MarijkeMalschd,Henk Elffersd
aAmsterdamUniversityofAppliedSciences,ForensicSciences,P.O.Box1025,1000BAAmsterdam,theNetherlands
bVUUniversityAmsterdam,CriminologyDepartment,DeBoelelaan1105,1081HVAmsterdam,theNetherlands
cPoliceAcademyoftheNetherlands,Apeldoorn,theNetherlands
dNetherlandsInstitutefortheStudyofCrimeandLawEnforcementNSCR,Amsterdam,theNetherlands
ARTICLE INFO
Articlehistory:
Received17October2019
Receivedinrevisedform4May2020 Accepted7May2020
Availableonline13May2020
Keywords:
Forensicconclusions Criminaljusticeprofessionals Evidenceinterpretation Strengthofevidence Communicatinguncertainty
ABSTRACT
Forensicreportsusevarioustypesofconclusions,suchasacategorical(CAT)conclusionoralikelihood ratio(LR).Inordertocorrectlyassesstheevidence,usersofforensicreportsneedtounderstandthe conclusion and itsevidentialstrength. Theaimofthis paperis tostudytheinterpretationof the evidentialstrengthofforensicconclusionsbycriminaljusticeprofessionals.Inanonlinequestionnaire 269 professionals assessed 768 reports on fingerprint examination and answered questions that measuredself-proclaimedandactualunderstandingofthereportsandconclusions.Thereportsentailed CAT,verbalLRandnumericalLRconclusionswithloworhighevidentialstrengthandwereassessedby crimesceneinvestigators,policedetectives,publicprosecutors,criminallawyers,andjudges.Theresults showthataboutaquarterofallquestionsmeasuringactualunderstandingofthereportswereanswered incorrectly. The CATconclusion was best understood for the weakconclusions, the three strong conclusionswereallassessedsimilarly.TheweakCATconclusioncorrectlyemphasizestheuncertaintyof anyconclusiontypeused.However,mostparticipantsunderestimatedthestrengthofthisweakCAT conclusioncomparedtotheotherweakconclusiontypes.Lookingattheself-proclaimedunderstanding ofallprofessionals,theyingeneraloverestimatedtheiractualunderstandingofallconclusiontypes.
©2020TheAuthors.PublishedbyElsevierB.V.ThisisanopenaccessarticleundertheCCBYlicense (http://creativecommons.org/licenses/by/4.0/).
1.Introduction
In the process of investigating and proving a crime, many different types of evidence may play a role, including witness statements, DNA, digital traces, fingerprints, observations, and shoeprints.Itisimpossible,andundesirable,forallprofessionals workingwithormakingdecisionsbasedonthisevidencetohave expertiseinallthesedifferentareas.Intheforensicdomain,there isadifferentexpertforeverytypeofevidencewhowillexamineit anddescribehisorherfindingsandconclusion(s)inareportthatis used by the professionals in the criminal justice system. For adequate functioning of the criminal justice process, three elementsareimportant:(1)thereportisclear,soundandcorrect, (2)theprofessionalhasabasicunderstandingofthecontentofthe evidenceandthereport,and(3)theprofessionalisabletocorrectly assesstheevidentialstrengthoftheconclusion.
The first element actually means that every expert who investigates the same traces would come up with the same results.ThestudybyDeKeijseretal.[1]shows,however,thatthere canbeadiscrepancyinthecontentandlayoutofforensicreportsof differentexperts.Theyasked19DNAexpertsaroundtheworld from 13 different forensic laboratories in seven countries to evaluatethesamemixedDNAprofilesandcaseinformationand write a DNA reportin thewaythey usuallydo in theiractual practice. The resulting reports differed in size, explanation of technicalissues,useofexplanationinappendices,evidencelevelof theproposition,useofcontextualinformation,andthetypeand contentoftheconclusions.Thisimpliesthatreportsonthesame DNA mark may be strikingly different and may yield different conclusions. It also gives a strong indication of the inherent uncertaintiesofincompleteandmixedDNAprofiles[1].Inastudy byDrorandHampikian[2]17DNAexpertsworkinginthesame forensiclaboratorywerepresentedwithmixtureDNAandtheDNA profilesofsuspectsfromarealadjudicatedcriminalcase.Basedon thepresentedevidence,the17independentDNAexaminersvaried intheirconclusionsfrom‘cannotbeexcluded’to‘excluded’and
‘inconclusive’.Bothstudies[1,2]showthatthesameDNAevidence can be assessed differently by DNA experts, regardless of the
* Correspondingauthorat:AmsterdamUniversityofAppliedSciences,Forensic Sciences,P.O.Box1025,1000BAAmsterdam,theNetherlands
E-mailaddress:e.k.van.straalen@hva.nl(E.K. vanStraalen).
http://dx.doi.org/10.1016/j.forsciint.2020.110331
0379-0738/©2020TheAuthors.PublishedbyElsevierB.V.ThisisanopenaccessarticleundertheCCBYlicense(http://creativecommons.org/licenses/by/4.0/).
ContentslistsavailableatScienceDirect
Forensic Science International
j o u r n a l h o m e p a g e : w w w . e l s e v i e r . c o m / l o c a te / fo r s c i i n t
forensiclaboratorytheyareworkingin.Forfingerprintevidence, researchhasshownpossiblebiasinexperts’conclusionsaswell [3,4]. An example of a mistaken identification by a fingerprint expertis theMayfield case. On March11,2004, several bombs explodedoncommutertrains inMadrid, eventuallykilling 191 peopleandwoundingover200.Thefingermarksleftonabagwith detonatorsconnectedwiththeattackwerebelievedtomatchthe fingerprintsof BrandonMayfield. This fingerprintidentification wasmadebythreemembersoftheFBILaboratory’sLatentPrint Unit.AfterMayfieldhadbeendetainedfortwoweeks,theSpanish NationalPolicearrestedanAlgeriannationalwhomtheyidentified asthesourceofthefingermarks. Heturnedouttobe‘a better match’thanMayfield[5].Theabovestudiesshowthatfindingsof forensic investigations can be interpreted differently. Research thereforeisnecessarytogainbetterunderstandingofwhyexperts candrawdifferentconclusionsaboutthesameevidence.Theusers of forensic reports need to be aware of this human factor in decision-making.Besidestheassumptionthatreportsarecorrect, it is also important that a report clearly states what context informationwasusedfortheassessmentandtheconclusion,and what context information was left out of the assessment. By describingthis,itispossiblefortheusersofthereportstomakea thought-throughdecisionbasedonallavailableevidence.
Forthesecondelement,professionalshavetobeabletoassess themeaningoftheevidenceintheforensicreport.Iftheylackthe necessarybackgroundtounderstandaforensicreport,theymay undertaketrainingor gain advice fromanindependent source.
Several studies [6–8] have shown that criminal justiceprofes- sionalshaveaquitehighlevelofself-proclaimedunderstandingof forensicreports,whichishigherthantheiractualunderstanding.
To understand that they need to seek training or advice, professionalsneedtobeawareoftheirlackofknowledge.Besides understandingthecontentofareport,professionalsalsohavetobe awareofthepossiblesubjectivityofexpertsassessingtheevidence andwritingthereportandconclusion.
Thethirdelementistheassessmentoftheevidentialstrength oftheconclusion.Doprofessionalshaveacorrectunderstandingof forensicconclusions?Dotheyunderstandtheevidentialstrength and the degree of uncertainty that are expressed in forensic conclusions?Dodifferenttypes of professionalshave thesame understandingoftheseforensicconclusions?Finally,dodifferent formulations that can be used to verbalise forensic evidence actuallyhavethesamemeaning?
The main focus in this article is this third element, the understanding of forensic conclusions by criminal justice pro- fessionals.Itisvaluabletoknowwhethertheevidentialstrengthof differentformulationsofthesameconclusionis assessedinthe samewaybydifferentprofessionalsinthecriminaljusticesystem.
Additionally,this paper examinesprofessionals’self-proclaimed understandingofforensicconclusions.
1.1.Understandingthe(un)certaintyandevidentialvalueofforensic conclusions
Forensic conclusions should always entail some degree of uncertainty,becausenoabsolutecertaintyabouttheuniquenessof tracesexists.Toillustratethis,wewillusefingerprintevidenceas anexample.Althoughwehavesomeknowledgeofthefactorsthat influencethedevelopment offingerprints[9], we do notknow exactlyhowfrictionridgepatternsoriginate.Sinceitispractically impossibletocompareallthefingerprintsof theentireworld’s population,therecanneverbe100%certaintyabouttheunicityof fingerprints.Furthermore,fingermarksfoundatacrimesceneare hardly ever of perfect quality and quantity. A fingerprint examinationisthereforeusuallybased onthecharacteristicsof onlypartofafingermark,whilethecharacteristicsofthemissing
part remain unknown. The Organization of Scientific Area Committees for ForensicScience (OSAC) hasdrafted guidelines for fingerprint examination conclusionsin which they state“A conclusion shall not be communicated as a fact. It is an interpretation of observationsmade by theexaminer and shall beexpressedasanexpertopinion",[10]. Uncertaintyshouldbe expressed in all forensic conclusions. When it comes to the interpretationoftheuncertaintyexpressedinforensicconclusions, several typesofmistakescanbemade.Koehler [11] providesa detaileddescriptiononpossiblefallacieswithinforensicdecision making. We will discuss a few of these fallacies here. The prosecutor’s fallacy and the defence fallacy are two common mistakesinthejudgementofprobabilities[6,12].Wewillexplain thesefallaciesusingthelikelihoodratio(LR),whichrepresentsthe expert’sviewoftherelativeprobabilityoftheobservedfeaturesof atraceunderalternativehypothesesaboutthesourceofthetrace.
Anexpertstates:
‘Therewere12correspondingminutiaebetweenthefingermarkandthe referencefingerprintofsuspectB.Smith,nodifferenceswerefound.Two hypotheseshavebeenformulated.
Hypothesis1:Thefingermarkoriginatedfromthesuspect.
Hypothesis2:Thefingermarkoriginatedfromanunknownperson.
Thefindingsofthisexaminationare5milliontimesmoreprobablewhen Hypothesis1istrueversuswhenHypothesis2istrue’.
Prosecutor’sfallacy→Itis5milliontimesmoreprobablethatthefingermark belongstothesuspect.
Correctunderstanding→Thefindingsofthecomparisonare5milliontimes moreprobablewhenthefingermarkbelongstothesuspect.
Defencefallacy→Sincethetotalpopulationis17million,thechancesare greaterthatthefingermarkisfromsomeoneelseinthepopulationthanthat itisfromthesuspect.
Correctunderstanding→Notallofthese17millionpeoplehavethesame chancesofleavingafingermarkatthatspecificcrimescene(thosewhohave neverbeennearthecrimesceneorarephysicallynotcapableofenteringthe crimescene).
Anotherpossiblemistakeintheinterpretationoftheevidence occurs when evidence is incorrectly usedto answer evidential questions. When evaluating evidence, a certain hierarchy of propositionsisfollowed todistinguishthesourcelevel,activity level, and offence level. Information about the source level is insufficienttoanswerquestionsabouttheoffencelevel[13–15].1 Forexample,thefingerprintexaminationconclusionaboutsuspect B.Smithprovides onlyinformationaboutthesourcelevelasto whetherthefingermarkmightbelongtothesuspect.Althoughthe conclusion provides information about the likelihood that an objectistouchedbythesuspect,thisconclusiondoesnotprovide any information about theactivity or crimerelatedness of the fingermark.Moreevidenceisneededtodrawconclusionsabout the activity level (did B. Smith perform a certain activity?) or offencelevel(didB.Smithcommitthecrime?).InanarticlebyDe Rondeetal.[16]thedifferencebetweensourceandactivitylevelin relationtofingermarkevidenceismoreprofoundlyexplained.
1.2.Interpretationofforensicreportsandconclusions
Whatisknownabouttheinterpretationofforensicconclusions inpractice?Severalstudieshavebeenconductedonthistopic.
Forconclusionsexpressedwithaverbaldescriptor,thechoiceof words can influence its comprehension. In a study of the interpretation of probabilityphrases, Willems et al. [17] asked participantstoassessvariousphrasesbyassigningeach apoint
1 Inthecurrentstudy,weonlyfocusonthesourcelevel:ontheidentificationof thesourceofafingermark.
estimateona0–100%scale.Theresultsshowthattheinterpreta- tion differs greatly among the participants. For more extreme wordssuchas‘always’and‘never’,thisdifferenceisthesmallest [17].Mullenetal.[18]askedstudentstopointoutthestrengthofa forensicstatementcontainingoneoftenverbalscaledescriptors, suchas‘weaksupport’,‘moderatesupport’,and‘extremelystrong support’.Themajoritydidnotcorrectlyunderstandthemeaningof thetermsusedintheverbalscale.Theyattributedmorevaluethan theyshouldtotheweakerverbalconclusionsandlessvaluethan theyshouldtothestrongerconclusions.Carteretal.[19]examined the utility of additional conclusions to the current categorical evidencescaleusedforfingerprintexaminationsaswasproposed bytheFrictionRidgeSubcommitteeoftheOSAC.Thiscommittee suggests using a 5-conclusion scale, adding the conclusions
‘supportfordifferentsources’and‘supportforcommonsources’ tothecurrent‘exclusion’,‘inconclusive’,and‘identification’.The study shows that fingerprint examiners used these additional
‘supportfor’conclusionsabout35%ofthetime,whichsupportsthe needformorequalifiedconclusionsforfingerprintcomparisons.
Carteretal.[19]discussthetheoreticaldifferencebetweenusing categorical conclusions and likelihood ratio conclusions for fingerprint examinations. In a study by Arscott et al. [20]
participants were asked to read a brief case summary that included a shoeprint evidence statement containing a verbal expressionofthecomparisonconclusionandtoratetheperceived strength of the expression. The results showed that when the numericalvalueswereleftout,bothprofessionalsandlayperson perceivedthethreehighestgradationsintheverbalscalesimilarly.
Ingeneral,thestrongertheactualevidentialstrength,thestronger theparticipantsassumedtheevidentialstrengthtobe[20].These studies showthat theinterpretation of the evidential valueof verbaldescriptionsofprobabilitiescanvaryamongthoseassessing them,and thataddingverbalconclusions toexistingscalescan result in a rather differentevidential value chosen by forensic experts.
InreactiontoArscotetal.[20],BergerandStoel[21]wondered whetheritisnecessarytouseonlyverbalexpressionswhenthere istheLRtoaccompanyit.Wintleetal.[22]investigatedwhether the understanding of verbal probability expressions could be supportedbyprovidingnumbersalongsidetheseexpressions.Lay participants read statements containing verbal expressions of probabilities.Thenumericalprobabilitiesoftheverbalexpressions (veryunlikely,unlikely,likely,verylikely)werepresentedinthree differentways:inatableinaseparatebrowserwindow,inatooltip thatappearedwhenamousehoveredoverit,orbetweenbrackets behind theverbal expression.A fourth (control)group didnot receiveanynumericalprobabilities.Afterreadingthestatements, participantshad toestimate theminimum,best,andmaximum numerical probability belonging to the verbal expression. The resultsshowed a highvariabilityin thenumericalprobabilities assigned totheverbalprobabilities. Thisvariability waslowest whenthenumericalvaluewaspresentedinbracketsbehindthe verbalexpression[22].InamootcourtexerciseLangenburgetal.
[23]explainedandpresentedalikelihoodratiostatingthestrength of fingerprint evidence to a mock jury. The mock jurors were positiveaboutthecredibilityoftheevidenceandunderstoodmost of the testimonies. However, fingerprint experts watching the moot court exercise were less positive about the use of a probabilitymodelforfingerprintexaminations[23].Thesestudies showthatverbalprobabilitydescriptionscanbeambiguousand thatnumericalexpressionscanhelpreducethisambiguity.When explained clearly, jurors can be positive about the use and understandabilityofnumericalexpressions.However,whenusing a‘new’ typeof conclusionin anexistingfield ofexpertise,itis importanttonotonlyfocusontheusersoftheevidence,butalso ontheexpertsconductingthecomparisons.
Areforensicconclusionsassesseddifferentlydependingonthe way they are expressed? In the study by De Keijser et al. [6], professionals assessed reports on facial comparison and DNA evidenceinwhichtheconclusionwaspresentedinaverballyor visuallyexpressedLR.Theprofessionalsdidnotassessthereports and conclusionssignificantlydifferently. Otherstudies didfind differencesintheinterpretationofconclusionsdependingonthe waytheywereexpressed.McQuiston-SurrettandSaks[24]studied how jurors and judges assessedconclusions that werephrased differently.Thestudyshowedthatjurorsandjudgesconsidered theevidentialvalueofconclusionstobehigheriftheycontained thewords‘match’or‘similarinall ... characteristics’orifthey were described with an objective single-probability (the more qualitative conclusions) than if they were described with a subjectiveprobabilityoranobjectivemulti-frequencyconclusion (the more quantitativeconclusions)[24].In a study byMartire etal.[25],layparticipantsreceivedabriefcasesummarywiththe expert testimony of a fingerprint examiner about fingermarks foundatthecrimescene.Theexpertconclusion,whichhadalow orhighevidentialstrength,waspresentednumerically,verballyin atable,orwithavisualLR.Thelowevidentialstrengthconclusion that was expressed with a verbalLR was deemedthe weakest comparedtoalltheotherconclusions[25].
Thompsonetal.[26]askedjury-eligibleadultstoassesspairsof comparisonsentailingdifferentphrasingsofforensicconclusions onDNAorfingerprintexaminations. Theystudiedtheinterpre- tations of six differentexpression typeswith weakand strong evidential strength:LRs, strengthof supportstatements, match frequencies and random match probabilities, likelihood of observedsimilarity,sourceprobabilitystatements,andcategorical conclusions.In general,theyfoundthatstatementsdesignedto suggestthatthestrengthofevidencewaslowormoderatewere correctlyperceivedasweakerthanstatementsdesignedtosuggest thatthestrengthofevidencewashigh.Conclusionsphrasedwith the term‘match’ wereassessed as beingextremelystrong and assessedcomparabletoan‘LRof10million’.Conclusionsphrased withtheterms‘identification’wereassessedasbeingconsiderably strongerthanthosephrasedintermsofan‘individualisation’,but these were considered significantly weaker than conclusions phrased in terms of ‘match’ or ‘LR of 10 million’. Conclusions expressedwiththephrasing‘extremelystrong’wereconsideredto be weaker than those using the term ‘individualisation’, even thoughthisphrasingisintendedtobecomparabletoan‘LRof10 million’[26].Thedifferenttermsusedtoformulatetheforensic conclusionswereperceiveddifferentlythantheywereintended.
ResultsofthestudybyGarrett,Michell,andScurich(2018)show thatjuryeligibleadultsassessedcategoricalconclusionssimilaras match probabilities ranging from 1.000.000 to 10. Participants were similar in their decision to convict a suspect based on fingerprint evidence, regardless the conclusiontype or specific strengthoftheevidence.Ratingsofthelikelihoodthedefendant left prints and committed the crime differed more between participants.Althoughthoselikelihoodswereingeneralassessed asbeingslightlyhigherforthecategoricalconclusions,overallthey wereassessedsimilarlycomparedtothehighestmatchprobabili- ties.Thehighestpresentedprobability(1.000.000)wasassessedas having a significantly higher likelihood compared tothe other probabilities,butthelowerprobabilitiesrangingfrom100.000to 10 were assessed similarly. Bayer et al. [27] studied the interpretation by jurors of differentlikelihood ratio presenting methods:onlyLR,LRwithconversiontable,LRwithconversion table and figure relating prior and posterior probabilities, and match and nonmatch control groups. In general, participants overestimated the guilt of the suspect prior to the forensic evidence,andtendedtoundervaluetheweightoftheevidence.
Mostunderestimationandoverestimationoftheguiltwasfound
