Specific source vs. Common source
reference scores to determine the
evidential strength of Glock
aperture shear marks
Research Project - Thesis I.R. van Gilse Master Forensic Science University of Amsterdam Den Haag, 17-12-2018
2
Specific source vs. Common source reference scores to determine
the evidential strength of Glock aperture shear marks
Ingemarie van Gilse
Specific source vs. Common source reference scores to determine the evidential strength of
Glock aperture shear marks
Master Forensic Science – University of Amsterdam – 36 EC
Name: Ingemarie van Gilse BSc.Student ID: 11388439
Period: 1 July 2018 – 31 December 2018
Supervisors: Martin Baiker – Sørensen (NFI), Erwin Mattijssen (NFI) Examiner: Prof. Dr. Marjan Sjerps (Uva/NFI)
Proposed Journal: Forensic Science International Date of submission: 17-12-2018
3
Abstract:
Introduction: Firearms are often involved in criminal cases with human casualties. In those cases, a
comparison of firearm marks is important. Depending on what evidence is available, a common source (CS) or specific source (SS) database can be set up to analyze cartridge cases. These databases consist of same source and different source distributions, which can be set up with similarity scores. The CS and SS approaches both have their pros and cons, but they are often applied incorrectly in practice due to time restrictions and investments. Therefore, this study poses the following research question:
“Does the evidential strength differ between the specific source vs. common source reference databases of Glock aperture shear mark evidence?”
Methods: 20 Glock pistols were used, each of which fired 25 test shots with 9mm Luger Fiocchi
cartridges. The cartridge cases were scanned with a 3D surface scanner. With scratch mark comparison software, the aperture shear marks were cropped. Similarity scores were calculated with Matlab to build 20 CS same source and 20 CS different source distributions and 500 SS same source and 500 SS different source distributions. The likelihood ratio (LR) was calculated between 0 and 1 with steps of 0.001 and presented in a log10(LR) gradient. LR ratios were calculated to compare the CS LRs with the SS LRs.
Results: Visually, the CS distributions of the 20 firearms seem to correspond very well. The SS
distributions seem to differ between the different firearms. Also, the LR gradients differ for the SS distributions. The LR ratios seem to differ between the weapons. For some firearms, the CS LR seems to be larger than the SS LR, while for other firearms the SS LR is larger than the CS LR.
Conclusion: The same source and different source distributions of the CS and SS approaches differ
from each other. This results in different evidential strengths (LR) between the SS and CS reference database of Glock aperture shear mark evidence.
Keywords: Glock pistol, cartridge cases, aperture shear marks, specific source database, common source database, likelihood ratio.
4
1. Introduction
Firearms frequently play a role in criminal cases with human casualties. Comparison of firearm marks is very important in those cases. It can be critical in proving that a crime has taken place, it can be used to support the link between a weapon and a victim or a weapon and an attacker, and it can help with the reconstruction of the crime (Jackson & Jackson, 2017).
During firing, striated and impression marks are left on a bullet and/or cartridge case. A cartridge is built up out of four parts: primer, powder, cartridge case and a projectile, known as the bullet (Roth, Carriveau, Liu, & Jain, 2015) (Gerules, Bhatia, & Jackson, 2013). The primer is hit with the firing pin, causing deflagration of the gunpowder in the cartridge case. Pressure is built up and the bullet is released out of the barrel (Roth, et al., 2015). This pressure causes a recoil; The cartridge case is forced backwards and is pressed into the firing pin hole. The slide of the firearm also recoils and the barrel is tilting downwards. This forces the cartridge case to move downward and out of the firing pin hole, causing striated marks on the lower edge of the embedded primer. These striated marks are the aperture shear marks as seen in fig. 1 (Hamby, et al., 2016). These marks are very characteristic for a specific firearm, since they contain individual characteristics; small imperfections on parts of the firearm that are accidentally produced during manufacture (Jackson, et al., 2017). It is known that the Glock pistols leave aperture shear marks, which are one of the clearest marks on the primer (Hamby, et al., 2016).
Fig. 1: Aperture shear mark are shown in this figure. On the right, two aperture shear marks are compared. (Hamby, et al., 2016)
The marks on a questioned cartridge case and bullet created by a firearm can be compared with marks on cartridge cases and bullets from test fires produced with a suspected firearm. This is done by comparing differences and similarities of a specific mark left by a particular firearm (Hamby, Norris, & Petraco, 2016) (Tai & Eddy, 2018). Traditionally, forensic firearm examiners compare the marks by using 2D comparison microscopy. Illuminating oblique light is used to accentuate the marks, which are then aligned manually. During this subjective procedure the examiners have to evaluate the differences and similarities between the sets of marks (Riva & Champod, 2014). Also the strength of the evidence is judged, which requires the repeatability and individuality of the particular type of mark, estimated by the examiner (Riva, et al., 2014).
5 The President’s Council of Advisors on Science and Technology (PCAST) and the National Research Council (NRC) expressed their concerns about the subjective character of firearm mark comparison (Tai, et al, 2018) (PCAST, 2016) (Roth, et al., 2015). The individuality and repeatability of firearm mark comparison is questioned (Riva, et al., 2014) (Roth, et al., 2015). In particular, the expertise of examiners to attempt to determine if a questioned cartridge case or bullet is associated with a specific firearm and the lack of an objective method. Although a lot of research has been done, there is still need for more research to validate the individuality and repeatability since this has not been established (Riva, et al., 2014) (PCAST, 2016). Also a more objective approach is wanted to minimize the variation in the examination outcome.
In recent years several automated methods were developed for tool and firearm mark comparison using algorithms (Tai, et al, 2018) (Riva, et al.,2014) (Roth, et al., 2015) (Baiker, et al., 2014), (Chumbley, 2012) (Petraco, et al., 2012) (Song, 2015) (Lilien, 2015) (Zhang, & Luo, 2018) (Riva, Hermsen, Mattijssen, Pieper, & Champod, 2017). Examples are the congruent matching cell (CMC) method that is suggested by NIST (Song, 2015) (Mingsi Tong, Chu, & Thompson, 2014) (Chu, Ott, & Song, 2013), pairwise comparison is also suggested by NIST and CSAFE (Tai, et al., 2018) and similarity scores by Riva and Champod (Riva, et al., 2014). The individuality and repeatability of the tool and firearm marks can be determined quantitatively. For aperture shear marks, the automated systems can be used to present a measurement of similarity (Brand, 2017).
An objective measure, like similarity scores, can be used to calculate a LR. The similarity scores are needed to build reference distributions, which are needed to calculate a LR (Tai, et al., 2018). These reference distributions can be built via two different approaches: CS and SS (Tai, et al., 2018) (Ommen, & Saunders, 2018). The CS and SS databases both consist of two distributions: the same source distribution (Hp) and different source distribution (Hd). How the distribution are set up, differs between the CS and SS approaches. The density of the same source distribution is divided by the density of the different source distribution to calculate an LR.
For the CS approach, a reference (or alternative) population is used to set up the same source and different source distributions. For example, in case of a firearm examination, reference firearms are used to create an alternative cartridge case population. To build up the same source distribution, the similarity scores between cartridges cases from the reference population that originate from the same source are used. The different source distribution is created by similarity scores obtained between cartridge cases from the reference population that originate from different sources. Since the distributions are not dependent on the crime scene cartridge cases, the same source and different source distribution do not change for every specific case (Ommen, et al., 2018).
With the SS approach, the same source and different source distributions change for every specific case, as well as the reference population. As mentioned above, in case of a firearm examination, the specific source is a firearm found on the crime scene and the unknown source evidence a cartridge case found on the crime scene. The reference population could for example include firearms with the same characteristics, like caliber and/or brand. The same source distribution is built with the known source. Test shots are fired with the specific firearm and these cartridge cases are compared with each other to generate similarity scores for the same source distribution. The different source distribution is built by similarity scores obtained from comparing the crime scene cartridge case to the cartridge cases of the reference population (Ommen, et al., 2018).
The databases both have their strengths and weaknesses. The CS database is more practical and less time consuming, since the distributions do not change for different cases. On the other hand, the CS database is less specific, since specific tool properties are not taken into account. The opposite is the case for the SS database. It is very time consuming due to the building of the distributions, but by this, specific tool properties are taken into account.
6 Depending on the question asked, a common source or specific source database will be used. The identification of a common source, means that two objects share a common source. It is not necessary to determine a specific source. An example of such a question in firearm examination could be: “Are the two cartridge cases of unknown source fired from the same (but unknown) firearm?” (Ommen, et al., 2018). The following hypotheses that can be formulated for a common source problem for firearms:
Prosecution hypothesis (Hp): The two cartridge cases found on the crime scene are fired from the same (unknown) firearm;
Defence hypothesis (Hd): The two cartridge cases found on the crime scene are fired from two different (unknown) firearms.
Theoretically, the specific source approach is used to compare if an unknown source evidence does originate from a specific source (Ommen, et al., 2018). An example for firearm comparison could be, if a cartridge case that is found on the crime scene is shot with the firearm found on the crime scene. Test shots are fired from the found firearm and are compared with the crime scene cartridge case(s). The following hypotheses can be formulated for a specific source problem for firearms:
Hp: The cartridge case is fired from the specific firearm.
Hd: The cartridge case is fired from some other firearm in the alternative source population.
As mentioned before, a LR is calculated by dividing the density of the same source distribution by the density of the different source distribution. When a LR is larger than 1, it supports the Hp (Jackson, et al., 2017). Thus, in case of a firearm common source problem: It is more likely that the two cartridge cases originate from the same unknown firearm than that they originate from two different unknown sources. In case of a firearm specific source problem: it is more likely that the cartridge case originate from the specific firearm, than from some other firearm in the alternative population. When the LR is between 0 and 1, the LR supports the Hd (Jackson, et al., 2017). Thus, in case of a firearm common source problem: it is more likely that the two cartridge cases originate from two different unknown sources, than that they originate from a same unknown source. In case of a firearm specific source problem: it is more likely that the cartridge case does not originate from the specific firearm, but from some other source in the alternative population (Roth, et al., 2015).
Often, the CS and SS approaches are wrongly used for the question that has to be answered. Theoretically, for the question if two objects share the same (unknown source) is a CS database used and a SS database when the question is if a object origins from a specific source. In practice, it seems that the CS and SS approaches, in particular for SS problems, are incorrectly used. The SS approach takes a lot of time, therefore the CS approach is often used instead. A study by Ommen & Saunders (Ommen, et al., 2018) expresses their concerns about the wrongly used databases. They are of the opinion that it should not matter which model is possible to apply, based on the available items, but on the question that is asked [25].
Ommen & Saunders (Ommen, et al., 2018)] are theoretically right, but in practice it seems to work differently. A SS problem can be approached via CS or SS. In case of a firearm examination, a specific source (firearm) and crime scene cartridge case would theoretically use a SS approach. However, the firearm can create test shots, which can be used for a CS approach.
In this study, it is assumed that a cartridge case and firearm are found on the crime scene. In theory, the CS database is used for CS problems and SS for SS problems. But in practice, several factors have an influence on this choice of approach, like time and investment, which influences the feasibility of the approaches. This study will be focused on the difference in LR and how this is influenced by the SS
7 and CS approach, regardless of the feasibility. Aperture shear marks on cartridges fired from Glock pistols are used to set up same source and different source distributions for both the CS and SS approaches. With the CS and SS distributions, LRs can be calculated. The LRs for the CS and SS approach will be compared to analyse the difference. The following research question will be studied:
“Does the evidential strength differ between the specific source vs. common source reference databases of Glock aperture shear mark evidence?”
8
2. Materials and Methods
2.1 Materials & Data
Appendix 1 lists the firearms used in this study. When there is referenced to a firearm with a number, this number will correspond to the firearm in Appendix I. The Glock pistol collection of the NFI was used. A selection of the firearms was made, based on the following features: the firing pin hole was not damaged and the corners of the firing pin hole were rounded. In total, 20 Glock pistols were used in this study.
For each firearm, 25 shots were fired with 9mm Luger Fiocchi cartridges. A set of the resulting 500 test shots of 20 different types of Glock pistols were scanned. A focus variation system, Alicona IFM G4, was used to make 3D surface scans of the firing pin aperture shear mark [20]. A pilot was performed to determine the optimal parameters to scan the cartridge cases. Casts were made of the cartridge cases with Forensic Sil, since casts don’t suffer from disturbing reflections and thus yield more accurate data. Further, a combination with coaxial and ring light was used for the highest repeatability of the results. The following specifications were used: Vertical resolution of 200nm, lateral resolution of 2µm, magnification 20x, contrast 1, exposure time 22.5ms, illumination of coaxial light 0.01 and illumination of ring light 1.
2.2 Scratch
The scans of the cartridge cases were cropped with Scratch, a striated mark comparison software program of the Netherlands Forensic Institute (NFI). The cropped scans were used for calculation of mark similarity scores with automated algorithms in Scratch and Matlab[6,9]. Approximately the same part was cropped out of the scans, namely 185 pixels length and the width varied. When the length of a strip was set on 185 pixels, since a narrow length created very clear markings, but it was hard to reproduce the same location on the next cartridge case. With a longer length, it was easier to reproduce the same location for the cropping on each cartridge. The width of the crops depended on the disturbances on the cartridge cases, like imperfections in de scratch marks. Since disturbances influence the comparisons, it was not wanted to include these in the comparisons. The width of the cropped scans varied between 1540 and 2940 pixels.
The aperture shear markings can show different phases on the cartridge case, as shown in fig. 2. When only one phase was present, this phase was used to crop. With two or more phases, the second phase was used for cropping.
1 2 3
Fig. 2: Different phases of the aperture mark. On the left side is the firing pin hole. The second phase is chosen for comparison.
9
2.3 Database set up – (Statistical) Data presentation
Similarity scores were calculated with cross correlations in Matlab. Same source and different source distributions were determined for both the CS and SS databases with these similarity scores. This study did not contain a crime scene cartridge case. Therefore, every cartridge case was once treated as a crime scene cartridge case. How these crime scene cartridge cases were used for the built up of the CS and SS distributions, will be further explained.
Common Source database:
Same source and different source distributions were built up for the CS database in the following way: • Same source
From all 20 firearms (F1-F20) are 25 test shots (CC1-CC25) fired. To build a same source distribution, two crime scene cartridges were chosen, as seen in Fig. 3A. The firearm that shot those cartridge cases was not taken into account to build the distribution. The cartridge cases of the other 19 firearms were used to build the same source distribution. When for example firearm 1 shot the crime scene cartridge cases, the cartridge cases of firearm 2 – firearm 20 are used for comparison. As seen in Fig. 3A, the 25 cartridge cases that were fired from the same firearm were compared with each other to calculate similarity scores, resulting in 300 similarity scores per firearm. A total of 5700 similarity scores per same source distribution were included.
Each firearm was once taken as the firearm which shot the crime scene cartridge cases. Since this firearm was not taken into account, 20 different CS same source distributions were built.
Fig. 3: The built up of the CS same source and different source distributions are described. The red border includes the crime scene cartridge cases and the firearm which fired those cartridge cases. This ‘crime scene firearm’ is not included in the reference distributions. A, the built up of the same source distribution is described. The cartridge cases of the 19 other firearms are compared with each other. Only the cartridge cases that are fired with the same weapon are compared. B, the CS different source database built up is described. The cartridge cases of the 19 other firearms are compared with each other.
B A
10 • Different source
As seen in Fig. 3B, the crime scene cartridge cases and the firearm that shot them were not included in the different source distributions. The cartridge cases of the other 19 firearms were compared with each other. No comparisons were made between the cartridge cases of the same firearm. In Fig. 3B, a preview is given of how the cartridge cases are compared with each other. Firearm 1 was in that case the firearm that fired the crime scene cartridge cases, so was not included. CC1F2 is compared with CC1F3 up to and including CC25F20. CC1F3 is compared with CC1F4 up to and including CC25F20, and so on for the other firearms. This results in a total number of 106.875 similarity scores. Each firearm was once taken as the firearm that shot the crime scene cartridge cases, resulting in 20 different source distributions.
Specific Source database:
Same source and different source distributions were built up with the SS approach for all crime scene cartridge cases. This is performed in the following way:
• Same source:
Before building a SS same source distribution, a crime scene firearm and crime scene cartridge case were chosen. Each of the 500 cartridge cases, is once treated as a crime scene cartridge. As seen in
Fig. 4A, the crime scene cartridge was not used for building a same source reference distribution. The
other 24 test shots of the crime scene firearm were used to set up a same source distribution. For every firearm, all 25 cartridge cases were thus once treated as crime scene cartridge case, resulting in 25 same source distributions for every firearm. In total, this resulted in 500 same source distributions each containing 300 similarity scores.
• Different source:
As seen in Fig. 4B, the crime scene cartridge case was also not taken into account for building different source distributions. When for example c1w1 was the crime scene cartridge case, then the other 24 test shot cartridge cases of firearm 1 were used to compare with the cartridge cases from the alternative sources (F2 – F20). So CC2F1-CC25F1 are compared to CC1F2 - CC25F20. These similarity scores formed the different source distribution of CC1W1. The same is done for all cartridge cases, so 500 different source distributions are built for the SS database, containing 11.400 similarity scores per distribution.
11 Fig. 4: The built up of the SS same source and different source databases are described. CC in this figure means cartridge case. The red border includes the chosen crime scene cartridge. The purple border is the crime scene firearm and the purple line include the test shots of the crime scene firearm. A, the test cartridge cases are compared with each other to build a same source distribution. The other weapons are not included in this distribution. B, the test cartridge cases of the crime scene firearm are compared with the cartridge cases of the 19 other firearms. An example of the comparisons is shown for CC2 of the crime scene firearm with the cartridge cases of the other firearms.
Kolmogorov Smirnoff test
The Kolmogorov Smirnoff test was used to determine if there is a difference between the CS and SS distributions. The Kolmogorov Smirnoff test analyzed if the two distributions that were compared differ from each other by calculating a p-value. When this p-value is below the significance level, the distributions are significantly different. When the p-value is above this significance level, the distributions are significantly not different. The test was performed with the 20 same source distributions of CS, the 20 different source distributions of CS, the 500 same source distributions of SS and the 500 different source distributions of SS.
Comparisons between the CS same source distributions and the SS same source distributions were performed and between the CS different source and SS different source distributions. For every firearm, one CS same source distribution of the specific firearm was compared to 25 SS same source distributions. In total, 500 same source comparisons were made between the CS and SS. This was performed the same way for the different source distributions, resulting in 500 different source comparisons. The significance level α was set on 0.05.
12 Likelihood Ratio
To calculate the LR, it was necessary to fit a model over the data of the histograms. With Matlab were several parametric models fit over the data. It was visually determined that the different source distributions were normally distributed and the same source distributions did not follow any parametric model. Therefore, the Gauss distribution was used for the different source distributions and a non-parametric model was used for the same source distributions, namely the Kernel density estimation (KDE) (Silverman, 2018). With KDE, the bandwidth value determines the smoothness of the curve and was set on 6 pixels.
After fitting a model over the data, the LR was calculated in Matlab by dividing the density of the same source distribution by the density of the different source distribution. A correlation score, depending on the similarity score between the cartridge cases, was used to determine
the point in the histogram where the LR should be calculated. Fig. 5 shows an example of the calculation of the LR. The same source distributions is shown in red and the different source distribution in blue. The green line is the similarity score, where the LR should be calculated. At his point the density of the same source distribution is divided by the density of the different source distribution to calculate an LR.
LR gradients were determined and show the continuing of the LR for both the CS as the SS approach. To create this gradient, the LR was calculated between the range of 0 and 1 in steps of 0.001. The LR for the CS approach was calculated for all firearms, so 20 LR gradients. The LR gradients for the SS approach were calculated for all cartridge cases of 10 firearms, resulting in 250 LR gradients. The 10 firearms were randomly chosen, so that not only the firearms were chosen from which the same source and different source were distributed best. Therefore, all uneven firearms from Appendix 1 were chosen for the LR calculation. The gradients were plotted in a log10(LR) graph.
A Wilcoxon signed rank test was performed, to compare whether the median of the LRs for the different approaches differ from each other. Since the SS LR gradients are only calculated for 10 firearms, also only the 10 CS LR gradients of these firearms are used for these comparisons. Thus, the Wilcoxon signed rank test was calculated between 10 CS LR gradients and 250 SS LR gradients. For example the CS LR gradient of firearm 1 was compared with the 25 SS LR gradients of firearm 1, CS LR gradient of firearm 2 with the 25 SS LR gradients of firearm 2, and so on for the other 8 firearms. The significance level α was set on 0.05.
To determine how much the CS and SS LR gradients differ from each other, a LR ratio was determined by dividing the CS Log10 LRs by the SS Log10 LRs. This was performed for the 10 firearms for which the LR gradients were determined. The ratios were plotted in a graph. When the LR of the SS distribution is the same as the CS LR, than the ratio should be 1. Otherwise, when the ratio is higher than 1, the CS approach has a higher LR than the SS approach. When the ratio is below 1, the SS approach has a higher LR than the CS approach.
Fig. 5: A same source and different source histogram are shown. KM is the same source distribution and KNM is the different source distribution. The X-axis represents the cross correlation scores and the Y-axis is the frequency. The green line is the similarity score where the LR should be calculated.
13
3. Results
3.1 Common source and Specific source distributions
The common source distributions of the 20 firearms are shown in Fig. 6. The 20 different source distributions seem comparable at first glance. The20 same source distributions differ a bit at the top of the histograms, but have their peaks around the same similarity score and around the same height.
Fig. 6: CS same source and different source distributions of the 20 weapons are shown. In the legend are the colors linked to the distributions. KM means the same source distribution, KNM means the different source distribution and W is the used firearm. The x axis represents the similarity scores and the y-axis is the normalized frequency.
The SS same source and different source distributions of all cartridge cases are plotted in a histogram, shown in Fig. 7. This is, to give an overview of how the plots are distributed. At first glance, the same source distributions between the different firearms appear to differ from each other, as well as the different source distributions. The peaks of the distributions are located on different similarity scores for the different firearms and the peak heights differ. The distributions of the different cartridge cases of the same firearm do appear to correspond.
An example of the individual distributions of the cartridge cases per firearm is seen in Fig. 8 for the 25 cartridge cases of firearm 9. The other individual distributions are seen in Appendix II. The different same source distributions and the different common source distributions of the cartridge cases of the specific firearm seem to correspond very well. Also the separation between the same source and different source distributions is very clear for firearm 9. The same source comparisons contain only similarity scores above 0.6, while the different source comparisons only have similarity scores below 0.6.
14 Fig. 7: The SS same source and different source distributions of the 500 cartridges are shown. The x-axis represents the similarity scores and the y-axis is the normalized frequency. On the left are the different source distributions and on the right are the same source distributions
Fig. 8: The SS same source and different source distributions of firearm 9. The X-axis represent the similarity scores and the y-axis is the normalized frequency. The left distributions are the different source distributions and on the right are the same source distributions.
15 The different source and same source distributions of the CS database appear to correspond very well based on visual examination. The same source and different distributions between the different firearms of the SS database seem to differ a lot. Since the SS distributions seem to differ, it is expected that the CS and SS distributions also will differ from each other. To test this, a Kolmogorov-Smirnov test was performed and the results are shown in Table 1 and Fig. 9.
When the CS and SS same source distribution are compared, the majority (95,8%) is significantly different from each other. The CS and SS different source distributions are all significantly different from each other. This outcome was already expected, since the SS distributions differ a lot between the firearms and this would also have an effect on the comparison with the CS distributions.
There is only tested IF the distributions differ from each other, not how much they differ. How much the LR is affected is later estimated with a LR ratio. Further research could provide information about how much the different distributions differ from each other.
Table 1: Kolmogorov-Smirnov test is performed on the distributions. Same source is calculated in two ways: the significance of the distributions of the same firearm and the significance of the SAME SOURCE distributions between the firearms. For different source is the significance calculated between the different source distributions of the firearms. The significance level is adjusted with the Bonferroni correction.
Distribution Number of distribution combinations Significance level Not significantly different % Significantly different % Same source CS – same source SS 500 0.05 21 4,2% 479 95,8% Different source CS – different source SS 500 0.05 0 0% 100 100%
16 Fig. 9 The number of comparisons is shown of the CS vs. SS distributions that are significantly different. On the X-axis are the firearms displayed and the Y-axis is the number of significantly different comparisons. The different source
distributions are all significantly different between the CS and SS. For the same source distributions are only some of the distributions of firearm 11 and 17 not significantly different.
25 25 25 25 25 25 25 25 25 25 14 25 25 25 25 25 15 25 25 25 0 5 10 15 20 25 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 SIGN IF ICA N TL Y DIF FE R EN T FIREARM
CS vs. SS Same Source distributions
14 25 21 25 25 25 25 25 25 25 13 25 25 25 17 25 25 25 25 25 0 5 10 15 20 25 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 SIG N IF ICA N TL Y DIF FE R EN T FIREARM
CS vs. SS Different Source distributions
25 25 25 25 25 25 25 25 25 25 14 25 25 25 25 25 15 25 25 25 0 5 10 15 20 25 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 SIG N IF ICA N TL Y DIF FE R EN T FIREARM
CS vs. SS Same Source distributions
25 25 25 25 25 25 25 25 25 25 25 25 25 25 25 25 25 25 25 25 0 5 10 15 20 25 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 SIG N IF ICA N TL Y DIF FE R EN T FIREARM
17
3.2 LR calculation and ratio
With the built same source and different source histograms are LRs calculated. LR gradients were calculated for a similarity score range of 0 to 1 with steps of 0.001. This is done for both the CS as the SS approach. The CS LR gradients are shown in Fig. 10, two examples of the SS approach are seen in Fig. 11 and 12 and the SS gradients of all 10 firearms are shown in Fig. 13. The SS LR graphs of the other firearms are found in Appendix III.
Fig. 10: CS LR gradients The LR gradients of the 20 firearms are shown in this figure. The X-axis represents the correlation scores and the Y-axis is the Log10(LR). In the legend, the different firearms are presented.
Based on visual examination, the CS LR gradients seem to correspond very well. From a similarity score of 0.7 and higher, the log10(LR) of the different firearms has a linear course. For the SS LR
gradients, this is different. Especially for the lower correlation scores as seen for Firearm 1 in Fig. 11, the gradient is not linear. Not sufficient data is available for these lower correlation scores,
therefore the scores below a similarity score of 0.7 can be neglected. The LR gradients of the cartridge cases of firearm 9 in Fig. 12 do have a continuous line, since the same source histogram of firearm 9 does not have similarity scores below 0.65. At first glance, they do correspond very well. Only the LR ratio at a correlation score of 0.95 or higher seem to vary a bit. Since the continuous line starts around a correlation score of 0.65, the LR will be 0 below this similarity score and thus in favor of the Hd. The differences between the LR gradients of firearm 1 and 9 are also seen in the
distribution histograms.
In Fig. 13 are the LR gradients of the 10 firearms plotted. The differences between the firearms can be seen. The maximum LR varies between the firearms, as well as the similarity score for which this maximum is reached. These differences between firearms were also seen for the same source and different source distributions with the SS approach. Therefore, this result was expected. Nevertheless, the intervals of the different cartridge cases between a specific firearm seem to visually resemble. Still, for the lower similarity scores, the SS LR goes up and down and the line is not continuous, but interrupted.
18 Fig. 11: SS LR gradients of the 25 cartridge cases of firearm 1. The X-axis represents the correlation scores and the Y-axis is the Log10(LR). In the legend, the different cartridge cases are presented.
Fig. 12 SS LR gradients of the 25 cartridge cases of firearm 9. The X-axis represents the correlation scores and the Y-axis is the Log10(LR). In the legend, the different cartridge cases are presented.
19 Fig. 13: SS LR gradients of the 10 firearms. The X-axis represents the correlation scores and the Y-axis is the Log10(LR). In the legend, the different cartridge cases are presented.
Wilcoxon signed rank test is calculated to compare if the medians of the LR gradients contradict. A summary of the results are shown in Table 2. The table shows that the population median between the CS and SS LR differ for a lot of the firearms. Only the CS and SS LR gradients of firearms 1, 3, 11 and 15 have some LR intervals that are significantly not different. To see how large the LR difference is between the two approaches, a LR ratio is calculated.
Table 2: Wilcoxon signed rank test.
Firearm: Significantly different Significantly not different 1 11 14 3 6 19 5 25 0 7 25 0 9 25 0 11 8 17 13 25 0 15 23 2 17 25 0 19 25 0
20 The LR ratio is calculated and an example is shown in Fig. 14 for firearm 1 and in Fig. 15 for all firearms together. The separate LR ratios of the other weapons are shown in Appendix IV.
In Fig. 14 is seen that the LR between the CS and SS approach differ in the lower correlation scores, since the LR ratio varies and is not around 1. The LR ratio for firearm 1 seems to be around 1 for correlation scores 0.8 – 1, which indicates that the LR between the CS and SS approach do not differ at this point. The largest differences are seen around correlation score 0.7. Looking at the distribution histograms of firearm 1, this is the correlation score where the same source distributions cross the different source distributions.
In Fig. 15 the LR ratios are seen for all firearms. The point where the same source
distributions cross the different source distributions has the most and largest differences in LR, since the LR ratio is on these points very high and low. For all firearms, it seems that the LR ratio is around 1 between correlation score 0.8-0.95. Above 0.95, the ratio seem to vary a bit.
In Table 3 an overview is given of the mean log10(LR) ratios between similarity scores 0.8-0.95 for the different firearms. Also the LR ratios between a similarity score of 0.8-0.8-0.95 and between 0 -1 are given. The ratios differ for the different firearms, some firearms seem to have an log10(LR) ratio score around 1, while other firearms have a mean ratio of 0.4 or 1.3. These differences seem very small, but when looked at the LR ratios, there is seen that these differences are of great impact on the LRs. It is possible, that the verbal scale of the LR between the CS and SS differ. Resulting in misleading LRs. The CS LRs for firearm 5, 13 and 17 seem to be way larger than the SS LRs. The low LR ratio of firearm 9 implies that the CS LR is way smaller than the SS LR.
Fig. 14: LR ratio of firearm 1. The X-axis represents the correlation score and the Y-axis the LR ratio. In the legend are the different cartridge cases presented of firearm 1. The similarity scores between 0.8-1 seem to have an LR ratio around 1. Below a similarity score of 0.7, the LR ratio’s differ a lot.
21 Fig. 15: The LR ratios of the 10 firearms are presented. On the X-axis are the correlation scores and on the Y-axis the LR ratio. Between a similarity score of 0.8 and 0.95, the LR ratio seem to be around 1. Below a similarity score of 0.8, the LR ratios differ a lot.
Table 3: The mean LR ratio between the CS LR and SS LR are shown for similarity scores between 0.8-0.95. The maximum and minimum ratios are mentioned that were found for the 0.8-0.95 range.
Firearm Mean log10(LR) ratio between 0.8-0.95 Min ratio between 0.8-0.95 Max ratio between 0.8-0.95 Mean LR ratio between 0.8-0.95 Mean LR ratio between 0 - 1 1 1,004 0,924 1,420 1,554 4,342 3 1,033 0,958 1,092 1,253 1,674 5 1,299 1,216 1,404 4,960 3,498 7 0,608 0,589 0,700 0,961 0,760 9 0,445 0,312 0,583 0,0003 0,014 11 0,966 0,914 1,017 0,877 3,093 13 1,244 1,102 2,166 11,08 15,80 15 0,757 0,669 1,060 0,425 3,023 17 1,293 1,148 1,656 5,756 12,77 19 0,764 0,673 0,804 0,105 2,456
22
4. Discussion
Fig. 7 shows the SS same source and different source distributions. A few of the distributions appear
to differ from the rest. This can be due to the firearm characteristics that are taken into account with the SS approach. As seen in the figure, some SS different source distributions are more located to the left side of the graph, thus including lower similarity scores. It is possible that the crime scene firearm has very specific characteristics that do not match with the other firearms. This results in lower similarity scores and thus a different source distributions that is more located to the left. For the SS same source distribution, some distributions are also located more to the left side. This has a different explanation, since the SS same source distributions do only contain similarity scores between the test shots of the crime scene firearm. Apparently, the aperture shear marks on the various cartridge cases differ from each other and the marks on the test shots seem less repeatable. Hereby, the similarity scores between the test shots appear to be lower, since the markings are less similar. It is also possible that the various test shots show different aperture shear phases and this influences the similarity scores.
In Fig. 13 are the LR gradients of the SS approach shown. Also for the SS LR gradients are differences seen between the firearms. For some firearms, the LR is higher than the other firearms. This can be the case, when the same source distribution and the different source distribution are well separated. When the LR is then calculated for high similarity scores, the density of the same source is high, while the density of the different source is very low. This results in a high LR. This is for example seen for Firearm 9 in Fig. 8. The same source distribution does not cross the different source distribution, so the distributions are well separated. This resulted in high LRs, which are seen in Fig. 12. Firearm 9 is one of the firearms that differed from the other firearms in Fig. 13. When the same source and different source distribution are less separated, the LR will be lower. Thus, the same source and different source distributions influence the LRs.
Noteworthy are the small slopes that are seen in some of the SS same source and SS different source distributions for several firearms. These small slopes are also seen at the LR gradients. These slopes can be caused by a lack of datapoint or due to outliers of the distributions. These outliers are low similarity scores within the same source distributions. An explanation for these low scores could be that the aperture marks of two cartridge cases are not properly aligned by Matlab. This can be caused by the different widths that are used for the crops. The low similarity scores influence the same source distribution and therefore also influence the LR. Since only a few low similarity scores are generated for the same source distribution, the slopes seem to be separated from the rest of the distribution..
Also, the KDE that is used for the same source distributions can have an influence on the slopes and their influence on the LR. The KDE is influenced by the bandwidth that is used, since it has effect on how the data is followed. When the bandwidth is too large and not sufficient data is available, the curve does not follow the actual data points. When the bandwidth is too small, every datapoint is followed and the curve will not be smooth. When the distribution is displayed differently, this has an effect on the LR. Therefore, there has to be taken into account that the bandwidth of the KDE has an influence on the LR and should be chosen wisely.
In Table 3, the Wilcoxon signed rank test showed that some of the LR gradient of the CS and SS of firearm 1, 3, 11 and 15 are not significantly different. The gradients of the other firearms do differ between the CS and SS approach. An explanation could be that the max log10(LR) of these firearms is about the same, while the other differ.
23 LR ratios were assessed, to give an indication how much the CS and SS LR intervals do differ from each other. The ratios differ the most at the point where in the histograms the same source distribution crosses the different source distribution. Since this crossing point can differ between the distributions of the different cartridge cases and between the firearms, the most ratio differences are found at this point.
The LR ratio differs a lot at the point where the same source crosses the different source distributions in the histograms. This is probably caused by the SS same source distributions. Outliers are seen more clearly, since the SS same source distribution has less datapoints than the CS same source distribution. The outliers cause a less distributed histogram and therefore the LRs are also influenced.
For one firearm, one extra shot was fired, because a deformation was found on the aperture shear. Probably due to an extra impact of the firing pin. In this study, firearms are used from which the firing pin hole is not damaged. In practice, it is possible to found firearms with a damaged firing pin hole. This could have an influence in the chosen reference database. Also dependent values were chosen to built the databases instead of independent values. When one cartridge case is an outlier, this can generate 24 ‘outlying’ values for the SS. How the evidential value is influenced when independent values are chosen to built same source and different source distributions, can be researched in a future study.
5. Summary and Conclusion
In theory, the SS approach is used for specific source problems and the CS approach for common source problems. Since building a database with the SS approach takes a lot of time, there is researched if the evidential value of the CS and SS approaches differ from each other.
The different distributions are set up with a CS and SS approach. Visually, the CS distributions seem to correspond very well, but the SS distributions between the various firearms seem to differ. The Kolmogorov Smirnoff test supported this. LR intervals were calculated and also in this case, as expected for the SS, differences were seen for the SS LR intervals between the firearms. The Wilcoxon signed rank test showed that the CS and SS LR intervals were for most firearms not significantly the same. The LR ratio determined how large the differences in LR were between the CS and SS approach. The LR ratios seem to differ a lot between the different firearms, indicating that the CS and SS
With these main findings in mind, the research question of this study is revisited: “Does the
evidential strength differ between the specific source vs. common source reference databases of Glock aperture shear mark evidence?” When the log10(LR) ratios are compared, some firearms have a
log10(LR) ratio around 1. When the LR ratios are determined, these differences between the CS LR and SS LR appear to be way larger and can have considerable influence for the verbal scale. The evidential strength between the SS and CS reference databases thus differ. Further research can be done, to probably get inside in how much the different same source and different source distributions differ and how much this is of influence on the LR. Also, the feasibility in practice could be tested.
24
6. Recommendations & Future research
Several specific recommendations are formulated on how research into the CS and SS approaches could proceed. Especially to give a stronger opinion about the differences between the CS and SS distributions on the LR.
Future research can for example study other firearms with aperture shear marks. It is possible that the LR calculated with the CS and SS approach for Glock pistols differs less (or more) with each other than a different type of firearm. When this research would be performed, it would be best to use the same ammunition as for this study. In this case, differences due to ammunition type are ruled out. Different ammunition brands can also influence the LR. Therefore, it is also possible to do the same research but with different ammunition. The LRs can be compared and an LR ratio measured. Some types of ammunition show better markings, because of the material. Softer material leaves better markings. This can probably cause higher similarity scores and better distributed histograms. When the distributions are better separated, it is possible that the differences in CS LR and SS LR are smaller. This study focused on aperture shear marks. Different types of striated marking or impression marks can also be studied. For example the firing pin or breech-face impressions. It is possible that for these marks the CS and SS approach calculate very different LRs.
Also the amount of test shots that are needed to set up a SS reference database can be studied. Limited research has been found on this subject. In the study by Law (Law, Morris, & Jelsema, 2018) was stated that 15 test shots should be sufficient, but that 25 test shots cover the whole distribution. For this study, 25 test shots were used, while 15 test shots probably also would have satisfied. To test this, new distributions will be set up with for example 10, 15, 20 and 25 test shots. LR scores can be calculated with these histograms and can be compared with each other.
In this study, an approach is made if the distributions differ, but not how much the distributions differ. Future research could be done to how much these distributions differ and what effect this has on the LR. Probably the outliers have less influence on the histograms and LR when a larger dataset is used. The outliers cause a large effect on the LR for the lower similarity scores. Also which firearms are used for the reference database are of influence.
Acknowledgements
The author would like to thank Martin Baiker and Erwin Mattijssen for their supervision and
feedback on the study. Also Prof. dr. Marjan Sjerps is thanked for her feedback and role as examiner in the master Forensic Science of the University of Amsterdam.
Within the NFI, the author would like to thank the Weapons and Tools group and in special Rene for shooting the test shots.
25
References
Alberink, I., de Jongh, A., & Rodriguez, C. (2014). Fingermark evidence evaluation based on automated fingerprint identification system matching scores: the effect of different types of conditioning on likelihood ratios. Journal of forensic sciences, 59(1), 70-81.
Baiker, M., Keereweer, I., Pieterman, R., Vermeij, E., van der Weerd, J., & Zoon, P. (2014). Quantitative comparison of striated toolmarks. Forensic science international, 242, 186-199.
Bolton-King, R. S. (2016). Preventing miscarriages of justice: A review of forensic firearm identification. Science & Justice, 56(2), 129-142.
Brand, N. (2017). Bewijskracht en correlatie van glock slagpingatschaafsporen: Studie naar de correlatie tussen likelihood ratios van onderzoekers en een 3D vergelijkingsmethode.
Chu, W., Ott, D., & Song, J. F. (2013). Validation tests for the congruent matching cells (CMC) method
using cartridge cases fired with consecutively manufactured pistol slides (No. Journal of the Association
of Firearms and Toolmarks Examiners).
Chumbley, L. S. (2012). Quantification of Toolmarks, Final Technical Report. BiblioGov.
Gerules, G., Bhatia, S. K., & Jackson, D. E. (2013). A survey of image processing techniques and statistics for ballistic specimens in forensic science. Science & Justice, 53(2), 236-250.
Hamby, J. E., Norris, S., & Petraco, N. D. (2016). Evaluation of Glock 9 mm firing pin aperture shear mark individuality based on 1,632 different pistols by traditional pattern matching and IBIS pattern recognition. Journal of forensic sciences, 61(1), 170-176.
Helmli, F. (2011). Focus variation instruments. In Optical Measurement of Surface Topography (pp. 131-166). Springer, Berlin, Heidelberg.
Hepler, A. B., Saunders, C. P., Davis, L. J., & Buscaglia, J. (2012). Score-based likelihood ratios for handwriting evidence. Forensic science international, 219(1-3), 129-140.
Jackson, A. R., & Jackson, J. M. (2017). Forensic science. Pearson Education.
Law, E. F., Morris, K. B., & Jelsema, C. M. (2018). Determining the number of test fires needed to represent the variability present within firearms of various calibers. Forensic Science International. Leegwater, A. J., Meuwly, D., Sjerps, M., Vergeer, P., & Alberink, I. (2017). Performance Study of a Score‐based Likelihood Ratio System for Forensic Fingermark Comparison. Journal of forensic
sciences, 62(3), 626-640.
Lilien, R. (2015). Applied research and development of a three-dimensional topography system for firearm identification using gelSight. Acton, MA, USA.
Natrella, M. (2010). NIST/SEMATECH e-handbook of statistical methods.
Ommen, D. M., & Saunders, C. P. (2018). Building a unified statistical framework for the forensic identification of source problems. Law, Probability and Risk, 17(2), 179-197.
26 Ott, D., Soons, J., Thompson, R., & Song, J. (2017). Identifying persistent and characteristic features in firearm tool marks on cartridge cases. Surface topography: metrology and properties, 5(4), 045009. Petraco, N. D., Shenkin, P., Speir, J., Diaczuk, P., Pizzola, P. A., Gambino, C., & Petraco, N. (2012). Addressing the National Academy of Sciences’ challenge: a method for statistical pattern comparison of striated tool marks. Journal of forensic sciences, 57(4), 900-911.
President’s Committee of Advisors on Science and Technology. Report to the President on forensic science in criminal courts: ensuring scientific validity of feature-comparison methods. Washington, DC: Executive Office of the President, 2016.
Riva, F., & Champod, C. (2014). Automatic comparison and evaluation of impressions left by a firearm on fired cartridge cases. Journal of forensic sciences, 59(3), 637-647.
Riva, F., Hermsen, R., Mattijssen, E., Pieper, P., & Champod, C. (2017). Objective Evaluation of Subclass Characteristics on Breech Face Marks. Journal of forensic sciences, 62(2), 417-422.
Roth, J., Carriveau, A., Liu, X., & Jain, A. K. (2015, September). Learning-based ballistic breech face impression image matching. In Biometrics Theory, Applications and Systems (BTAS), 2015 IEEE 7th
International Conference on (pp. 1-8). IEEE.
Silverman, B. W. (2018). Density estimation for statistics and data analysis. Routledge.
Song, J. (2015). Proposed “congruent matching cells (CMC)” method for ballistic identification and error rate estimation. AFTE J, 47(3), 177-185.
Tai, X. H., & Eddy, W. F. (2018). A Fully Automatic Method for Comparing Cartridge Case Images. Journal of forensic sciences, 63(2), 440-448.
Tong, M., Song, J., Chu, W. and Thompson, R.M. Fired cartridge case identification using optical images and the congruent matching cells (CMC) method. Journal of Research of the National Institute of
Standards and Technology, 119:575–582, 2014.
Zhang, K., & Luo, Y. (2018). Slight variations of breech face marks and firing pin impressions over 3070 consecutive firings evaluated by Evofinder®. Forensic science international, 283, 85-93.
27
Appendix
Appendix I: Glock pistols
Table A1: Glock pistols used for test fires.
Nr. Glock pistol/ zaaknummer Model Caliber 1 13.10.07.133 26 9mm Parabellum 2 13.11.11.180 26 9mm Parabellum 3 15.06.09.047 19 9mm Parabellum 4 15.11.30.171 19 9mm Parabellum 5 16.04.18.156 19 Gen4 9mm Parabellum 6 17.01.18.187 26 9mm Parabellum 7 17.12.01.039 26 9mm Parabellum 8 17.012.01.002 26 9mm Parabellum 9 3206 17 9mm Parabellum 10 3210 19 9mm Parabellum 11 3255 17 9mm Parabellum 12 3256 17 9mm Parabellum 13 3374 19 9mm Parabellum 14 3381 19C 9mm Parabellum 15 3383 19 9mm Parabellum 16 3410 19 9mm Parabellum 17 3424 26 Gen4 9mm Parabellum 18 3444 26 9mm Parabellum 19 3445 26 9mm Parabellum 20 3448 26 Gen4 9mm Parabellum
28 Appendix II: SS same source and different source histograms
38 Appendix III: Log10(LR)
CS& SS gecombineerd: CS is zwart.
CS
46 Appendix IV: LR ratios
