Research project
Master Forensic Science
36 EC (January 2020-August 2020)
Forensic Evaluation of Objective Methods for
Automated Analysis of Glock Cartridge Case
Marks
Author:
Jorit Delen
12345393
Supervisor:
Martin Baiker-Sørensen, PhD
Netherlands Forensic Institute
(NFI)
Examiner
Erwin Mattijssen, MSc
Netherlands Forensic Institute
(NFI)
August 24, 2020
Abstract
In the current system of firearm examination, when comparing a questioned bullet or cartridge case with a reference often the examiner has to make several subjective calls during the process. There have been several calls for a better statistical foundation for current methods of firearm examination, and the move towards more automatic systems which will be less subjective.
In this study, we will use one of those new automatic systems, a program named Scratch, that provides similarity scores of comparisons based on Areal Cross Correlation Function (ACCF) and Congruent Matching Cells (CMCs). We use this program to analyze the individuality and repeatability of firing pin impressions and breech face impressions that are left on cartridge cases by Glock firearms. To do this, we took a set of 200 Glock firearms of type 17, 19 and 26, and fired each 4 times for a set of 800 cartridge cases. A cast of these cases was made, and these cast were scanned in with 3d scanning equipment. The 3d data can be compared to create Known Match and Known Non Match distributions for both the ACCF and CMC fraction.
The primer material could also be an issue, so each firearm was shot twice with Fiocchi-branded cartridge cases which had a nickel primer (which is softer), and Sellier & Bellot-branded cartridge cases which had a brass primer (which is harder).
For the breech face impressions, it was found that a part of the S&B ammunition had a dot-like pattern on the primer which massively influenced the comparison scores. These were filtered out.
For both mark types and similarity measures, the results seemed reasonably good, however firing pin impressions performed better than breech face impressions on both ACCF and CMC fractions. For firing pin impressions, it was found that softer materials provided better results for both the ACCF scores and CMC fractions. For breech face impressions, it was found that it was more important that both cartridge cases had the same primer material, then what that material was.
Keywords: • Firearm examination • Objective comparison • Repeatability • Individuality • Primer material 1
1
Introduction
Forensic firearm examination is a very important discipline within forensic science. Crimes containing a firearm are often very severe and violent, sometimes with lethal results. If a crime occurs with a firearm, a question that is often important to the case is ”Was this bullet or cartridge case fired by this firearm?”. A firearms examiner can assist in answering this question by providing likelihood ratios based on the evidence. To do this, the examiner compares marks on test shots fired through the firearm in question with marks on the found bullets and cartridge cases.
When the trigger of a firearm is pulled, the following actions happen to the cartridge case that result in measurable traces: The firing pin hits the primer, which results in the firing pin impression. The primer detonates a small charge, which causes the powder in the cartridge to burn up and become a mixture of gasses. These gasses expand rapidly which pushes the bullet forward through the barrel. The expanding gasses also push the cartridge backwards, into the breech face. This results in the creation of breech face marks on the back end of the cartridge case. In most semiautomatic pistols, the slide also gets pushed backwards, and the rear end of the barrel will move slightly downwards. This combined with the cartridge case being pushed against the firing pin hole can lead to the creation of firing pin aperture shear marks on the cartridge case. If the firing pin remains in contact with the primer during this moment, this will also cause the creation of a firing pin drag marks. Finally, the cartridge case will be extracted from the chamber by the extractor and ejected through the ejector. In some types of firearm this can lead to ejector marks and extractor marks on the cartridge.
A picture of some of the marks described can be found in Figure 1.
Historically the comparison of these marks has been a very subjective process. Almost always it is done
Figure 1: A cartridge case and bullet with the following marks: Firing pin impression (A), Ejector mark (B), Breech face mark (C), Firing pin drag mark (D). Pictures taken from Ultra Electronics Forensic Technology, Inc. (http://ultraforensictechnology.com/pubs#bro)
by a human examiner who manually compares the questioned bullets or cartridge cases to each other, often with the help of optical microscopes. The examiner aligns the bullets or cartridge cases by hand, after which the differences and similarities are subjectively determined based on the training and experience of the examiner. After this, the examiner makes an often subjective call on the strength of the evidence, based on their personal experiences and knowledge of individuality and repeatability of different types of marks. [1]
There have been questions by multiple parties about the scientific validity of this process, and reports that call for more objective comparison methods. Examples of these are the NAS report of 2009, and the PCAST report of 2016 [2] [3]. According to these reports, it is also concerning that there is a lack of a statistical background to back up the current methods.
A great way to reduce the subjectivity in this process would be to introduce more automated systems. Automated systems have several advantages, including:
• Less subjectivity in the process
• It is easier to determine the statistical background of these systems and what can be expected of them • It is easier to set up an objective database which can be used for statistical evaluation, as the automated system will, when provided with the same elements for comparison, always deliver the same results, without any subjectivity that a human examiner would introduce.
These systems can be used to objectively compare marks and give some sort of objective measure of com-parison. Creating and using a (preferably large) score based database for these objective measures can help with building a statistical foundation as well. The scores in these databases can be used to create probabil-ity densprobabil-ity functions (through for example kernel densprobabil-ity estimation), which can in turn be used to create score-based likelihood ratios (SLRs).
Since the NAS and PCAST reports have come out, several automated methods have been developed for firearm mark comparison, both for impression marks [4] [5] [1] and for striation marks [6]. Examples include the congruent matching cell method [4], pairwise comparison algorithms [5] and similarity scores [1]. These methods all provide a much more objective way to evaluate firearm related evidence, and can be used to build up objective databases that can be called upon later.
2
Research questions
In this study, one of these automated systems will be tested. We will be using an automatic comparison program (called Scratch) to build an objective database and analyze and compare the probability distribu-tions. With these, we can possibly determine a likelihood ratio for the scores that the program provides. This study will be focused on firing pin impressions and breech face marks.
In real cases, there are many different brands of ammunition that are used. Cartridge cases of different brands can be made from different materials, such as brass or nickel. These different materials all have different levels of hardness. If a cartridge case is made of a very hard material, it could be that the traces that are left on that cartridge case will not be as deep or as clear than if they were left on a cartridge case made of a softer material. This could also mean that an automated program such as the one we will be using might have problems when comparing cases of different materials. If the different materials provide significantly different score distributions, this will be something important to be aware of to prevent drawing incorrect or otherwise inaccurate conclusions in the future. [7] [8]
This leads us to the following research questions that will be covered in this study:
• What is the repeatability (or the within variation) and individuality (or between variation) of firing pin impressions left by Glock firearms?
• What is the repeatability (or the within variation) and individuality (or between variation) of breech face impressions left by Glock firearms?
• Are there significant differences in the score distribution of the marks for different materials?
3
Materials and Methods
3.1
Setup of the experiment
For this experiment, 200 Glock-branded firearms were used. These firearms were of the types 17, 19 and 26. A full list of all firearms used can be found in Appendix A. All firearms were given a number in the following way: first a number was chosen to represent the firearm, often based on the serial number. Then, the origin of the firearm is also represented in the code. For example, the firearm with serial number 2011183837 from Amsterdam is represented as 3837 Amsterdam. Each of these firearms was fired four times and the resulting cartridge cases collected. This set was already available at the NFI and has been used in previous studies [9]. To study the influence of the primer material on the marks, they were fired twice with ammunition with a nickel primer (of the brand Fiocchi), and twice with ammunition with a brass primer (of the brand Sellier & Bellot). From now on the cartridge cases originating from one firearm will be referred to as F1, F2, SB1 or SB2, followed by the number of the firearm as specified in Appendix A. So the second shot with nickel primer from firearm 3837 Amsterdam will be referred to as F2 3837 Amsterdam.
From these 800 cartridge cases, a cast was made using Forensic SIL [10]. Creating a cast of the car-tridge cases using this method was found to not be detrimental to the quality of the impressions that can be recovered, and can prevent distorting reflections when using a light-based scanning system [9].
These casts were used to make 3d-scans of the breech face impressions and of the firing pin impressions. This was done using an Alicona InfiniteFocusSL 3d-measurement system [11]. To be certain that there were enough small details available in the scans to properly perform a comparison a vertical resolution of 100 nm and a horizontal resolution of <2 µm were chosen. The size of the area scanned for the breech face is 3.8x3.8 mm. For the firing pin impression, it is 1.95x0.95 mm. All other parameters that were used can be found in the appendix. An example of what one of the scans looks like can be found in figure 2.
Figure 2: A scan of the breech face impression (A) and firing pin impression (B) of F2 0023 Amsterdam. Due to the choice of resolution, small details are properly visible. Actual size of scanned area: 3.8x3.8 mm
3.2
Scratch
Scratch is the program used to compare these 3d data sets with each other. It was developed by a cooper-ation between the Dutch Forensic Institute (NFI) and the Ncooper-ational Institute of Standards and Technology (NIST). This program was developed with the purpose of comparing both bullets and cartridge cases, and can compare impression marks (such as the firing pin and breechface impression mark) as well as striation marks (such as the firing pin drag mark). In this study we will only be testing the firing pin impressions and breech face impressions, which are both impressions marks.
After data as represented in figure 2 is loaded into the program, it will appear as in figure 3. The col-ors represent the ”height” of the original data, with purple being more away from the viewer, and yellow being towards them.
Figure 3: The 3D data of F1 0817 Amsterdam as it is loaded into Scratch. The left picture represents a firing pin impression, while the right is used for analyzing a breech face impression. The colors represent the ”height” of the original data.
Not all data from figure 3 is relevant. The data is preprocessed by selecting the important regions and applying a filter to make the contrast clearer. An example of how the data looks after preproccesing can be found in figure 4.
For comparison of impression marks, scratch uses algorithms based on the following similarity measures: Areal cross-correlation function (ACCF) and congruent matching cells (CMC). In general a similarity mea-sure will be a score that will be higher the more similar two datasets are.
3.2.1 Areal cross-correlation function (ACCF)
The areal cross-correlation function can be used as a measure of how similar two three-dimensional data sets are. The data sets that have been preprocessed as in figure 4 can be displayed as a two dimensional matrix of size and width of the original three dimensional image, with the values of the matrix being the height at that point.
When comparing two matrices A and B, with dimensions MxN, and with µ and σ being the mean and standard deviation of the matrices respectively, the ACCF can be calculated as follows [12]:
1 M N PN j=1 PM i=1((Aij− µA)(Bij− µB)) σAσB (1) 5
Figure 4: The 3D data of F1 0817 Amsterdam after it has been preprocessed. All regions of importance have been selected and a filter has been applied to improve the contrast. The left picture represents a firing pin impression, while the right is used for analyzing a breech face impression. The colors represent the ”height” of the original data.
Using this equation has the following results [12]:
• When the data sets are identical (A = B at every point), then the ACCF will be 1.
• When data sets are almost identical or closely related (as we hope to find in cartridge cases fired from the same firearm), the ACCF will be close to 1. When rotating and translating the data sets relative to each other, the ACCF function will have a maximum at the rotation and translation where the data sets are aligned perfectly relative to each other.
• When the data sets are completely unrelated (as we hope to find in cartridge cases fired from different firearms), the ACCF will be lower. If the function has a clear maximum related to a certain rotation or translation (this does not have to be the case), it will be lower than when comparing datasets that are related.
These results are used in the first step of the comparison. The program will try to find the ”best” relative rotation and translation of the two data sets, based on the maximum of the ACCF function. It will record this value among with other values such as the relative position and rotation at that point.
3.2.2 Congruent matching cells (CMC)
After the highest ACCF has been decided, the datasets will be placed back in their original position and rotation and the program will move on to the next similarity measure: An algorithm based on Congruent Matching Cells (CMCs) as proposed by Song in 2013, and improved upon in 2015 [13] [4]. What follows is only a short description of this algorithm, a more detailed version including estimations of the possible error rates can be found in [4] and [14].
The first data set is split up into smaller square cells. The number of cells is dependant on the cell size and can be chosen, however Song recommends a number between 30 and 40 [4]. As an example, figure 5 shows the firing pin impression of F2 0023 Amsterdam, split up in 34 cells.
For each of these cells, the corresponding location in the other data set is checked for similar topological features. If the other data set contains a cell that:
Figure 5: The firing pin impression of F2 0023 Amsterdam split up into multiple cells after having been preprocessed. The different colors represent different z-values as found in the scan of the original cartridge case
• Is located at most a certain distance away from the corresponding location of the cell in the first set • Is rotated at most a certain amount of degrees when compared with the orientation of the cell in the
first set
Then this cell is called a ”congruent matching cell”. All of the the parameters used can be found in the appendix.
An example of this is displayed in figure 6, where the firing pin impression of F2 0023 Amsterdam is compared with the firing pin impression of SB1 0023 Amsterdam, and congruent matching cells are displayed in black. For this experiment, we used a cell size of 125 by 125 (as we were looking at a comparison of firing pin impressions), with the following parameters to determine what is a CMC:
• The cross-correlation function has to be >20%
• The cell is located at most 350 units away from the corresponding cell • The cell is rotated at most 35 degrees compared to the corresponding cell
We are most interested in the fraction of congruent matching cells over the total number of cells, as for example stating only a score of 15 CMCs can be misleading. This can mean 15 out of 15 total cells, or 15 out of 45 which paint a completely different picture. For that reason we will be using the CMC fraction instead.
3.3
LR and statistical calculations of significance
The data that results from these comparisons can be gathered in a histogram. This will be done for both the ACCF and the fraction of CMCs over the total number of cells. Here we can make separate histograms for comparisons between cartridge cases fired from the same weapon, or known matches (KM), and comparisons of cartridge cases from different weapons, or known non matches (KNM). Those histograms can be used to
Figure 6: A comparison of the firing pin impressions of F2 0023 Amsterdam and SB1 0023 Amsterdam, with cells visible. 19 of the 34 total cells were matching congruent cells. Cells that had no equivalent in the other data set are displayed in red in the left image
determine the probability density functions, using kernel density estimation [15].
3.3.1 LR calculation
As mentioned in the introduction, one of the main questions of firearms examination is whether a certain found cartridge case was fired from a certain weapon. To determine a likelihood ratio, the question can be divided in the following hypotheses:
• Hp: The cartridge was fired by this weapon • Hd: The cartridge was fired by a different weapon
The likelihood ratio that correspond to this question is then given by:
LR = P (E|Hp)
P (E|Hd) (2)
The E in this equation represents the evidence, which will in this case be represented by a certain value of the comparison, for either the ACCF or the CMC fraction. Now the LR can be calculated by, for that specific value E, dividing the value of the probability density function of the KM comparisons by the value of the probability density function of the KNM comparisons.
3.3.2 Material comparison
Due to the two different types of primer material, there are three different types of comparison. A nickel-nickel comparison (or F-F), a brass-brass comparison (or SB-SB), and a mixed comparison (or F-SB). Using similar techniques, we can for the KM comparisons create separate probability density functions for each of these types. The nickel primers are generally softer than the brass primers. We expect that softer primer material will result in clearer impressions, and more displacement of the material. The expectation is that this will result in the KM F-F comparison scores being higher than F-SB or SB-SB comparisons of the same weapon.
To test if there is actually a significant difference between these types of comparisons, we will use a Wilcoxon rank-sum test [15]. By ranking the data points, this test can be used to determine if two samples came from the same underlying distribution. When the sample size of both samples is larger then 10, a normal approximation can be used. As all sample sizes in this study will be of sufficient size, we will do this as it is significantly less computationally extensive [15]. The significance level α will be set to 5 percent in all following tests.
4
Results
In this section the results of the comparisons will be discussed. For the KM, all possible comparisons were performed, leading to a total of 1200 firing pin impression comparison scores and 1200 breechface impressions scores. The total possible amount of different KNM comparisons is 318400 for both the FPIs and the BFIs. However, due to time constraints it was not feasible to perform all possible KNM comparisons, so a representative selection was made. In total there are 6870 KNM FPI scores, and 10204 KNM BFI scores available. Sometimes, when firing a cartridge case the case can get damaged. This can influence the score. Some example comparisons (one ”normal” KM comparison, one KM comparison in which one of the cases was damaged and one KNM comparison) can be found in Appendix C
4.1
Firing pin impressions
We will cover the firing pin impressions first. As mentioned in the previous section, all probability density functions were created using kernel density estimation. For all PDFs, a Gaussian kernel was used. The bandwidth was first decided based on Silverman’s rule of thumb, or 0.9 ∗ ˆσ ∗ n1/5with ˆσ the sample standard
deviation and n the sample size. From there, it was slightly lowered or raised to make sure that no over- or under-smoothing took place.
4.1.1 General results
All KM data and all KNM data is collected in the following figures. Figure 7 shows the comparison in CMC fraction scores, ranging from 0 to 1. Figure 8 shows a similar comparison for the ACCF, which ranges from 0 to 100. In both figures there seems to be a clear difference in the distributions, with very little overlap.
4.1.2 Material specific results
As mentioned before, the total distributions as displayed in figure 7 and figure 8 can be split up for the different types of comparison based on the materials involved. This results in the graphs as shown in figure 9 and 10 for the CMC fraction, and figure 11 and 12 for the ACCF scores.
Visually, there does not appear to be much difference in the KNM distribution, in both figure 9 and figure 11. This will be statistically analyzed in the next section. A more clear difference can be seen in the KM distributions of figure 10 and 12. In these, it appears that the F-F comparisons are concentrated most to the right, which would imply that on average these comparisons provided the highest scores for both CMC fraction and ACCF score. This was in line with expectations, as Fiocchi cartridge cases have a nickel primer, which is softer and would lead to clearer and deeper impressions.
Interestingly, it would appear that when visually examining the CMC fraction graphs, the F-SB comparisons performed slightly better than the SB-SB.
-2 0 2 4 6 8 10 12 14 16 0 0,1 0,2 0,3 0,4 0,5 0,6 0,7 0,8 0,9 1 fKNM(x) fKM(x)
Figure 7: Known Match vs Known Non Match distributions for the Consecutive Matching Cell fractions that resulted from firing pin impression comparisons. The Known Match distribution is based on 1200 Known Match scores, while the Known Non Match distribution is based on 6870 scores.
-0,01 0 0,01 0,02 0,03 0,04 0,05 0,06 0,07 0,08 0,09 0,1 0 10 20 30 40 50 60 70 80 90 100 fKNM(x) fKM(x)
Figure 8: Known Match vs Known Non Match distributions for the Areal Cross Correlation Function scores that resulted from firing pin Impression comparisons. The Known Match distribution is based on 1200 Known Match scores, while the Known Non Match distribution is based on 6870 scores.
-1 0 1 2 3 4 5 6 7 8 9 10 0 0,1 0,2 0,3 0,4 0,5 0,6 0,7 0,8 0,9 1 F-F SB-SB F-SB
Figure 9: The Known Non Match distribution of the CMC fractions that resulted from firing pin impression comparisons, divided into the material specific comparisons Nickel-Nickel (F-F), Brass-Brass (SB-SB) and Nickel-Brass (F-SB). 0 0,5 1 1,5 2 2,5 3 0 0,1 0,2 0,3 0,4 0,5 0,6 0,7 0,8 0,9 1 F1-F2 SB1-SB2 F-SB
Figure 10: The Known Match distribution of the CMC fractions that resulted from firing pin impression comparisons, divided into the material specific comparisons Nickel-Nickel (F1-F2), Brass-Brass (SB1-SB2) and Nickel-Brass (F-SB).
4.2
Breechface impressions
4.2.1 General results
We will proceed in much the same way as we did with the firing pin impressions. Figure 13 shows the comparison in CMC fraction scores, ranging from 0 to 1. Figure 14 shows a similar comparison for the11
0 0,01 0,02 0,03 0,04 0,05 0,06 0,07 0,08 0 10 20 30 40 50 60 70 80 90 100 F-F SB-SB F-SB
Figure 11: The Known Non Match distribution of the ACCF scores that resulted from firing pin impression comparisons, divided into the material specific comparisons Nickel-Nickel (F-F), Brass-Brass (SB-SB) and Nickel-Brass (F-SB). 0 0,005 0,01 0,015 0,02 0,025 0,03 0,035 0,04 0 10 20 30 40 50 60 70 80 90 100 F1-F2 SB1-SB2 F-SB
Figure 12: The Known Match distribution of the ACCF scores that resulted from firing pin impression comparisons, divided into the material specific comparisons Nickel-Nickel (F1-F2), Brass-Brass (SB1-SB2) and Nickel-Brass (F-SB).
ACCF, which ranges from 0 to 100. Once again, the KM and KNM distribution show clear differences for both the CMC fraction and ACCF score. As opposed to the firing pin impressions however, there appears
to be much more overlap. In general the KM distributions of the breechface impressions appear to be lower and more spread out than the KM distributions of the firing pin impressions.
-2 0 2 4 6 8 10 12 14 16 0 0,1 0,2 0,3 0,4 0,5 0,6 0,7 0,8 0,9 1 fKNM(x) fKM(x)
Figure 13: Known Match vs Known Non Match distributions for the Consecutive Matching Cell fractions that resulted from breech face impression comparisons. The Known Match distribution is based on 1200 Known Match scores, while the Known Non Match distribution is based on 10204 scores.
4.2.2 Clarification and filtration of the data
During the analysis of the data in the previous section, several firearms appeared for which the scores were highly unusual. For those, the F-F comparison was high, but all other F-SB and SB-SB comparison scores were extremely low. An example can be found in table 1, where all comparison scores for firearm 5045 Tilburg are displayed.
Table 1: The results of all six known match comparisons of firearm 5045 Tilburg, with both the ACCF and CMC fraction presented. It is very interesting that only the F-F comparison is reasonably high, while all other comparisons display values we would expect to see at Non Matches.
Case 1 Case 2 ACCF CMC Total cells CMC fraction F1 F2 48.677 21 38 0.553 F1 SB1 24.558 0 38 0 F1 SB2 19.662 0 42 0 F2 SB1 20.714 4 38 0.105 F2 SB2 16.570 0 42 0 SB1 SB2 11.885 0 30 0
After more examination it turned out that for those weapons, the two SB cartridge cases appeared to have a pattern of small dots, radiating outwards from the center. An example of this can be seen in figure 15, where the right image features this pattern. After cross referencing the weapons and visually examining the cases, it turned out that this pattern only appeared in a certain sub-type of the Sellier & Bellot cartridge cases.
0 0,01 0,02 0,03 0,04 0,05 0,06 0,07 0 10 20 30 40 50 60 70 80 90 100 fKNM(x) fKM(x)
Figure 14: Known Match vs Known Non Match distributions for the ACCF scores that resulted from breech face impression comparisons. The Known Match distribution is based on 1200 Known Match scores, while the Known Non Match distribution is based on 10204 scores.
Although all firearms were fired with S&B branded ammunition, different sub-types can appear within a brand, often based on production year. Several of these were used in this study. A sub-type can be identified by the stamp on the back of the cartridge case. The dot-like pattern appeared only in cartridge cases stamped with the text ”S&B 9x19 08” on them. From now on, these cases will be referred to as ”Type 8”.
As this pattern only appeared in type 8 cases and are not consistent within the same firearm (as evidenced by the low comparison scores), it is most likely that these patterns were already on the cartridge cases before they were fired, and were probably the unintentional side effect of a step in the production process. To study what the effect of the pattern on the similarity measures is, we can compare the KM scores of cases of type 8 with the KM scores of all other types. This is shown in figures 16 and 17 for the CMC fraction and the ACCF scores, respectively.
It seems that the dot-like pattern is deep enough to seriously overshadow the actual impressions we are interested in. This in turn impacts the ability of the system to properly compare these cartridge cases even if they were fired from the same firearm. For the rest of the study, all S&B cartridge cases of type 8 will be removed from the BF KM data set so as to not present biased results. This leaves us with a subset of 83 weapons for which all comparisons can be used. Of course, as this is only a problem in S&B ammunition, all 200 F1-F2 comparisons can still be used. A version of every relevant graph with the type 8 cases still included will be shown in the Appendix.
4.2.3 General results, filtered
We can remake the graphs in figure 13 and figure 14, but now for the filtered dataset. Figure 18 shows the comparison in CMC fraction scores, ranging from 0 to 1. Figure 19 shows a similar comparison for the ACCF, which ranges from 0 to 100. While the KM results seem to be higher when compared to the graph with type 8 still included, there is still more overlap than there was in the graphs of the firing pin impressions as shown in figures 7 and 8.
Figure 15: The processed breechfaces of F2 1734 Amsterdam and SB1 1734 Amsterdam. The left image looks as expected, however on the right image, a pattern of dots radiating outward from the center is visible.
0 0,5 1 1,5 2 2,5 3 3,5 4 4,5 5 0 0,2 0,4 0,6 0,8 1 8 SB-SB not 8 SB-SB 0 0,5 1 1,5 2 2,5 3 3,5 4 4,5 0 0,2 0,4 0,6 0,8 1 8 F-SB not 8 F-SB
Figure 16: The distributions of the CMC fractions resulting from comparisons of breech face impressions, from cartridge cases for type 8 vs not type 8. The left graph depicts the Brass-Brass (SB-SB) comparisons, and the right graph the Nickel-Brass (F-SB) comparisons.
4.2.4 Material specific results
Once again, we can split up the graphs in figures 18 and 19 into their components based on the primer material.
When compared with the KNM distributions of the firing pin impressions of figure 9 and 11, there appears to be a bit more difference in figure 20 and especially in figure 22.
For the known match scores, it appears once again that F-F comparisons give the highest scores. How-ever as opposed to the FPIs scores, this time the SB-SB comparisons seem to be doing better than the F-SB comparisons.
0 0,005 0,01 0,015 0,02 0,025 0,03 0,035 0,04 0 20 40 60 80 100 8 SB-SB not 8 SB-SB 0 0,005 0,01 0,015 0,02 0,025 0,03 0,035 0,04 0,045 0 20 40 60 80 100 8 F-SB not 8 F-SB
Figure 17: The distributions of the ACCF scores resulting from comparisons of breech face impressions, from cartridge cases for type 8 vs not type 8. The left graph depicts the Brass-Brass (SB-SB) comparisons, and the right graph the Nickel-Brass (F-SB) comparisons.
-2 0 2 4 6 8 10 12 14 16 0 0,1 0,2 0,3 0,4 0,5 0,6 0,7 0,8 0,9 1 fKNM(x) fKM(x)
Figure 18: Known Match vs Known Non Match distributions for the Consecutive Matching Cell fractions that resulted from breech face impression comparisons. The Known Match distribution is based on 498 Known Match scores, while the Known Non Match distribution is based on 10204 scores.
4.3
Likelihood ratios
For the calculation of the likelihood ratios, the results have shown a major problem for both the ACCF and CMC fraction for both types of traces. All four KNM distributions have extremely little data available at the right part of the distribution. The result of this is that their scores for that value are extremely low. As a further result, when calculating the LR as described in formula (2), P (E|Hd) will be very close to zero, and the resulting LR will be nonsensically high. This will be a problem when presenting results in court if this program were to be used in an actual case. Several solutions for this problem have been offered, all of which include putting a minimum and maximum on how high the LR can go. At the time of writing, no way to determine these boundaries has been agreed upon. For now, we will present a graph of the LR, evaluated only at the area where both the KM and KNM distribution had enough information (i.e. were not close to
0 0,01 0,02 0,03 0,04 0,05 0,06 0,07 0 10 20 30 40 50 60 70 80 90 100 KNM KM
Figure 19: Known Match vs Known Non Match distributions for the ACCF scores that resulted from breech face impression comparisons. The Known Match distribution is based on 498 Known Match scores, while the Known Non Match distribution is based on 10204 scores.
-2 0 2 4 6 8 10 12 14 16 0 0,1 0,2 0,3 0,4 0,5 0,6 0,7 0,8 0,9 1 F-F SB-SB F-SB
Figure 20: The Known Non Match distribution of the CMC fractions that resulted from breech face impres-sion comparisons, divided into the material specific comparisons Nickel-Nickel (F-F), Brass-Brass (SB-SB) and Nickel-Brass (F-SB).
zero). For the FPI, this is displayed in figure 24. For the BFI, this is displayed in figure 25.
0 0,2 0,4 0,6 0,8 1 1,2 1,4 1,6 1,8 2 0 0,1 0,2 0,3 0,4 0,5 0,6 0,7 0,8 0,9 1 F1-F2 SB1-SB2 FIX F-SB FIX
Figure 21: The Known Match distribution of the CMC fractions that resulted from breech face impression comparisons, divided into the material specific comparisons Nickel-Nickel (F-F), Brass-Brass (SB-SB) and Nickel-Brass (F-SB). -0,01 0 0,01 0,02 0,03 0,04 0,05 0,06 0,07 0,08 0,09 0,1 0 10 20 30 40 50 60 70 80 90 100 F-F SB-SB F-SB
Figure 22: The Known Non Match distribution of the ACCF scores that resulted from breech face impression comparisons, divided into the material specific comparisons Nickel-Nickel (F-F), Brass-Brass (SB-SB) and Nickel-Brass (F-SB).
0 0,005 0,01 0,015 0,02 0,025 0,03 0,035 0 10 20 30 40 50 60 70 80 90 100 F1-F2 SB1-SB2 FIX F-SB FIX
Figure 23: The Known Match distribution of the ACCF fractions that resulted from breech face impression comparisons, divided into the material specific comparisons Nickel-Nickel (F-F), Brass-Brass (SB-SB) and Nickel-Brass (F-SB). 0,01 0,1 1 10 100 0 0,02 0,04 0,06 0,08 0,1 0,12 0,14 0,01 0,1 1 10 15 20 25 30 35 40
Figure 24: The LRs of the firing pin impressions evaluated at points at which there was data available for both the KM and KNM distribution. These were determined by dividing the KM scores of figure 7 and 8 respectively by the KNM scores. Keep in mind that these graphs use a logarithmic scale. The figure on the right might look counter-intuitive near the left point, however in the range 15-18 there was little to no data for both the KM and the KNM there.
5
discussion
5.1
Repeatability and individuality
In general, for all traces we hope to see as little overlap between the KM and KNM graphs as possible. If there is a lot of overlap, the LR will be closer to 1 over a larger part of the domain of the functions. This means there are more scores that can be achieved which will provide only weak evidence for one of the hy-potheses. If there is little to no overlap, the LRs will be more ”extreme” (further away from 1, or closer to 0).
0,01 0,1 1 10 100 0 0,02 0,04 0,06 0,08 0,1 0,12 0,14 0,01 0,1 1 10 100 0 5 10 15 20 25 30 35 40
Figure 25: The LR scores of the breechface impressions evaluated at points at which there was data available for both the KM and KNM distribution. Keep in mind that these graphs use a logarithmic scale
A visual examination of the graphs in figures 7, 8, 18 and 19 seem to provide hopeful results. All KM and KNM distributions seem to be decently well separated. However, there appears to be a bit more overlap in the functions of the breech face impressions. The KM breech face impression distributions also seem to be a bit more concentrated to the left and in general seem to be more spread out over the domain. This can be clearly seen when comparing figure 8 and 19.
The LR graphs as seen in figure 24 and 25 seem to confirm what was suspected after a visual examina-tion: There appears to be more overlap in the KM and KNM scores of the breechface impressions than there is in the firing pin impressions. This can be seen when comparing the minima of the LR functions over the selected domain. For the firing pin impressions, these were around 0.02 and 0.05 for the CMC fraction and ACCF, respectively, while for the breech face impressions, these were only 0.1 and 0.1. A higher overlap between distributions results in likelihood ratios that are closer to 1 over the domain. When this happens, the evidence a certain score can provide is considered weaker.
5.2
Material specific results
After the visual examinations of the previous section, we can use statistical methods to confirm or disprove these claims. As mentioned, we will use the Wilcoxon rank-sum test for this. This is a non-parametric test (meaning it is always independent of the underlying statistical distributions) that can be used to find out if two samples came from the same distribution. We will therefore use the following hypotheses:
• H0: The two samples came from the same distribution • H1: The two samples came from different distributions
This corresponds to using a two sided test. We will set the signficance level α to 0.05, meaning we reject the null hypotheses if we find a p-value lower than 0.05. The results are shown in Tables 7-5
Table 2: The p values for the Wilcoxon rank sum test of the KNM distributions of the firing pin impressions. The significance level is set to 0.05
Figure type distribution 1 distribution 2 p significant?
7 CMC F-F SB-SB 0.22 No 7 CMC F-F F-SB 0.75 No 7 CMC F-SB SB-SB 0.09 No 9 ACCF F-F SB-SB 0.73 No 9 ACCF F-F F-SB 0.62 No 9 ACCF F-SB SB-SB 0.93 No
Table 3: The p values for the Wilcoxon rank sum test of the KM distributions of the firing pin impressions. The significance level is set to 0.05
Figure type distribution 1 distribution 2 p significant? 8 CMC F-F SB-SB 4.31 ∗ 10−10 Yes 8 CMC F-F F-SB 2.13 ∗ 10−12 Yes 8 CMC F-SB SB-SB 0.22 No 10 ACCF F-F SB-SB 8.34 ∗ 10−8 Yes 10 ACCF F-F F-SB 6.64 ∗ 10−10 Yes 10 ACCF F-SB SB-SB 0.34 No
Table 4: The p values for the Wilcoxon rank sum test of the KNM distributions of the breechface impressions. The significance level is set to 0.05
Figure type distribution 1 distribution 2 p significant? 16 CMC F-F SB-SB 1.83 ∗ 10−9 Yes 16 CMC F-F F-SB 1.53 ∗ 10−4 Yes 16 CMC F-SB SB-SB 0.91 No 18 ACCF F-F SB-SB < 1 ∗ 10−15 Yes 18 ACCF F-F F-SB < 1 ∗ 10−15 Yes 18 ACCF F-SB SB-SB 6.23 ∗ 10−12 Yes
Table 5: The p values for the Wilcoxon rank sum test of the KM distributions of the breechface impressions. The significance level is set to 0.05
Figure type distribution 1 distribution 2 p significant?
17 CMC F-F SB-SB 0.39 No 17 CMC F-F F-SB 2.01 ∗ 10−7 Yes 17 CMC F-SB SB-SB 0.01 Yes 19 ACCF F-F SB-SB 0.46 No 19 ACCF F-F F-SB 3.96 ∗ 10−5 Yes 19 ACCF F-SB SB-SB 0.04 Yes
The following results can be obtained from the tables: From Table 7, we can confirm that there are no significant differences in the primer materials when it comes to the KNM distributions of the firing pin impressions. This is the case for both the CMC fractions as displayed in figure 9 and the ACCF scores displayed in figure 11.
Almost the opposite seems to be true for the KNM distributions of the breechface impressions. When looking at table 4, the only not significant result was found when comparing the F-SB and SB-SB distributions of the CMC scores. This confirms our visual findings that there were some minor differences visible in the distributions in figures 20 and 22. As the sample sizes are fairly large (with the total amount of scores being
10204), even differences that appear minor when examined visually can still imply large statistical differences.
From table 3, another of our earlier visual suspicions appear to be confirmed. When looking at the firing pin impression data in figure 10 and figure 12, we can reject the hypotheses that the KM F-F comparisons follow the same distribution as the other KM comparisons for both the CMC fractions and the ACCF scores. This, combined with the observation that the F-F distributions appear to be focused more on the right than the other distributions, means that F-F comparisons were in fact performing better than the others in this study. It appears that softer primer materials do result in traces that are easier to analyze. On the other hand, the differences between the F-SB and SB-SB distributions were not found to be significant. So, with the data currently available we can not statistically confirm that one performs better than the other, although it would certainly appear so when examining the charts visually.
The opposite can be found in the KM distributions for the breechface impressions as displayed in table 5. Here, all test involving the F-SB comparisons were found to be significant, while the comparisons between F-F and SB-SB scores were not found to be significant. When looking at the graphs in figure 21 and figure 23, we see that the F-SB scores appear to be more concentrated near the left. This would mean that both the F-F comparisons and SB-SB comparisons significantly outperform the F-SB comparisons. This in turn implies that for breechface impressions, in order to get the highest score when using this method, it is mainly important that comparisons are performed between cartridge cases with similar primer hardness. The type of the material is less important in the comparisons than whether or not the cartridge cases are made of the same material. For firing pin impressions this does not seem to be the case.
6
Conclusion
In this study, an automated comparison system based on the CMC algorithm was tested for both firing pin impressions and breechface impressions. For both CMC fraction and ACCF scores, there was a clear distinction between the known match and the known non match distribution for both traces. However, for breechface marks there was more overlap between the KM and KNM distributions, and in general the KM distribution was spread out further. This would mean that firing pin impressions performed slightly better and gave clearer scores.
Also for the breechfaces, a certain subtype of the S&B ammunition was found to have a dot-like pattern that severely impacted the ability of the program to properly compare cartridge cases. This pattern is most likely the result of the production process of the cartridge cases. This is something that should definitely be kept in mind if this system is to be used in actual cases later, as this can seriously affect the scores the program provides.
For firing pin impressions, the softer nickel was found to provide better results than the harder brass. Comparisons between cartridge cases with nickel primers performed significantly better than comparisons between cartridge cases with brass primers, or comparisons between one nickel and one brass primer. Be-tween the other two types of comparisons no significant difference was found.
For breechface impressions, another result was found. Here it was found that comparisons with one nickel primered cartridge case and one brass primered cartridge case performed significantly worse than compar-isons between cartridge cases that either had both a nickel primer or both a brass primer. There were no significant differences found between comparisons of cases that had both a nickel primer and those that both had a brass primer. This would mean that the type of the primer material is not as relevant as whether or not the primer material was the same in both cartridge cases.
7
Acknowledgements
The author would like to thank Caroline Gibb MSc, who assisted at the beginning of the experiment with the scanning of the casts. Also thanks go to my fellow intern Jeroen Letteboer, who helped me out on many
occasions during this project.
I would also like to thank my supervisor Martin Baiker-Sørensen and my examiner Erwin Mattijssen, for their guidance and assistance during the process, and their valuable feedback.
References
[1] Riva F, Champod C. (2014). Automatic comparison and evaluation of impressions left by a firearm on fired cartridge cases. Journal of Forensic Sciences, 59 (3) 637-647
[2] Committee on Identifying the Needs of the Forensic Sciences Community, National Research Council, Strengthening Forensic Science in the United States: A Path Forward, 2009
[3] PCAST, Report on Forensic Science in Criminal Courts: Ensuring Scientific Validity of Feature-Comparison Methods, 2016
[4] Song J. Proposed “Congruent Matching Cells (CMC)” Method for Ballistic Identification and Error Rate Estimation. AFTE Journal, 47(3), 2015, 177-185
[5] Tai X, Eddy W. A Fully Automatic Method for Comparing Cartridge Case Images. Journal of forensic sciences, 63(2), 2018, 440-448.
[6] Baiker M, Keereweer I, et al. Quantitative comparison of striated toolmarks. Forensic science interna-tional, 242, 2014, 186-199.
[7] Chumbley L, Kreiser J, et al. Clarity of microstamped identifiers as a function of primer hardness and type of firearm action. AFTE journal, 44(2), 2012, 145-155
[8] Manzalini V, Michele F, et al. The effect of com-position and morphological features on the striation of .22LR ammunition. Forensic Science International, 296(2019), 2019, 9-14
[9] Brand N. Bewijskracht en correlatie van Glock slagpingatschaafsporen. Final report HBO Forensic Science, Leeuwarden. 2017
[10] https://www.lociforensics.nl/forensic-sil/
[11] https://www.alicona.com/en/products/infinitefocussl/
[12] Vorburger T, Yen J et al. Surface Topography Analysis for a Feasibil-ity Assessment of a National Ballistics Imaging Database. Retrieved from https://www.nist.gov/system/files/documents/pml/div683/grp02/nistir2007-7362.pdf
[13] Song J. Proposed NIST Ballistics Identification System (NBIS) using 3D Topography Measurements on Correlation Cells. AFTE Journal, 45(2), 2013, 184-189
[14] Chu W, Tong M, Song J. Validation Tests for the Congruent Matching Cells (CMC) Method Using Cartridge Cases Fired with Consecutively Manufactured Pistol Slides. AFTE Journal, 45(4), 2013, 361-366
[15] Wasserman L, All of nonparametric statistics, 2006. New York, NY. Springer
A
Firearms used in the study
The following is a table of all firearms used in the study. There were some firearms for which the type and serial number was unknown.
File name of print
Serial number
Model
Region
6288
2013076288
26
Tilburg 2
6289
2013076288
26
Tilburg 3
6290
2013076288
26
Tilburg 1
0023
2011290023
17
Amsterdam_1_4
0066
2014080066
26
Tilburg 2
0105
2012140105
17L
Tilburg 3
0110
2015020110
17
Amsterdam_12
0276
2015120276
17 gen4
Amsterdam_9
0312
2015030312
17
Amsterdam_14
03127
2011.08.03.127
26
NFI_oud_2
0454
2015120454
26
Tilburg 3
0466
15-2012060466
19
Overig_1
05066
2009.06 05 066
26
NFI_oud_1
0647
2011200647
19
Amsterdam_1_2
0806
2011180806
19
Overig_2
0817
2015110817
19
Amsterdam_9
0826 (56)
2015030826
26
Amsterdam_14
0826 (58)
2015030826
26
Amsterdam_14
0876
1500-2015281606
19
Haaglanden 3
100188
2012200054
19
Rotterdam_3
100550
2012227709
19
Rotterdam_3
10331
2007.12.10.331
19
NFI_oud_3
104850
2013141799
26
Rotterdam_4
105681
2013282567
26
Rotterdam_5
10621
2013336160
17
Rotterdam_5
1144
2012229940
26
Rotterdam_2
12353
12-353
26
Elst 1
12470
2011125112
17
Elst 1
12474
2012028434
19
Elst 1
12479
2012060938
17
Elst 1
13153
CPT934
17
Elst 1
13216
13-216
26
Elst 2
13258
13-258
26
Elst 2
13382
13-382
26
Elst 2
13393
13-393
26
Overig_1
13411
13-411
19
Elst 2
14065
14-065
26
Elst 1_1
14079
14-079
26
Elst 1_1
14130
14-130
17
Elst 1_2
14134
14-134
26
Elst 1_1
14216
14-216
26
Elst 1_1
14242
14-242
26
Elst 1_2
1425
2010271277
Rotterdam_2
14250
14-250
26
Elst 1_1
1436
2014131436
26
Amsterdam_16
15027
15-027
17
Elst 1_2
15085
15-085
19
Elst 1_2
15264
15-264
26
Elst 1_2
15431
2013054981-15431
17
Tilburg 1
16049
16-049
Elst
16058
16-058
Elst
16239
16-239
Elst
1697
2012091697
19
Amsterdam_1_2
17021
01.05.17.021
26
NFI_oud_2
Firearm
1734
2009211734
26
Amsterdam_5
1757
15-2011000973
26
Haaglanden 2
18069
08.04.18.069
19
NFI_oud_2
1815
1500-2014288193
17
Haaglanden 4
1831
2013231831
26
Amsterdam_1_2
1852
2011151852
17L
Overig_1
1891
2013191891
19
Amsterdam_1_3
1969
2011261969
17
Amsterdam_7
20015
2009.11.20.015
26
NFI_oud_3
2065
2011032065
26
Tilburg 2
21031
05.06.21.031
26
NFI_oud_3
21042
2010.12.21.042
19
NFI_oud_4
2166
2011282166
26
Amsterdam_1
2168
16-20140718671
26
Haaglanden 2
2178
2012332178
19C
Amsterdam_6
22085
2004.06.22.085
26
NFI_oud_3
22116
2011.08.22.116
26
NFI_oud_4
22116II
NFI
2292
1500-2015147297
19
Haaglanden 1
23038
2004 01 23 038
17C
NFI_oud_4
2319
2011318923
Overig_2
2365
2013302365
26
Amsterdam_5
2370
1500-2015042360
19 automaat
Haaglanden 1
2373
2011302373
26
Amsterdam_1_1
2374
1500-2015042551
19
Haaglanden 3
2422
1500-2014232999
17
Haaglanden 4
2488
1500-2014191969
19
Haaglanden 3
2587
2014302587
17
Amsterdam_14
2594
2011212594
19
Overig_2
26008
2009.03.26.008
26
NFI_oud_4
26017
2001.04.26.017
17
NFI_oud_1
28058
08.10.28.058
19
NFI_oud_2
2816
15-2011009851
17
Haaglanden 1
2816
2014072816
26
Amsterdam_15
2919
2014092919
34
Tilburg 2
2958
2015032958
17
Amsterdam_10
30001
2012.12.30.001
19
Overig_1
3001
15-201109997
19
Haaglanden 4
30064
2009.01.30.064
19C
NFI_oud_1
3046
2015063046
19
Amsterdam_9
31018
2000.10.31.018
26
NFI_oud_4
3178
2010024620
Rotterdam_4
3190
16-2012124307
19
Haaglanden 3
3215
2014213215
19
Amsterdam_13
3215-75
2014213215
17
Amsterdam_16
3227
2014143227
19
Amsterdam_13
3294
2009093294
17
Tilburg 1
3298
2015023298
26
Amsterdam_9
3325
2013193325
19
Amsterdam_1_4
3334
2010233334
19
Amsterdam_8
3408
2013313408
26
Amsterdam_13
3502
2009213502
26
Amsterdam_8
3512
2014173512
26
Amsterdam_15
3581
2014113581
26
Tilburg 1
3613
2013203613
19
Amsterdam_1_1
3793
2011373793
19
Overig_2
3837
2011183837
17
Amsterdam_1_4
3849
2015043849
19
Amsterdam_11
3875
2013143875
19
Amsterdam_6
3883
2011.11.15.054
19
NFI_oud_2
4000
2012508938
19C
Rotterdam_2
4039
2012582590
19
Rotterdam_2
4163
2011264163
19
Amsterdam_1_3
4277
2011.02.03.127
-
NFI_oud_1
4286
1500-2015358717
19
Haaglanden 1
4430
1500-2015112600
17
Haaglanden 2
4510
2010075981
Rotterdam_1
4650
2009334650
19C automaat
Amsterdam_8
4706
2009134706
19C
Overig_1
4773
2009 01 30064
19
NFI_oud_3
4831
2014314831
17 gen4
Amsterdam_12
4948
2013044948
19C
Amsterdam_1_4
5018
2009175018
26
Amsterdam_6
5025
2012285025
19
Amsterdam_1_4
5045
2011025045
26
Tilburg 3
5121
2011315121
26
Amsterdam_1
5227
2009217351
19
Amsterdam_1_2
5229
2015095229
19
Amsterdam_10
5234
2009217351
19
Amsterdam_1_2
5298
2012335918
26
Rotterdam_2
5318
2012354204
Rotterdam_3
5539
2012235539
26
Tilburg 1
5722
Haaglanden
5755
2009165755
19
Amsterdam_7
5777
2015155777
19
Amsterdam_12
5787-68
2014075787
17
Amsterdam_15
5787-73
2014075787
17
Amsterdam_15
5881
2012205881
26
Amsterdam_5
5931
2015135931
19
Amsterdam_11
6008
2014296008
19
Amsterdam_13
6160
2009336160
17
Amsterdam_7
6297
2010073278
17
Rotterdam_4
6337
Amsterdam
6463
2015026463
19
Amsterdam_14
6543
2013066543
19
Tilburg 3
6558
15-2012110093
19
Haaglanden 3
6708
2014006708
26
Amsterdam_16
6821
2009309746
Rotterdam_1
6867
2010.06.25.077
17
NFI_oud_1
7065
2009147065
19C
Amsterdam_7
7071
2014227071
17
Amsterdam_13
7187
2014237187
19
Amsterdam_12
7246
1500-2016046876
26
Haaglanden 1
7256
2015127256
19
Amsterdam_10
7260-2877
2011107260
Overig_2
7260-9476
2011107260
Overig_2
7351
2009217351
19C
Amsterdam_1_1
7398
2015067398
17
Amsterdam_11
7417
2015117417
17
Amsterdam_9
7550
2012057550
26
Amsterdam_1_3
7604
2014297604
19
Amsterdam_12
7736
2009437867
26
Rotterdam_1
7845
2009227845
26
Amsterdam_5
7945
2015147945
17
Amsterdam_10
8017
2011306553
17
Amsterdam_1_1
8027
2014068027
19
Amsterdam_15
8030
2011306553
19
Amsterdam_1_2
8044
2009218044
26
Amsterdam_6
8148
2013296605
19
Rotterdam_5
8172
2014108172
17
Amsterdam_16
8217
2013088627
26
Rotterdam_5
8359
2010295631
Rotterdam_1
8364
2009.03.09.105
26
NFI_oud_3
8392
1500-2015098392
17
Overig_1
8567
2013068567
19C
Amsterdam_8
8585
2013170830
17
Rotterdam_5
8701
2013030400
Rotterdam_5
8743
2013236255
17
Rotterdam_5
8837
2013308837
26 gen4
Amsterdam_16
8899
Rotterdam
8951
2015138951
19
Amsterdam_10
9052
2012069052
26
Tilburg 3
9264
2015029264
17
Amsterdam_11
93269
2009420951
34
Rotterdam_1
93270
PL2673/09-111701
Rotterdam_4
9355
2013140504
Rotterdam_4
9361
2013140504
26
Rotterdam_4
9527
2009149527
17
Amsterdam_5
9571
2012149571
17
Amsterdam_1_1
9753
079753
17
Tilburg 2
9887
2009259887
19
Amsterdam_6
K271
1500-2015131271
26
Haaglanden 2
W390
15-2011092226
17C
Haaglanden 2
B
Parameters and other settings used in collecting the data
B.1
Measuresuite parameters
Table 6: The parameters used in Measuresuite, the program that came with the Alicona InfiniteFocusSL 3d-measurement system . All parameters not mentioned in this table were left on their default value
Parameter Value Notes
Optic 20x SX Brightness 10.0 ms Contrast 1.00
Replica mode On This is found in expert measurement settings Vertical resolution (VR) 200 nm This changes later
Horizontal resolution (HR) 1.5 µm This changes later
Lateral downsampling 2 This is found in advanced measurement settings. This sets VR to 100 nm and HR to <2 µm Dimension 2x1/4x4 2x1 for FPI, 4x4 for BF
B.2
Scratch parameters
Table 7: The parameters used in Scratch. All parameters not mentioned in this table were left on their default value
Parameter Value Notes
Version Scratch 3.0.3-SNAPSHOT-5776b7c
Cell size x 125/400 125 for FPI, 400 for BF Cell size y 125/400 125 for FPI, 400 for BF N reg image reduction 2
N cell reg image reduction 2 Shift angle max 35 Shift angle min -35
Shift x max 350/750 350 for FPI, 750 for BF Shift x min -350/-750 -350 for FPI, -750 for BF Shift y max 350/750 350 for FPI, 750 for BF Shift y min -350/-750 -350 for FPI, -750 for BF
C
Some example comparison
In this appendix, three example comparisons will be shown. In the examples, first we cover what the dataset looks like, then we create a figure as in figure 6, and finally we show screenshots of the resulting tables
C.1
a ”normal” KM comparison for FPIs
In this example, we will be comparing the firing pin impressions of F1 0817 Amsterdam and F2 0817 Amsterdam.
Figure 26: The 3D data of the firing pin impressions of F1 0817 Amsterdam and F2 0817 Amsterdam after they have been preprocessed. The colors represent the ”height” of the original data, with purple being away from the viewer, and yellow being toward them
Figure 27: A comparison of the firing pin impressions of F1 0817 Amsterdam and F2 0817 Amsterdam, with cells visible. 33 of the 42 total cells were matching congruent cells. Cells that had no equivalent in the other data set are displayed in red in the left image
Figure 28: Screenshots of the result of the comparison of F1 0817 Amsterdam and F2 0817 Amsterdam. We are most interested in the ACCF, or Correlation Coefficient in the left table (72.12 %) and the CMC fraction in the right table (78.57 %).
C.2
a KM comparison with a damaged case
In this example we will compare SB1 68 Amsterdam with SB2 68 Amsterdam. While SB2 5787-68 Amsterdam looks as expected, SB1 5787-5787-68 Amsterdam suffered a burst primer during firing which severely impacted the area of the firing pin impression.
Figure 29: The 3D data of the firing pin impressions of SB2 68 Amsterdam with SB1 5787-68 Amsterdam after they have been preprocessed. The colors represent the ”height” of the original data, with purple being away from the viewer, and yellow being toward them. As can be seen, the firing pin im-pression area of SB1 5787-68 Amsterdam is severely impacted by the damage, to the point that the original patterns that were appearing are almost not visible anymore.
Figure 30: A comparison of the firing pin impressions of SB2 68 Amsterdam with SB1 5787-68 Amsterdam, with cells visible. Only 2 of the 32 total cells were matching congruent cells. Cells that had no equivalent in the other data set are displayed in red in the left image. As expected, the damage seriously impacted the amount of CMCs.
Figure 31: Screenshots of the result of the comparison of SB2 68 Amsterdam and SB1 5787-68 Amsterdam. We are most interested in the ACCF, or Correlation Coefficient in the left table (20.37 %) and the CMC fraction in the right table (6.25 %). Here too we can see the effect of the damage, on both the CMC fraction and the ACCF score.
C.3
A KNM comparison
In this final example we will be comparing F1 2374 Haaglanden with SB2 4948 Amsterdam. These is a KNM comparison for which the elements were chosen at random. We expect the results from the previous example to be closer to the following results than to the results of the ”normal” KM comparison.
Figure 32: The 3D data of the firing pin impressions of F1 2374 Haaglanden and SB2 4948 Amsterdam after they have been preprocessed. The colors represent the ”height” of the original data, with purple being away from the viewer, and yellow being toward them
Figure 33: A comparison of the firing pin impressions of F1 2374 Haaglanden with SB2 4948 Amsterdam with cells visible. Only 2 of the 36 total cells were matching congruent cells. Cells that had no equivalent in the other data set are displayed in red in the left image. This appears similar to the comparison with the damaged cartridge case as covered in the previous example
Figure 34: Screenshots of the result of the comparison of F1 2374 Haaglanden with SB2 4948 Amsterdam. We are most interested in the ACCF, or Correlation Coefficient in the left table (18.89 %) and the CMC fraction in the right table (5.56 %). As expected, these are both significantly lower than the ”normal” KM comparison, and similar to the KM comparison with the damaged case
D
Graphs with type 8 cartridge cases still included
-2 0 2 4 6 8 10 12 14 16 0 0,1 0,2 0,3 0,4 0,5 0,6 0,7 0,8 0,9 1 fKNM(x) fKM(x)Figure 35: The Known Match and Known Non Match distributions of the CMC Fraction of the breech face impressions. This figure is equivalent to figure 18, however with data of type 8 still included in the KM distribution. There is significantly more overlap between the graphs in this figure versus the ones in figure 18. 0 0,01 0,02 0,03 0,04 0,05 0,06 0,07 0 10 20 30 40 50 60 70 80 90 100 fKNM(x) fKM(x)
Figure 36: The Known Match and Known Non Match distributions of the ACCF scores of the breech face impressions. This figure is equivalent to figure 19, however with data of type 8 still included in the KM distribution. There is significantly more overlap between the graphs in this figure versus the ones in figure 19.
-0,5 0 0,5 1 1,5 2 2,5 3 3,5 4 0 0,1 0,2 0,3 0,4 0,5 0,6 0,7 0,8 0,9 1 F1-F2 SB1-SB2 F-SB
Figure 37: The distributions of the CMC fractions of the known matches of the breech face impressions, split into the different types of comparison nickel-nickel (F1-F2), brass-brass (SB1-SB2) and nickel-brass (F-SB). This figure is equivalent to figure 21, however with data of type 8 still included. The graphs for both SB1-SB2 and F-SB are significantly lower than in figure 21
0 0,005 0,01 0,015 0,02 0,025 0,03 0,035 0 10 20 30 40 50 60 70 80 90 100 F1-F2 SB1-SB2 F-SB
Figure 38: The distributions of the ACCF scores of the known matches of the breech face impressions, split into the different types of comparison nickel-nickel (F1-F2), brass-brass (SB1-SB2) and nickel-brass (F-SB). This figure is equivalent to figure 23, however with data of type 8 still included. The graphs for both SB1-SB2 and F-SB are significantly lower than in figure 23
0,1 1 10 100 0 0,02 0,04 0,06 0,08 0,1 0,12 0,14 0,1 1 10 100 0 5 10 15 20 25 30 35 40
Figure 39: The LRs of the breech face distributions with type 8 still included, evaluated at points at which there was data available for both the KM and KNM distribution. This figure was made by dividing the KM distribution of figures 35 and 36 by the KNM distribution of their respective graph. Keep in mind that these graphs use a logarithmic scale. When compared to the graphs in figure 25, the entire left area (the domain 0-0.06 for the left figure, and 0-15 in the right figure) is closer to 1 in this figure. This implies that there is even more overlap in these functions, and getting a score like this will be even weaker evidence.
