Specific Source versus Common Source approach to determine the evidential strength of automated comparisons of firing pin and breech face impressions on Glock cartridge cases

(1)

Specific Source versus Common Source approach to determine the

evidential strength of automated comparisons of firing pin and

breech face impressions on Glock cartridge cases

Research Project

Jeroen Letteboer // 10748873

Master Forensic Science, University of Amsterdam

Netherlands Forensic Institute (NFI)

36 EC

February – August 2020

Date of submission: 24-08-2020

Supervisor:

dr. ir. Martin Baiker-Sørensen

Netherlands Forensic Institute (NFI)

Examiner:

drs. Erwin Mattijsen

University of Amsterdam (UvA)

Netherlands Forensic Institute (NFI)

(2)

2

Abstract

Firearms are often used in crimes. The comparison of marks on cartridge cases can then play an important role in the forensic investigation. Depending on the situation a Specific Source (SS) or Common Source (CS) approach can be used to determine the strength of the evidence. Since the CS approach takes less time, it is of interest to know whether it can be used for situations that in theory would require a SS approach. Therefore the following research questions are studied in this report: ‘Is

there a significant difference between Common Source and Specific Source distributions for firing pin impressions on Glock cartridge cases?’ and ‘Is there a significant difference between Common Source and Specific Source distributions for breech face impressions on Glock cartridge cases?’. For this study

20 Glock pistols were used and with each pistol 25 test shots with Fiocchi 9mm Luger cartridges were fired. A 3D measurement system was used to scan the cartridge cases. The software Scratch was used to crop the firing pin and breech face impression from each scan. Similarity scores between the different cartridge cases were calculated in order to build SS and CS distributions for all 20 firearms. Statistical tests were performed to find out if there is a significant difference in the distributions for both mark types. The results showed that for a substantial part of the firearms the SS and CS distributions are significantly different for both the firing pin impressions as well as for the breech face impressions on Glock cartridge cases. Therefore it is concluded that it is unjustified to use the CS approach for a specific source situation, since the difference in the distributions could lead to a difference in the evidential strength.

Keywords: Glock cartridge cases, specific source, common source, firing pin impression, breech face impression, strength of the evidence

(3)

3

1. Introduction

Firearms are frequently used in criminal cases. The examination of marks on cartridge cases and bullets can play an important role during these cases. The examination results are often used to study whether a specific firearm was used in a crime or not. The firearm examination can help with the reconstruction of a crime as well (Jackson & Jackson, 2017).

A cartridge consists of four parts: the primer, powder charge, cartridge case and bullet. Once the firearm is fired the firing pin of the weapon hits the primer, this causes the powder in the cartridge cases to deflagrate. The deflagration leads to a large pressure and the bullet is pushed through the barrel. The pressure also causes a recoil of the cartridge case and the cartridge case is pushed back against the breech face of the firearm, before the extractor removes the cartridge case out of the firearm. During this process several striation and impression marks are left on the cartridge case. For example the firing pin hitting the primer leaves the firing pin impression and the recoil of the cartridge case against the breech face causes the breech face impression. These are the two marks that are of interest in this study and example of a cartridge case with these marks can be seen in Fig. 1.

Figure 1: Several marks on a cartridge case including the breech face impression (BFI) and firing pin impression (FPI) with a scale in mm to show the size of the marks, figure adapted from Roth et al. (2015).

Some of the impression and striation marks left on a cartridge case can be characteristic for an individual firearm. The different marks are created by different parts of the firearm and these different parts are independently produced from each other. Therefore the different mark types are seen as independent evidence from each other (Morris et al., 2016). When a cartridge case is brought in for forensic examination, the marks on the cartridge case can be compared with other questioned cartridge cases or with cartridge cases from test fires of a suspected firearm. This is done by comparing the similarities and differences of a specific mark and then combining the results of all marks (Hamby et al., 2016). Currently, forensic firearm examiners analyse marks using 2D comparison microscopy.

(5)

5 Illuminating oblique light is used to highlight the marks, and then the examiner aligns the marks of two cartridge cases by hand. In this process the differences and similarities of the set of marks are subjectively determined based on the training and experience of the examiner. The examiner also makes a judgment regarding the strength of the evidence. This requires knowledge about the individuality and repeatability about each type of mark, which is estimated by the examiner (Riva & Champod, 2014).

Over the past few years the National Academy of Sciences (NAS) and the President’s Council of Advisors on Science and Technology (PCAST) have expressed their concern regarding the subjectivity in current forensic firearm examination (Community et al., 2009; PCAST, 2016). The lack of a statistical underpinning for determining the strength of the evidence is also seen as a negative aspect of the current approach (Riva & Champod, 2014; Roth et al., 2015). Therefore there is a need for a new method for forensic firearm examination that has less subjective steps and has a solid statistical foundation for the individuality and repeatability for every type of mark. As a possible solution for this problem, several automated methods have been developed for firearm mark comparison using algorithms (Riva et al., 2017; Song, 2015). Since an algorithm determines the degree of similarity between two marks, this is more objective than the current method. The repeatability and individuality of a specific mark is then based on the distribution of scores in a reference database. At the Netherlands Forensic Institute (NFI) an automated system with an algorithm is developed that is able to present a measurement of similarity for striation marks (Baiker et al., 2014). This system is proven to be successful to present a degree of similarity for aperture shear marks on Glock cartridge cases (Brand, 2017; Gilse, 2018). This study aims to present a degree of similarity for firing pin impressions and breech face impressions on Glock cartridge cases. The method is based on the what Song explained in his study (Song, 2015).

The automated system makes use of similarity scores to compare marks on cartridge cases. To determine the strength of the evidence, it’s required to know the what the possibility is of observing the evidence given both the hypothesis of the prosecution and the defence. Therefore, reference distributions of similarity scores are needed in order to know how frequent a certain similarity score occurs under both hypotheses. These reference distributions lead to a statistical underpinning of the strength of the evidence. There are two different approaches to build reference distributions: Common Source (CS) or Specific Source (SS). These approaches are built up in a different way, which approach is used depends on the task that the forensic examiner is asked to perform (Ommen & Saunders, 2018; Tai & Eddy, 2018). Both approaches consist of a same source and a different source distribution. A same source distribution is built up with similarity scores of cartridge cases that are fired with the same firearm, whilst a different source distribution is built up with similarity scores of cartridge cases that are fired with different firearms.

For the CS approach a reference population of firearms is used to set up the same source and different source distributions. The firearms that are included in the population leave similar types of marks as those that are found on the crime scene cartridge case, for example because they are of the same brand of firearms. The firearm that potentially fired the crime scene cartridge case is not included to build the CS distributions. Each firearm in the population fires a number of test shots. The similarity scores of cartridge cases of the same firearm are used to create the same source distribution. The similarity scores between cartridge cases of different firearms are used to build the different source distribution. Since the firearm that potentially fired the crime scene cartridge case is not used to build the distributions, the distributions don’t change for each investigation. The same distributions can be used for another investigation of cartridge cases with similar mark types. With the SS approach on the other hand, the same source and different source distributions change for every specific case. The

(6)

6 specific source for this approach is the firearm that is brought in as evidence. Test shots are fired with this specific firearm and the similarity scores of those cartridge cases are used to build the same source distribution. Similarity scores between the cartridge cases of the specific firearm and cartridge cases of other firearms are used to build the different source distribution. The CS approach is less time consuming, since it doesn’t require a new set of distributions to be created for every new investigation. The SS approach on the other hand is more specific, because it takes specific tool properties of a firearm into account (Ommen & Saunders, 2018).

Theoretically, it depends on the task that the firearm examiner is asked to perform whether a CS or SS approach is used. The CS approach is used when it is questioned whether two cartridge case are fired from the same, unknown firearm. Since the firearm is unknown in that scenario, the specific source of the cartridge cases can’t be determined. The SS approach is used when the question is whether a crime scene cartridge case is fired from a specific firearm, since the specific properties of the firearm are then taken into account. In practice however the CS approach is sometimes used to answer a SS question, because this is a lot less time consuming (Ommen & Saunders, 2018).

It is of great interest whether it is justified to use the CS approach for situations that theoretically require a SS approach. In other words, it is important to know whether the use of a CS approach over a SS approach leads to a significant change in the strength of the evidence. Therefore, this study will investigate the difference between the CS and SS approach for firing pin and breech face impressions on Glock cartridge cases. Same source and different source distributions are created using both the CS and SS approach for both mark types in order to answer the following two research question:

‘Is there a significant difference between Common Source and Specific Source distributions for firing pin impressions on Glock cartridge cases?’

‘Is there a significant difference between Common Source and Specific Source distributions for breech face impressions on Glock cartridge cases?’

2. Material and Methods

2.1 Cartridge case collection

For this study 20 Glock pistols of models 17, 19 and 26 were used, an overview of the firearms can be seen in Appendix A. These three models of Glock pistols leave similar type of firing pin and breech face impressions on cartridge cases. The firearms were taken from the collection of the NFI. 25 test shots were fired with each firearm, the cartridge cases are all Fiocchi 9mm Luger with brass as both the primer and casing material. This results in a total collection of 500 cartridge cases. Each cartridge case is scanned with either the Alicona InfiniteFocusSL or the Alicona InfiniteFocusG5 optical 3D measurement system to measure the 3D surface of the cartridge case. Lighting on the cartridge cases can lead to reflections that might complicate the accurate acquisition of the surface data. Therefore casts were made with Forensic Sil to prevent this issue and assure the accuracy of the data. The Glock pistols and cartridge cases in this study are the same as those used by Van Gilse in her study on aperture shear marks (Van Gilse, 2018). The settings of the Alicona 3D measurement systems that were used for this study can be seen in Table 1 below.

(7)

7

Table 1: Setting of the Alicona 3D measurement systems for scanning the Forensic Sil casts of the cartridge cases.

Parameter Setting Vertical Resolution (VR) 200 nm Lateral Resolution (LR) 1,5 μm Subsampling Factor 2 Exposure time ± 11 ms Contrast 1 Magnification 20x Field of view 4 x 4 (3,8 mm x 3,8 mm)

Replica mode Enabled

2.2 Quantitative comparison of firearm marks

The software program Scratch is used to compare 3D datasets of marks with each other. The program is specially developed by the NFI and the National Institute of Standards and Technology (NIST) to compare marks on cartridge cases and bullets. Scratch shows a 2D image of the entire breech face area of the cartridge case that was scanned, whilst the z-value of the surface is expressed by a colour range. Next the firing pin impression is manually selected for each cartridge case by placing a layer on the area of the scan that is the firing pin impression. The firing pin impression is then saved using plane levelling and the R2 filter from Scratch. Similarly, the breech face impression is selected by placing a layer over the area of interest. The breech face impression is saved by using plane levelling and the R2 filter as well. An example of a scan of the entire breech face area, the breech face impression and the firing pin impression is shown in Fig. 2.

Figure 2: Scan of the entire breech face area loaded into scratch and the marks of interest cropped from the scan. The colour range on the images represent the z-value of the 3D scan.

A: Scan of the entire breech face area of a cartridge case in (size 3800 x 3800 pixels; 3,8 x 3,8 mm). B: The breech face impression, cropped and filtered with Scratch (size 2750 x 3100 pixels; 2,8 x 3,1 mm). C: The firing pin impression, cropped and filtered with Scratch (size 1250 x 700 pixels; 1,3 x 0,7 mm)

The next step is to compare marks from different cartridge cases with each other in order to obtain similarity scores. The value of the parameters in Scratch that are used to crop the marks and for these comparisons can be seen in Table 2 below.

(8)

8

Table 2: Setting used in Scratch to crop firing pin and breech face impressions from the original scan and the setting used for comparing marks with each other.

Parameter Firing pin impression Breech face impression

Levelling Plane Plane

Filter R2 R2

Mark size (varies between marks)

1000-1300 x 650-800 2700-3000 x 3100-3400

Cell size 125 x 125 400 x 400

n Reg Image reduction 2 2

n Cell Reg Image reduction 2 2

Shift Angle range -35 to +35 -35 to +35

Shift X and Y range -350 to +350 -750 to +750

In this study two methods to calculate a similarity score between marks on cartridge cases are used, namely the Areal Cross-Correlation Function (ACCF) and Congruent Matching Cells (CMC) methods. These two methods will be explained in the next sections.

2.2.1 Areal Cross-Correlation Function (ACCF)

Prior to the calculation of the ACCF the marks are globally aligned first. This global alignment is done by shifting one of the marks in the horizontal and vertical direction as well as rotating the mark. The alignment is done to get the maximum value of the similarity score between the two marks. If one cartridge case is scanned under a slightly different angle than another, the alignment correct this. This increases the repeatability and minimizes the user variability of the system. The ACCF is a form to express the similarity of two 3D data sets. The data set of a mark is given as a two dimensional matrix with the length and width of the 3D image, whilst the height is the value of the matrix at that point. If the data sets of two marks are the two dimensional matrices A and B, with M x N as dimensions, µ the mean of the matrix and σ the standard deviation of the matrix, then the ACCF can be calculated using equation 1:

𝐴𝐶𝐶𝐹 =

1 𝑀×𝑁 ∑ ∑ (𝐴𝑖𝑗−𝜇𝐴)(𝐵𝑖𝑗−𝜇𝐵) 𝑀 𝑖=1 𝑁 𝑗=1 𝜎𝐴×𝜎𝐵

(1)

The value of the ACCF is between -1 and 1. If two datasets are exactly equal, the ACCF is 1. If two data sets are closely related the value is close to one, whilst the value is closer to zero when there is no relation between the data sets. If two data sets are equal, but with the z-value mirrored along the xy-surface the ACCF is equal to -1. The ACCF can also be expressed as a percentage, between -100 and 100%. In order to find the maximum value of the ACCF between two data sets, Scratch translates and rotates the marks; the parameters that are used can be seen in Table 2. The value of the ACCF% is a similarity score between two marks (Vorburger et al., 2007). An example of the calculation of the ACCF% of two firing pin impressions on cartridge cases of the same firearm can be seen in Fig. 3.

(9)

9

Figure 3: The calculation of the ACCF% of two firing pin impressions of cartridge cases, that were fired with the same firearm. The ACCF% for this comparison was 85,55%.

A: The filtered surface of the firing pin impression of cartridge case A, with the z-value expressed by the colour. B: The filtered surface of the firing pin impression of cartridge case B, with the z-value expressed by the colour. C: The difference in the surface of the firing pin impressions of cartridge case A and B, expressed by the colour. D: The distribution of the ACCF% across the firing pin impression surface.

2.2.2 Congruent Matching Cells (CMC)

Another method to get a similarity score between two marks is the calculation of the CMC. The algorithm behind this method is explained in an article by Song (Song, 2015). When two marks are compared, the marks are split in a number of cells. The number of cells depends on the size of the mark and the parameters for the cell size that are used, the parameters used in this study can be seen in Table 2. Each cell is compared to a cell on the same location in the other mark. In order for two marks to have a congruent matching cell, a number of requirements have to be met:

- The cross-correlation function between the areas of the cells is above the threshold value - The registration angles for the cell pairs are similar

- The x-y spatial distribution pattern of the cells is congruent, or at least nearly so

The absolute number of congruent matching cells can be used as a similarity score between the two marks according to the Song’s article (Song, 2015). In this study the fraction of CMCs of the total amount of cells is also used as a similarity score. An example of the measurement of the number of CMCs of two firing pin impressions on cartridge cases of the same firearm can be seen in Fig. 4.

(10)

10

Figure 4: The calculation of the number of congruent matching cells (CMC) of two firing pin impressions on cartridge cases of the same firearm. In this example 43 of the 48 cells are CMCs; that is a fraction of 0,8958.

A: The firing pin impression of cartridge case A split into cells, the colour of the mark expresses the z-value of the mark. The red cells do not have a matching cell in the other mark, whereas the black cells do have a matching cell in the other mark. B: The firing pin impression of cartridge case B split into cells, the colour of the mark expresses the z-value of the mark. The cells all match with a cell of the mark on cartridge case A.

2.3 Determination of the evidential strength

The calculated similarity scores are used to build same source and different source distributions for both the CS and SS approach for all 20 firearms. The databases are built for both the firing pin impression and the breech face impressions using the ACCF%, the number of CMCs and the CMC fraction as similarity scores. How the CS and SS databases are build up is explained in the sections below.

2.3.1 Common Source (CS)

For each of the 20 firearms same source and different source distributions are created. To build the same source for a firearm, the scores are calculated from the cartridge cases from the remaining 19 firearms. For each firearm 25 test shots are fired and these cartridge cases are all compared with each other, this leads to 300 similarity scores per firearm. Thus the same source distribution in total consists of 19 x 300 = 5700 similarity scores. The same is done for each firearm. Fig. 5A illustrates how the CS same source distributions are created.

Equivalently, for the different source distributions the cartridge cases from a firearm are not included in its own distribution. The distributions consist of similarity scores between the cartridge cases of the other 19 firearms. Between the cartridge cases of the 19 firearms it is possible to calculate 106.875 similarity scores, however that is larger than necessary to build a distribution and it would take too much time to build such a distribution. Therefore 5400 similarity scores were calculated for each different source distribution, with the scores equally distributed between the firearms. In this way a CS different source distribution is created for all firearms. Fig. 5B illustrates how the CS different source distributions are created.

(11)

11

Figure 5: The build-up of same source (A) and different source (B) distributions for the Common Source (CS) approach. In this example the distributions are created for firearm 1, the process is the exactly the same for the other 19 firearms.

A: The creation of the CS same source distribution for firearm 1. The cartridge cases of the other 19 firearms are compared with the cartridge cases fired with the same firearm.

B: The creation of the CS different source distribution for firearm 1. Firearm 1 is not included in the distribution. The cartridge cases of the other 19 firearms are compared with cartridge cases fired with different firearms.

Figure adapted from Van Gilse (2018).

2.3.2 Specific Source (SS)

For each of the 20 firearms same source and different source distributions are created. To build the same source distribution for a firearm, the scores are calculated only with the cartridge cases fired from that specific firearm. For each firearm 25 test shots are fired and these cartridge cases are all compared with each other, this leads to 300 similarity scores per firearm. Thus the same source distribution of one firearm for the SS approach consists of 300 similarity scores, the same is done for each firearm. Fig. 6A illustrates how the SS same source distributions are created.

Correspondingly, a SS different source distribution is created for each firearm. The distributions consists of similarity scores between the cartridge cases of the specific firearm with cartridge cases of the other 19 firearms. It is possible to determine a total of 11875 similarity scores for such a distribution, however that is larger than necessary to build a distribution and would take too much time as well. Therefore 600 similarity scores were calculated for each different source distribution, with the scores equally distributed between the firearms. In this way a SS different source distribution is created for all firearms. Fig. 6B illustrates how the SS different source distributions are created.

(12)

12

Figure 6: The build-up of same source (A) and different source (B) distributions for the Specific Source (SS) approach. In this example the distributions are created for firearm 1, the process is the exactly the same for the other 19 firearms. A: The creation of the SS same source distribution for firearm 1. The cartridge cases of firearm 1 are compared with each other.

B: The creation of the SS different source distribution for firearm 1. The cartridge cases of firearm 1 are compared to the cartridge cases of the other 19 firearms.

Figure adapted from Van Gilse (2018).

2.3.3 Overview Common Source and Specific Source distributions

In total there are 6000 similarity scores for same source distributions and 6000 similarity scores for different source distributions. For each firearm there are four distributions: a CS same source distribution, a SS same source distribution, a CS different source distribution and a SS different source distribution. Table 3 gives an overview of the amount of similarity scores that are in each distribution for one firearm.

Table 3: The number of similarity scores for each type of distribution. All four types of distributions are created for all 20 firearms.

Same source distribution Different Source distribution

Common Source Specific Source Common Source Specific Source

5700 similarity scores 300 similarity scores 5400 similarity scores 600 similarity scores

2.4 Comparison of the CS and SS approach

To determine whether there is a significant difference between the CS and SS approach a statistical test is required to compare the distributions. The Wilcoxon rank-sum test is a nonparametric test that can establish if two distributions are significantly different from each other (Fay & Proschan, 2010). The CS same source distributions are compared to the SS same source distributions and the CS different source distributions are compared to the SS different source distributions for each firearm. The Wilcoxon rank-rum test calculates a p-value when it compares two distributions. When the p-value is below the significance level α, the two distributions are significantly different. When the p-value is above the significance level however, the two distributions are not significantly different. In this study α is set at 0,05.

(13)

13

3. Results

3.1 Firing pin impressions

3.1.1 All similarity scores

All data from the same source distributions is compared to all the data from the different source distributions for the firing pin impressions; both distributions consists of 6000 similarity scores. Fig. 7 shows the distributions with the ACCF% as the similarity score, Fig. 8 with the number of CMCs as similarity score and Fig. 9 with the CMC fraction as the similarity score.

Figure 7: Same source versus different source distributions for all firing pin impression data with ACCF% as the similarity score. Both distributions consist of 6000 similarity scores equally distributed over the 20 firearms. The similarity scores are grouped with a bin size of 5%.

Figure 8: Same source versus different source distributions for all firing pin impression data with the number of CMCs as the similarity score. Both distributions consist of 6000 similarity scores equally distributed over the 20 firearms.

0 10 20 30 40 50 0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 Rel at iv e fre q u en cy % ACCF%

Firing Pin Impression

Same Source vs Different Source

ACCF%

Same Source Different Source

0 5 10 15 20 25 30 35 40 45 0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50 Re lat iv e Fre q u en cy % Number of CMCs

Firing Pin Impression

Same Source vs Different Source

CMC

(14)

14

Figure 9: Same source versus different source distributions for all firing pin impression data with the CMC fraction as the similarity score. Both distributions consist of 6000 similarity scores equally distributed over the 20 firearms. The similarity scores are grouped with a bin size of 0,05.

0 10 20 30 40 50 60 70 0 0,05 0,1 0,15 0,2 0,25 0,3 0,35 0,4 0,45 0,5 0,55 0,6 0,65 0,7 0,75 0,8 0,85 0,9 0,95 1 Re lat iv e fre q u en cy % CMC fraction

Firing Pin Impression

Same Source vs Different Source

CMC fraction

(15)

15

3.1.2 Specific Source (SS) vs Common Source (CS)

For each firearm the same source and different source distributions with the CS and SS approach are plotted against each other. The distributions for firearm 1 (F1) with the ACCF% as similarity score are shown in Fig. 10, with the number of CMCs as similarity score in Fig. 11 and with the CMC fraction as similarity score in Fig. 12.

Figure 10: Common Source (CS) distributions and Specific Source (SS) distributions for firearm 1 (F1) plotted against each with the ACCF% as the similarity score. The relative frequency of the similarity scores is on the vertical axis with the scale for the different source distributions on the left side and the scale for the same source distributions on the right side. The similarity scores are grouped with a bin size of 5%.

Figure 11: Common Source (CS) distributions and Specific Source (SS) distributions for firearm 1 (F1) plotted against each with the number of CMCs as the similarity score. The relative frequency of the similarity scores is on the vertical axis with the scale for the different source distributions on the left side and the scale for the same source distributions on the right side.

0 5 10 15 20 0 10 20 30 40 50 0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 Same Sou rce Re lat iv e fre q u en cy % Dif fe re n t Sou rce Re lat iv e fre q u en cy % ACCF %

Firing Pin Impression F1

Specific Source vs Common Source

ACCF%

SS different source CS different source SS same source CS same source

0 1 2 3 4 5 6 7 0 5 10 15 20 25 30 35 40 45 0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 Same Sou rce Re lat iv e fre q u en cy % Dif fe re n t Sou rce Re lat iv e fre q u en cy % Number of CMCs

Firing Pin Impression F1

Specific Source vs Common Source

CMC

(16)

16

Figure 12: Common Source (CS) distributions and Specific Source (SS) distributions for firearm 1 (F1) plotted against each with the CMC fraction as the similarity score. The relative frequency of the similarity scores is on the vertical axis with the scale for the different source distributions on the left side and the scale for the same source distributions on the right side. The similarity scores are grouped with a bin size of 0,05.

Similar graphs with the CS and SS same source and different source distributions for each firearm can be seen in Appendix B.

For all three methods of similarity scores the same source and different source distributions with the CS and SS approach are statistically compared using the Wilcoxon rank-sum test for all 20 firearms. Table 4 shows the p-value of each comparison and whether that means if the SS and CS approach lead to significant differences in the distributions.

0 2 4 6 8 10 12 14 16 0 10 20 30 40 50 60 70 0 0,05 0,1 0,15 0,2 0,25 0,3 0,35 0,4 0,45 0,5 0,55 0,6 0,65 0,7 0,75 0,8 0,85 0,9 0,95 1 Same Sou rce Re lat iv e Fre q u en cy % Dif fe re n t So u rce Re lat iv e fre q u en cy % CMC Fraction

Firing Pin Impression F1

Specific Source vs Common Source

CMC Fraction

(17)

17

Table 4: Testing if there is a significant difference between the CS and SS distributions of the firing pin impressions using the Wilcoxon rank-sum test with the significance level α = 0,05.The p-value is given in the table, if the p-value is below the significance level the distributions are different (red), otherwise they are the same (black). The test is done for all 20 firearms with the ACCF%, the number of CMCs and the CMC fraction as similarity scores, for the same source and different source distributions.

Firearm ACCF % Number of CMC’s CMC fraction

Same source distribution Different source distribution Same source distribution Different source distribution Same source distribution Different source distribution 1 Different (7,9 e-12) Same (0,6114) Same (0,1419) Same (0,1875) Same (0,1573) Same (0,5381) 2 Different (7,8 e-13) Same (0,9504) Different (2,5 e-4) Same (0,3591) Different (0,0173) Different (1,0 e-4) 3 Different (0,0311) Different (1,2 e-6) Different (2,2 e-37) Different (1,7e-7) Different (4,8 e-23) Different (2,9 e-4) 4 Different (2,0 e-29) Different (1,2 e-13) Different (0,0036) Same (0,6745) Same (0,1423) Different (0,0015) 5 Different (3,0 e-4) Same (0,6392) Different (0,0011) Different (0,0139) Same (0,2566) Different (0,0058) 6 Different (1,2 e-4) Same (0,4693) Different (9,0 e-31) Same (0,7329) Different (2,7 e-41) Same (0,0901) 7 Different (2,7 e-21) Same (0,5142) Different (6,3 e-21) Different (1,3e-11) Different (1,1 e-16) Different (0,0087) 8 Different (5,1 e-35) Different (1,6 e-14) Different (0,0284) Different (0,0305) Different (0,0049) Different (0,0337) 9 Different (1,4 e-74) Different (1,1 e-55) Different (6,6 e-62) Same (0,6809) Different (8,9 e-23) Different (0,0229) 10 Same (0,1592) Different (0,0158) Different (1,4 e-23) Different (0,0034) Different (1,3 e-9) Same (0,094) 11 Different (3,3 e-18) Different (1,3 e-28) Different (9,6 e-42) Different (0,0136) Different (1,2 e-5) Same (0,0752) 12 Different (3,5 e-4) Different (0,0495) Same (0,7445) Different (0,0173) Same (0,1276) Different (0,0425) 13 Different (0,0023) Different (4,5 e-6) Different (1,7 e-20) Same (0,1568) Different (1,8 e-24) Different (0,0087) 14 Different (0,0202) Same (0,3505) Different (4,4 e-6) Same (0,3944) Same (0,5921) Same (0,8895) 15 Different (7,0 e-55) Different (0,0354) Different (7,2 e-71) Same (0,6934) Different (4,6 e-61) Same (0,9942) 16 Different (0,0354) Different (0,0094) Different (2,8 e-46) Same (0,7245) Different (6,8 e-37) Same (0,2437) 17 Different (2,2 e-9) Same (0,1541) Different (0,0050) Same (0,3354) Different (1,3 e-4) Same (0,2815) 18 Different (7,7 e-10) Same (0,0710) Different (1,3 e-12) Same (0,0542) Different (2,9 e-17) Same (0,1333) 19 Same (0,9754) Different (2,1 e-5) Different (5,8 e-43) Same (0,0617) Different (1,5 e-33) Different (0,0460) 20 Different (1,2 e-7) Different (0,0213) Different (4,9 e-13) Same (0,4944) Different (3,2 e-20) Same (0,7548)

(18)

18

3.2 Breech face impressions

3.2.1 All similarity scores

All the data from the same source distributions is compared to all the data from the different source distributions for the breech face impressions; both distributions consists of 6000 similarity scores. Fig. 13 shows the distributions with the ACCF% as the similarity score, Fig. 14 with the number of CMCs as the similarity score and Fig. 15 with the CMC fraction as the similarity score.

Figure 13: Same source vs different source distributions for all breech face impression data with ACCF% as the similarity score. Both distributions consist of 6000 similarity scores equally distributed over the 20 firearms. The similarity scores are grouped with a bin size of 5%.

Figure 14: Same source vs different source distributions for all breech face impression data with the number of CMC’s as the similarity score. Both distributions consist of 6000 similarity scores equally distributed over the 20 firearms. The relative frequency of the similarity scores is on the vertical axis with the scale for the different source distributions on the left side and the scale for the same source distributions on the right side.

0 10 20 30 40 50 60 70 80 0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 Re lat iv e fre q u en cy % ACCF%

Breech Face Impression

Same Source vs Different Source

ACCF%

Same Source Different Source

0 1 2 3 4 5 6 7 0 10 20 30 40 50 60 70 80 0 1 2 3 4 5 6 7 8 9 101112131415161718192021222324252627282930313233 Same Sou rce Re lat iv e fre q u en cy % Dif fe re n t Sou rce Re lat iv e fre q u en cy % Number of CMCs

Breech Face Impression

Same Source vs Different Source

CMC

(19)

19

Figure 15: Same source vs different source distributions for all breech face impression data with the CMC fraction as the similarity score. Both distributions consist of 6000 similarity scores equally distributed over the 20 firearms. The relative frequency of the similarity scores is on the vertical axis with the scale for the different source distributions on the left side and the scale for the same source distributions on the right side. The similarity scores are grouped with a bin size of 0,05.

0 1 2 3 4 5 6 7 8 0 10 20 30 40 50 60 70 80 0 0,05 0,1 0,15 0,2 0,25 0,3 0,35 0,4 0,45 0,5 0,55 0,6 0,65 0,7 0,75 0,8 0,85 0,9 0,95 1 Sam e Sou rce Re lat iv e fre q u en cy % Dif fe re n t Sou rce Re lat iv e fre q u en cy % CMC Fraction

Breech Face Impression

Same Source vs Different Source

CMC Fraction

(20)

20

3.2.2 Specific Source (SS) vs Common Source (CS)

For each firearm the same source and different source distributions with the CS and SS approach are plotted against each other. The distributions for firearm 1 (F1) with the ACCF% as similarity score are shown in Fig. 16, with the number of CMCs as similarity score in Fig. 17 and with the CMC fraction as similarity score in Fig. 18.

Figure 16: Common Source (CS) distributions and Specific Source (SS) distributions for firearm 1 (F1) plotted against each with the ACCF% as the similarity score. The relative frequency of the similarity scores is on the vertical axis with the scale for the different source distributions on the left side and the scale for the same source distributions on the right side. The similarity scores are grouped with a bin size of 5%.

Figure 17: Common Source (CS) distributions and Specific Source (SS) distributions for firearm 1 (F1) plotted against each with the number of CMCs as the similarity score. The relative frequency of the similarity scores is on the vertical axis with the scale for the different source distributions on the left side and the scale for the same source distributions on the right side.

0 5 10 15 20 25 30 35 0 10 20 30 40 50 60 70 80 0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 Same s o u rce Re lat iv e fre q u en cy % Dif fe re n t Sou rce Re lat iv e fre q u en cy % ACCF%

Breech Face Impression F1

Specific Source vs Common Source

ACCF%

SS different source CS different source SS same source CS same source

0 2 4 6 8 10 12 14 0 10 20 30 40 50 60 70 80 0 1 2 3 4 5 6 7 8 9 101112131415161718192021222324252627282930313233 Same Sou rce Re lat iv e fre q u en cy % Dif fe re n t Sou rce Re lat iv e fre q u en cy % Number of CMCs

Breech Face Impression F1

Specific Source vs Common Source

CMC

(21)

21

Figure 18: Common Source (CS) distributions and Specific Source (SS) distributions for firearm 1 (F1) plotted against each with the CMC fraction as the similarity score. The relative frequency of the similarity scores is on the vertical axis with the scale for the different source distributions on the left side and the scale for the same source distributions on the right side. The similarity score are grouped with a bin size of 0,05.

Similar graphs with the CS and SS same source and different source distributions for each firearm can be seen in Appendix B.

For all three methods of similarity scores the same source and different source distributions with the CS and SS approach are statistically compared using the Wilcoxon rank-sum test for all 20 firearms. Table 5 shows the p-value of each comparison and whether that means if the SS and CS approach lead to significantly distributions. 0 2 4 6 8 10 12 14 16 18 0 10 20 30 40 50 60 70 80 0 0,05 0,1 0,15 0,2 0,25 0,3 0,35 0,4 0,45 0,5 0,55 0,6 0,65 0,7 0,75 0,8 0,85 0,9 0,95 1 Same Sou rce Re lat iv e fre q u en cy % Dif fe re n t Sou rce Re lat iv e fre q u en cy % CMC Fraction

Breech Face Impression F1

Specific Source vs Common Source

CMC fraction

(22)

22

Table 5: Testing if there is a significant difference between the CS and SS distributions of the breech face impressions using the Wilcoxon rank-sum test with the significance level α = 0,05. If the p-value is below the significance level the distributions are different (red), otherwise they are the same (black). The test is done for all 20 firearms with the ACCF%, the number of CMC’s and the CMC fraction as similarity scores, for the same source and different source distributions.

Firearm ACCF % Number of CMC’s CMC fraction

Same source distribution Different source distribution Same source distribution Different source distribution Same source distribution Different source distribution 1 Different (2,5 e-63) Different (0,0219) Different (3,09 e-21) Same (0,3339) Different (2,0 e-24) Same (0,5739) 2 Different (6,1 e-31) Different (0,0023) Different (2,6 e-7) Same (0,0543) Same (0,3519) Same (0,0541) 3 Different (5,0 e-11) Same (0,9726) Different (1,59 e-31) Same (0,4920) Different (2,61 e-25) Same (0,6497) 4 Different (3,7 e-58) Different (3,8 e-52) Different (3,6 e-18) Different (0,0304) Different (1,03 e-14) Same (0,4605) 5 Different (1,5 e-25) Different (2,2 e-13) Different (7,0 e-23) Different (0,0016) Different (3,1 e-31) Same (0,3243) 6 Same (0,2236) Different (1,2 e-12) Different (1,4 e-54) Different (0,0055) Different (2,2 e-69) Different (0,0139) 7 Different (2,6 e-11) Different (7,7 e-5) Different (1,7 e-11) Same (0,1450) Different (3,4 e-20) Same (0,7048) 8 Same (0,8408) Same (0,0721) Different (4,4 e-101) Same (0,0775) Different (1,8 e-94) Same (0,5582) 9 Different (6,2 e-16) Different (0,0021) Different (1,6 e-79) Same (0,1935) Different (7,0 e-82) Same (0,6683) 10 Different (1,3 e-106) Different (7,0 e-79) Same (0,8329) Same (0,9771) Different (7,0 e-7) Same (0.1582) 11 Same (0,7588) Same (0,9504) Different (1,5 e-47) Same (0,7163) Different (1,2 e-55) Same (0,1725) 12 Different (1,9 e-43) Different (5,0 e-69) Different (5,0 e-14) Different (1,0e-5) Different (6,6 e-30) Different (0,0120) 13 Different (1,6 e-19) Different (7,0 e-18) Different (5,9 e-97) Same (0,3168) Different (4,0 e-98) Same (0,2596) 14 Different (4,2 e-5) Different (9,6 e-17) Different (7,4 e-34) Same (0,7266) Different (4,1 e-33) Same (0,9782) 15 Different (1,2 e-125) Different (5,3 e-32) Different (8,1 e-96) Same (0,1427) Different (7,9 e-93) Different (0,0343) 16 Different (2,2 e-6) Same (0,30320 Different (5,5 e-25) Same (0,8786) Different (4,7 e-8) Same (0,5458) 17 Different (1,1 e-37) Different (0,0126) Different (1,5 e-70) Same (0,0752) Different (1,0 e-73) Same (0,5441) 18 Same (0,8876) Different (0,0069) Different (3,0 e-8) Same (0,2977) Different (1,7 e-15) Same (0,7519) 19 Different (5,9 e-5) Different (3,0 e-9) Different (3,5 e-22) Same (0,1421) Different (4,0 e-16) Different (0,403) 20 Different (7,2 e-16) Different (2,1 e-10) Different (9,2 e-52) Same (0,0621) Different (4,9 e-67) Same (0,2385)

(23)

23

4. Discussion

Fig. 7, Fig. 8 and Fig. 9 show that the same source and different source distributions of firing pin impressions on Glock cartridge cases are very distinct when the ACCF%, number of CMCs or the CMC fraction is used as the similarity score. Similarly Fig. 13, Fig. 14 and Fig. 15 show that the same source and different source distributions of breech face impressions on Glock cartridge cases are very distinct when the ACCF%, number of CMCs or the CMC fraction is used as the similarity score. This shows that firing pin impressions and breech face impressions on Glock cartridge cases can be used as evidence in forensic investigations. The method used in this study has the potential to evaluate this evidence. Previous studies have already shown that automated methods can be suited to evaluate aperture shear marks on Glock cartridge cases as well (Brand, 2017; Van Gilse, 2018). Since the marks created by different parts of the firearm are considered to be independent, the strength of evidence of the three marks can be combined (Morris et al., 2016).

For some of the firearms the outcome of the Wilcoxon rank-sum test showed that the distributions of the CS and SS approach are significantly different, whilst the distributions appear to be very similar in the graphs in Appendix B. The Wilcoxon rank-sum test compares the values of all similarity scores of the distributions, whereas in the graphs the similarity scores are grouped with a certain bin size. If the bin size of these distributions is made smaller, it becomes apparent that there indeed is a difference between the distributions. An example can be seen in Fig. 19: the distributions of the CS and SS approach look very similar look very similar in Fig. 19A, but the Wilcoxon rank-sum test gives a p-value (0,0021) that indicates that the distributions are significantly different. The bin size is reduced from 5% to 1% in Fig. 19B and the difference between the distributions can be observed.

Figure 19: The different source distributions with the Specific Source and Common Source approach of the breech face impression for firearm 9 (F9) with the ACCF% as similarity score. In graph A the bin size is 5% and in graph B the bin size is 1%.

Table 4 shows the outcome of the Wilcoxon rank-sum test between the distributions of the SS and CS approach for firing pin impressions for each firearm. Table 6 below gives a quick overview for how many firearms the same source and different source distributions are significantly different for using all three similarity score systems for firing pin impressions.

Table 6: Overview of the results of the Wilcoxon rank-sum test between the Specific Source (SS) and Common Source (CS) distributions for the firing pin impressions. The same source and different source distributions are compared with the ACCF%, number of CMCs and the CMC fraction as similarity score. The table shows for how many of the 20 firearms the SS and CS approach lead to significantly different distributions.

ACCF% Number of CMCs CMC fraction

Same Source distribution Different Source distribution Same Source distribution Different Source distribution Same Source distribution Different Source distribution 18 / 20 12 / 20 18 / 20 7 / 20 13 / 20 10 / 20

(24)

24 The outcome of the test differs between the similarity score systems, but the results show that for at least 13 of the 20 firearms the same source distributions of the SS and CS approach are significantly different for the firing pin impressions. Likewise for at least 7 of the 20 firearms the different source distributions of the SS and CS approach are significantly different for the firing pin impressions. The fact that for such a large portion of the firearms the SS and CS approach leads to significantly different distributions, makes it unjustified to use the CS approach when the situation theoretically requires a SS approach for firing pin impressions on Glock cartridge cases. A significantly different set of distributions could namely change the strength of evidence, which is highly unwanted.

Table 5 shows the outcome of the Wilcoxon rank-sum test between the distributions of the SS and CS approach for breech face impressions for each firearm. Table 7 below gives a quick overview for how many firearms the same source and different source distributions are significantly different for using all three similarity score systems for breech face impressions.

Table 7: Overview of the results of the Wilcoxon rank-sum test between the Specific Source (SS) and Common Source (CS) distributions for the firing pin impressions. The same source and different source distributions are compared with the ACCF%, number of CMCs and the CMC fraction as similarity score. The table shows for how many of the 20 firearms the SS and CS approach lead to significantly different distributions.

ACCF% Number of CMCs CMC fraction

Same Source distribution Different Source distribution Same Source distribution Different Source distribution Same Source distribution Different Source distribution 16 / 20 16 / 20 19 / 20 4 / 20 19 / 20 4 / 20

The outcome of the test differs between the similarity score systems, but the results show that for at least 16 of the 20 firearms the same source distributions of the SS and CS approach are significantly different for breech face impressions. Likewise for at least 4 of the 20 firearms the different source distributions of the SS and CS approach are significantly different for the firing pin impressions. The fact that for a substantial portion of the firearms the SS and CS approach leads to significantly different distributions, makes it unjustified to use the CS approach when the situation theoretically requires a SS approach for breech face impressions on Glock cartridge cases. A significantly different set of distributions could namely change the strength of evidence, which is highly unwanted.

5. Conclusion

On paper the SS approach is used for specific source forensic investigations and the CS approach for common source forensic investigations. Creating distributions with the SS approach is a lot more time consuming, therefore it is of interest to know whether a CS approach could be used for specific source situations as well. In this study same source and different source distributions were created for firing pin impressions and breech face impressions on Glock cartridge cases using both the SS and the CS approach. These distributions were compared in order to answer the following set of research questions:

‘Is there a significant difference between Common Source and Specific Source distributions for firing pin impressions on Glock cartridge cases?’

‘Is there a significant difference between Common Source and Specific Source distributions for breech face impressions on Glock cartridge cases?’

(25)

25 The results showed that for a substantial part of the firearms there is a significant difference between the SS and CS distributions. Significantly different distributions can in turn lead to a significant difference in the strength of the evidence. A significant change in the strength of the evidence from the theoretically correct value is highly unwanted. Therefore it can be concluded that it is unjustified to use the CS approach for situations that theoretically require the SS approach for the comparison of firing pin impressions and breech face impressions on Glock cartridge cases.

Acknowledgements

The author would like to thank Martin Baiker-Sørensen for his supervision during the entire project, for his feedback on the report and for always being available to answer questions. Erwin Mattijsen is thanked for his role as the examiner in the Master Forensic Science. Furthermore the author would like to thank the Weapons and Tools team of the NFI for making it a nice team to be a part of.

(26)

26

References

Baiker, M., Keereweer, I., Pieterman, R., Vermeij, E., van der Weerd, J., & Zoon, P. (2014).

Quantitative comparison of striated toolmarks. Forensic Science International, 242, 186–199. https://doi.org/10.1016/j.forsciint.2014.06.038

Brand, N. W. (2017). Bewijskracht en correlatie van Glock slagpingatschaafsporen Bewijskracht en

correlatie van Glock slagpingatschaafsporen Studie naar de correlatie tussen likelihood.

Community, S., Science, C., Affairs, G., Applied, C., Statistics, T., Engineering, D., & Sciences, P. (2009). Strengthening forensic science in the United States: A path forward. In Strengthening Forensic

Science in the United States: A Path Forward. National Academies Press.

https://doi.org/10.17226/12589

Fay, M. P., & Proschan, M. A. (2010). Wilcoxon-Mann-whitney or t-test? Statistics Survey, 4, 1–37. https://doi.org/10.1214/09-SS051.Wilcoxon-Mann-Whitney

Gilse, I. R. Van, & Science, M. F. (2018). Specific source vs . Common source reference scores to

determine the evidential strength of Glock aperture shear marks.

Hamby, J. E., Norris, S., & Petraco, N. D. K. (2016). Evaluation of GLOCK 9 mm Firing Pin Aperture Shear Mark Individuality Based On 1,632 Different Pistols by Traditional Pattern Matching and IBIS Pattern Recognition. Journal of Forensic Sciences, 61(1), 170–176.

https://doi.org/10.1111/1556-4029.12940

Jackson, A. R. W., & Jackson, J. M. (2017). No Title. In Forensic Science (Fourth Edi, pp. 296–323). Pearson.

Morris, K. B., Law, E. F., Jefferys, R. L., & Dearth, E. C. (2016). Interpretation of Cartridge Case

Evidence Using IBIS and Bayesian Networks. 260.

Ommen, D. M., & Saunders, C. P. (2018). Building a unified statistical framework for the forensic identification of source problems. Law, Probability and Risk, 17(2), 179–197.

https://doi.org/10.1093/lpr/mgy008

PCAST. (2016). Report to the President: Forensic Science in Criminal Courts: Ensuring Scientific Validity of Feature-Comparison Methods. In Executive Office of the President President’s Council

of Advisors on Science and Technology (Issue September).

Riva, F., & Champod, C. (2014). Automatic comparison and evaluation of impressions left by a firearm on fired cartridge cases. Journal of Forensic Sciences, 59(3), 637–647.

https://doi.org/10.1111/1556-4029.12382

Riva, F., Hermsen, R., Mattijssen, E., Pieper, P., & Champod, C. (2017). Objective Evaluation of Subclass Characteristics on Breech Face Marks. Journal of Forensic Sciences, 62(2), 417–422. https://doi.org/10.1111/1556-4029.13274

Roth, J., Carriveau, A., Liu, X., & Jain, A. K. (2015). Learning-based ballistic breech face impression image matching. 2015 IEEE 7th International Conference on Biometrics Theory, Applications and

Systems, BTAS 2015, Btas. https://doi.org/10.1109/BTAS.2015.7358774

Song, J. (2015). Proposed “congruent matching cells (CMC)” method for ballistic identification and error rate estimation. AFTE Journal, 47(3), 177–185.

Tai, X. H., & Eddy, W. F. (2018). A Fully Automatic Method for Comparing Cartridge Case Images.

Journal of Forensic Sciences, 63(2), 440–448. https://doi.org/10.1111/1556-4029.13577

(27)

27 Foreman, C., & Ballou, S. (2007). Surface Topography Analysis for a Feasibility Assessment of a National Ballistics Imaging Database. Nistir 7362.

(28)

28

Appendix A – Glock pistols database

Table A1: Glock pistols used for test fires to create the CS and SS distributions

Number Glock pistol

serial / case number

Model Calibre

1 13.10.07.133_NFI 26 9mm Luger

2 13.11.11.180_NFI 26 9mm Luger

3 15.06.09.047_NFI 19 9mm Luger

4 15.11.30.171_NFI 19 9mm Luger

5 16.04.18.156_NFI 19 Gen4 9mm Luger

6 17.01.18.187_NFI 26 9mm Luger 7 17.12.01.039_NFI 26 9mm Luger 8 17.012.01.002_NFI 26 9mm Luger 9 3206_NFI 17 9mm Luger 10 3210_NFI 19 9mm Luger 11 3255_NFI 17 9mm Luger 12 3256_NFI 17 9mm Luger 13 3374_NFI 19 9mm Luger 14 3381_NFI 19C 9mm Luger 15 3383_NFI 19 9mm Luger 16 3410_NFI 19 9mm Luger

17 3424_NFI 26 Gen4 9mm Luger

18 3444_NFI 26 9mm Luger

19 3445_NFI 26 9mm Luger

(29)

29

Appendix B – SS and CS distributions for each firearms

Figure B1: Common Source (CS) distributions and Specific Source (SS) distributions for firearm 1 (F1) plotted against each other. The relative frequency of the distribution scores is on the vertical axis with the scale for the different source distributions on the left side and the scale for the same source distributions on the right side.

A: Distributions of firing pin impressions with ACCF% as similarity score, grouped with bin size of 5%. B: Distributions of firing pin impressions with the number of CMCs as similarity score.

C: Distributions of firing pin impressions with the CMC fraction as similarity score, grouped with bin size of 0,05. D: Distributions of breech face impressions with ACCF% as similarity score, grouped with bin size of 5%. E: Distributions of breech face impressions with the number of CMCs as similarity score.

(30)

30

(31)

31

(32)

32

(33)

33

(34)

34

Figure B620: Common Source (CS) distributions and Specific Source (SS) distributions for firearm 6 (F6) plotted against each other. The relative frequency of the distribution scores is on the vertical axis with the scale for the different source

distributions on the left side and the scale for the same source distributions on the right side. A: Distributions of firing pin impressions with ACCF% as similarity score, grouped with bin size of 5%. B: Distributions of firing pin impressions with the number of CMCs as similarity score.

(35)

35

(36)

36

(37)

37

(38)

38

(39)

39

(40)

40

(41)

41

(42)

42

(43)

43

Specific Source versus Common Source approach to determine the evidential strength of automated comparisons of firing pin and breech face impressions on Glock cartridge cases