University of Amsterdam Masters of Forensic Science
Title: The resulting psychological impacts and influences from introducing augmented reality
(AR) and virtual reality (VR) in forensic reporting within legal settings
Maura Cook 12565431 Word Count 7283
Supervisor: Erwin Mattijssen MSc Examiner: Dr. Marjan Sjerps 29 January 2021
1 ABSTRACT ... 1
2 INTRODUCTION ... 1
2.1 FORENSIC REPORTING ... 1
2.2 VIRTUAL REALITY (VR) AND AUGMENTED REALITY (AR) ... 2
2.2.1 Background ... 2
2.2.2 Use in Courtroom ... 3
3 MATERIALS AND METHODS ... 4
4 OVERVIEW OF PREVIOUS LITERATURE ... 5
4.1 PSYCHOLOGICAL IMPACTS AND INFLUENCES OF VR AND AR ... 5
4.1.1 Influence of Visual Evidence ... 5
4.1.2 Implicit Biases ... 6
4.1.3 Heightened Emotional Responses ... 7
4.1.4 Body Ownership Illusions (BOIs) ... 8
4.1.5 Virtual Memories Becoming Real Memories ... 9
4.1.6 Perception Inaccuracy ... 9
4.1.7 Psychological Conditions ... 9
4.1.8 Psychophysiological Responses ... 10
5 DISCUSSION ... 11
6 CONCLUSIONS AND RECOMMENDATIONS ... 12
7 REFERENCES ... 14
8 APPENDIX ... 17
1
1 Abstract
The use of immersion virtual environment (IVE) technology, specifically virtual reality (VR) and augmented reality (AR), as visualizations for forensic reporting has been a recent issue of debate within the forensic and legal community. There are many established advantages to using both VR and AR as a means to visualize forensic reports in court. Such advantages include using IVE technology for virtual crime scene reconstructions and crime scene
walkthroughs as well as assisting in explaining technical aspects of forensic expert witnesses reports. However, many have argued that IVE technology may have a prejudicial effect on members of a court when used for forensic reporting in legal proceedings. In this literature research paper, various psychological impacts and influences that IVE technology could have on the court actors in both adversarial and inquisitorial legal systems were identified. These include the influence of: Visual evidence on one’s decision making process; Implicit biases; Heightened emotional responses; Body ownership illusions; Virtual memories becoming real memories; Perception inaccuracy; Psychological conditions; Psychophysiological responses. Recommendations have been made in order to aid in the incorporation of IVE technology into a courtroom while minimizing the psychological impacts and influences of using IVE technology in forensic reporting.
2 Introduction
2.1 Forensic Reporting
Upon the completion of a forensic investigation, a forensic report is produced to the court. The forensic report details the tasks conducted during the forensic investigation, the results produced from the forensic investigation, the opinion of the specialized forensic investigator, and a summary of the conclusions of the forensic investigation, as well as various other aspects (1). Whether the courtroom is in an adversarial legal system or in an inquisitorial legal system, the readers of these reports are not trained in the field of forensic science, which may make the interpretation of the forensic reports challenging. The results and other aspects included within the report can be visualized in order to aid readers in the understanding of these reports. Therefore, it is important for the visualizations to be an easily understood, factual representation of the report. In some cases, these visualizations may be entered into the court as pieces of legal evidence; While in other cases, these visualizations are simply used by the court actors to aid in their comprehension of the report.
With technology rapidly progressing in today’s society, the types of visualizations available for use have increased. Advancements in photography, videography, computer science, and user-interface systems have been utilized to produce high quality visualizations for use in forensic reporting. With each new visualization type which becomes available for use in forensic reporting, it is important to examine the possible psychological impacts and influences of the visualization that may affect the members of the court.
For example, when major advancements in photography allowed for the production of high-quality images, photographs of the crime scenes, of the evidence, and of autopsies were attached to forensic reports to aid readers in the understanding of the report. The use of these photographs in the forensic reports raised questions as to if they created biases or influenced the court actors in some way. Studies have been conducted to analyse these possible
influences on the court actors. One study conducted by Grady, et. al. examined the influence of “gruesome photographic evidence on legal decisions” (2). The results of this study found that the use of gruesome photographic evidence did increase the number of guilty verdicts made by mock jury members. Therefore, photographs in forensic reporting can have
2 psychological influences on court actors. In order to minimize these influences, safeguard measures have been implemented within the legal systems. For example, within the U.S. adversarial legal system, under the Federal Rules of evidence 403, photographic evidence is admissible in court as long as “their probative value substantially outweighs their prejudicial impact” (2). However, in regard to IVE technology, it would be difficult to determine the admissibility of IVE technology as evidence or as a visual aid in a courtroom due to its widely undefined “prejudicial impact.”
2.2 Virtual Reality (VR) and Augmented Reality (AR)
2.2.1 Background
With recent technological advancements, new methods of visualizations for forensic
reporting have become available. Two methods of particular importance are: Virtual Reality (VR) and Augmented Reality (AR).
These two methods work by creating a virtual environment (VE) or an immersive virtual environment (IVE) for users to interact with. Virtual environments can be defined as
“synthetic sensory information” which leads the user to perceive the synthetic environment as if it were real (3). A person is able to interact with a VE through any “perceptual channel”, such as head mounted displays, earphones, nosepieces, gloves with mechanical feedbacks, etc. (3). Immersive virtual environments are similar to VEs except IVEs “perceptually surrounds the users” (3). Two key characteristics of IVEs are the users’ position, head, eyes, and facial expressions are “unobtrusively tracked” while interacting with the IVE and the sensory information coming from the real world is greatly minimized (3). A fundamental difference between a VE and an IVE is that in an IVE, the sensory information from the real world is substantially less “psychologically prominent” than the sensory information created from the virtual environment (3). In other words, an IVE feels much more realistic to the user than just a VE. Both virtual reality technology and augmented reality technology are used to create IVEs. In general, virtual reality and augmented reality can be referred to IVE
technology.
Virtual Reality is a type of technology which is capable of mimicking the real world “with a high degree of realism and immersion” (4). The present-day idea of VR was first presented over fifty years ago in 1965 as the “Ultimate Display” by Ivan Sutherland. Although, five years previously in 1960, the first virtual reality head mounted display was created, making this type of technology not as ‘new’ as most people believe it is (5). However, it was not until approximately 2016, after society’s vast technological advancements, that VR technology was widely available to the general public (6). The VR technology available now-a-days offers a deep immersive experience for the users’ bodies by being deeply intertwined with the users’ senses and is capable of shaping the users’ behaviours (7). When using today’s VR devices, the user becomes immersed in a 3-dimensional, realistic virtual environment in which the user is able to navigate and interact with (7). Due to this deep immersive
experience, VR technology allows its users to “actively engage” with the IVE which, in turn, helps facilitate effective learning by the user “without diminishing cognitive resources” (8). Augmented Reality is a type of technology which adds virtual objects into the real world, making it appear as though the virtual objects exist in the same space as the real world (9). AR systems are able to make use of lighting to create virtual objects, which fit purely in with the real-world environment, making the virtual scene appear more realistic (10). However, it has been shown that it is difficult to establish accurate depth perception using AR systems (10). Nonetheless, with the way AR technology is at the current time, AR systems allow users to behave and respond to the virtual objects as they would normally respond in the real world
3 due to the users experiencing a significant sense of presence (9). It is predicted that in the next few years, AR systems will be advanced to the point where the virtual objects will be “barely distinguishable” from the real objects (9).
With the advancements in technology, VR and AR systems will have higher “accuracy, competency and lack of biases” which will further make these systems ideal for use within a courtroom (11).
2.2.2 Use in Courtroom
There is an on-going global legal debate in regard to the acceptance of VR and AR
technology in a courtroom. Switzerland is one country that has made a definitive ruling on this matter, allowing VR technology into the courtroom as evidence “as long as the evidence is scientifically sound and not otherwise illegal” (12). Russia is another country that has made a definitive ruling regarding IVE technology, allowing VR technology to be used in court to help visualize the situation of a crime scene (13). Other countries who have not made a definitive ruling on this issue, such as the United Kingdom or Australia, have turned to the United States for guidance on whether to accept this type of evidence in court (14). This is due to the United States having a long-standing precedent for the acceptance of various types of visual evidence, including computer/virtual visual evidence, in a courtroom (14). In the United States’ adversarial legal system, admissibility of evidence is determined using the Federal Rules of Evidence as mentioned prior (15). As of 1992, the United States had established a precedent on the acceptance of three-dimensional simulations as evidence in court during the court case Stephenson v. Honda Motors Ltd. Of America (16). Other countries have tended to adopt a similar rule or precedent within their respective legal systems (15).
However, due to few legal systems having a definitive rule specifically for the acceptance of IVE technology in a courtroom, IVE technology has had limited use in the courtroom (17). This is shocking as IVE technology, which encompasses both VR and AR technology, has been shown to be extremely beneficial for various aspect of criminal legal proceedings. For starters, at the investigation aspect of legal proceedings, IVE technology has been proven to be an efficient tool within police and forensic investigations for accurate crime scene reconstruction purposes (18). With the high increase in law enforcement officers donning body worn cameras during their duty shifts and the increase use of 3D laser scanners at crime scenes, a crime scene can easily and efficiently be recreated into an IVE (19). These virtual crime scene reconstructions could then be used in a courtroom for all court actors to view. These reconstructed crime scenes using VR technology can also be used for crime scene walkthroughs. This would allow for investigators, or court actors, to later virtually visit the crime scene again, as well as allow other actors involved the investigation who normally would not have been able to visit the crime scene, to virtually visit the crime scene (12). The use of virtual reconstruction of crime scenes and virtual crime scene walkthroughs in a
courtroom can be particularly helpful for the forensic reporting of major crimes. Research has shown that IVE technology would be beneficial for major crime types, such as homicide, due to these crimes involving a high level of detail and IVE technology’s ability to include a high level of detail within its animations (17). The visualization of these details, of which are typically of importance in forensic investigations, will aid court actors in understanding their importance in forensic reports. Furthermore, these crime scene reconstructions using IVE technology can be utilized for forensic reporting within a courtroom to demonstrate the hypothesized scenarios or order of events which took place at the original time of the crime. This use of the virtual crime scene reconstructions can be beneficial in aiding to explain the technical components of a forensic expert witness’s testimony (19). For example, if a forensic
4 blood pattern analysis expert attempts to explain to court actors how a hypothesized, specific sequence of events had led to the production of the blood spatter patterns found at the crime scene, court actors may have a difficult time following and understanding the expert’s testimony. With the aid of IVE technology, the court actors would be able to visually see the specific sequence of events virtually take place at the recreated crime scene as the forensic expert explained. In addition to crime scene related IVE technology’s use within a courtroom, IVE technology has shown to be useful as an aid in witness or suspect interrogations and in forensic medical evaluation of injuries (12, 18, 19, 20). With IVE technology shown to be such a beneficial tool for crime investigations, it is surprising to see such a lack of evidence in the form of IVE technology in a courtroom today.
However, some have argued that despite the established benefits of using IVE technology in a courtroom, this type of technology offers up a high degree of prejudicial effect on the court actors (19). Despise these claims, there has been limited research conducted to examine the prejudicial effect IVE technology has on court actors. The limited research which is available on this topic has been primarily focused on court actors in the adversarial legal system – specially looking at the psychological effects of IVE on jury members.
With that, the focus of this research paper is to examine the resulting psychological impacts and influences on court actors when using IVE technology, specifically virtual reality and augmented reality, as visualizations in forensic reporting within a courtroom. Specifically, analysing the impacts and influences which virtual reality and augmented reality may have on the court actors in both the adversarial legal system and the inquisitorial legal system. These court actors are specified in Table 1.
Table 1: Court Actors in Two Main Legal Systems
Adversarial Legal System Inquisitorial Legal System Passive, Umpire-Like Judge Active, Magisterial Judge
Prosecuting Lawyers Public Prosecutor
Defence Lawyers Suspect (Defence)
Partisan-Experts Court-Appointed Experts
Jury
The adversarial legal system is employed in countries which include: United States of America, Australia, New Zealand, Canada, India, as well as a few others (21). On the other hand, the inquisitorial legal system is employed in most European countries such as the Netherlands and France (21). Many other countries employ a mixed legal system,
encompassing aspects of both the adversarial legal system and the inquisitorial legal system. The functions of the court actors in either legal system is not important for the purposes of this research question as the function of the court actors would not change the psychological impacts and influence IVE technology in forensic reporting would have on them.
3 Materials and Methods
Various literature was gathered through searches of online literature databases, which
included PubMed and Google Scholar. The literature gathered pertained to multiple different topics: IVE technology in general; Virtual reality and augmented reality technology; IVE technology used in a forensic context; Visual evidence in a courtroom; Psychology in relation to visual evidence in a courtroom; Psychology and psychophysiology in relation to virtual
5 reality and augmented reality technology. This search strategy is detailed in the attached appendix.
A review of the collected literature was conducted, and the information gathered from the literature was combined in order to answer the research question mentioned previously.
4 Overview of Previous Literature
4.1 Psychological Impacts and Influences of VR and AR
The use of virtual reality and augmented reality technology in a courtroom has the potential to have various psychological impacts and influences on the different court actors in both the adversarial and inquisitorial legal system. These psychological impacts and influences are detailed below.
4.1.1 Influence of Visual Evidence
It is important to note that evidence in the form of both virtual reality and augmented reality are classified as types of visual evidence. Visual evidence is influential in and of itself and has long been shown to be an important type of evidence in legal proceedings (14, 23). For the purposes of this paper, visual evidence will be an all-inclusive term referring to
photographic evidence, video evidence, VR evidence, AR evidence, visualizations of evidence (i.e., reconstructions, diagrams, computer generated reconstructions, etc.),
illustrations, or any other forms of evidence that is meant to be primarily visually perceived by other members of the court and not primarily verbalized or read in text. For most court actors, visual evidence has helped make the evidence to be more easily understood and more relevant to court actors (14). Non-visual types of evidence involving complex or technical issues make it difficult for actors of the court to understand and often leaves the court actors frustrated, distracted, or perplexed (14). Visual evidence tends to be more engaging which eliminates these issues for the court actors and helps increase attention to and understanding of the evidence being presented visually. However, this can be problematic in terms of some types of visual evidence because moving pictures on a screen are often easier for one to believe. This is due to the engaging aspect of visual evidence, which can negatively affect the cognition of the viewer, decreasing the ability to critically analyse a piece of evidence (14). This effect is compounded by the fact court actors’ attentiveness to a piece of evidence will be increased whenever the evidence is presented in a manner in which the individual is used to (14). With most individuals now-a-days having had grown up surrounded by digital screens, and in more recent years, surrounded by interactive digital screens, presented
courtroom evidence in a digitally visual manner will most likely increase the attentiveness of a court actor to a piece of particular evidence (14).
The content of visual evidence has also shown to be extremely influential. As mentioned previously, in studies concerning “gruesome” visual evidence, when mock jurors were presented with gruesome visual evidence, the conviction rates were significantly increased when compared to mock jurors who were not shown gruesome visual evidence (2, 22). The number of guilty verdicts, or convictions, in these studies were increased due to both the effect of the gruesome content of the visual evidence as well as the fact that the evidence was supplied in a visual manner (2). One finding that is of particular interest is the inculpatory weight of evidence rated by court actors who were shown gruesome visual evidence to be greater than rated by court actors who were shown non-gruesome visual evidence. In this same study, it was shown that gruesome evidence presented in a verbal manner to court actors did not have any influence on the emotional states of the court actors and did not change the number of convictions (22). This reinforces the fact that not only is the content of
6 the visual evidence vastly influential, but the visual manner in which the evidence is
presented is even more influential. If the content of evidence was as influential as the manner of presentation of the evidence, then the inculpatory weight of evidence rated by court actors who were verbally presented with gruesome evidence should have been the same as the inculpatory weight of evidence rated by court actors who had been visually presented with gruesome evidence.
An additional psychological influence of visual evidence on court actors deals with memory. It has been shown that evidence presented in a visual manner helps increase memory of court actors (14). One study had found that when information is presented in a visual manner, the average person will remember 87% of the information compared to when information is presented orally, the average person will then only remember 10% of the information (14). Another study found similar results: the average person remembered 65% of the information whenever the information was presented in a visual manner versus the average person
remembered 15% when the information was presented orally (14). Within visual presentation of information, it was found that there is an increase in memory when the evidence is
presented in an animated manner (14). In support of this, an additional study found that members of a jury remembered information with a 650% increase whenever the information was presented via computer animations compared to when information is presented in any other form (14). Therefore, as it can be argued that both VR and AR are the most advanced forms of computer animations available at the moment, it can be concluded that the use of VR and AR in a courtroom would result in an increase in memory retention in the members of the court. This could result in the court actors remembering the evidence and information presented through VR and/or AR more vividly and more readily than any other evidence, including other visual evidence.
4.1.2 Implicit Biases
Implicit bias is defined as the day-to-day associations people automatically make at an unconscious level (24). The fact that these biases form at an unconscious level is what makes these biases ‘implicit.’ Some individuals may try to avoid forming implicit biases. However, these biases are “an inevitable byproduct” for any individual who has a functioning cognitive system (8). Typically, implicit biases refer to the stereotypical associations one makes or the attitudes one may feel towards another person or group of people at an unconscious level. There are various types of implicit biases such as confirmation bias, gender bias, racial bias, affinity bias, horns effect, halo effect, etc. (25). For the purposes of this paper, only a few types of implicit biases will be discussed.
Implicit biases are an important psychological influence that must try to be minimized with any sort of evidence in a court of law. Studies have shown that the decision making of both judges and jurors throughout a trial can be greatly influenced by implicit biases (8). The use of VR or AR in a courtroom has been shown to both help minimize implicit biases made by legal actors, as well as cause implicit biases to more greatly influence legal decisions made by the court actors. One study produced results illustrating that VR technology aids in reducing racial implicit biases in court (8). This is due to the VR system allowing the user to view the given situation from another perspective, which in turn, helps to minimize the “otherness factor of outgroup individuals” and minimize implicit biases or stereotypes (8). The study went on to explain that VR technology in court could also help minimize the implicit biases or reactions to individuals of another race or ethnicity in a “standardized, interactive, engaging, and feasible” manner (8). It should be noted that individuals indeed do have the ability to control the influence of implicit biases, however, only under specific conditions. In order to monitor the influence of implicit biases, an individual must have
7 “vigilant self-regulation” which involves a high level of cognition requiring most of one’s mental capabilities (8). In other words, an individual’s ability to monitor the influence of implicit biases is significantly decreased when in situations where the individual must focus on many specific details or facts. This self-regulation does not seem plausible for court actors to maintain while having to view and process a high number of specific details and facts. Therefore, it could be hypothesised that the ability for the various court actors to monitor the influence of implicit biases is greatly diminished throughout legal proceedings. Which brings about the negative aspect of VR and AR technology on implicit biases: VR and AR
technology creates prejudicial impacts on court actors' decision making. Due to the fact that VR technology does not rely on the self-regulation of court actors throughout trial
proceedings, VR systems could potentially have a prejudicial effect on members of a jury (14). Without continuous self-regulation by the court actors, implicit biases can easily influence their decision making process. Additionally, a study has found that visual evidence generated on a computer may have prejudicial effects on members of a jury (14).
4.1.3 Heightened Emotional Responses
In addition to implicit biases, heightened emotional responses elicited by visual evidence psychologically influences the legal actors in a courtroom. Studies have concluded that the viewers of graphical evidence may not be able to assess the possible influences the visuals have had on their decision making process (14). A lot of the visuals used in a courtroom can be full of “emotive” content which can have a prejudicial effect on the court actors. This addition of emotion to visual evidence or visual aids has been referred to as “Disneying-up the evidence” (14).
A study conducted by Bright and Goodman-Delahunty is a great source in demonstrating how visual evidence can elicit high emotional responses from members of a jury in particular and how it can influence the decision making of jury members (22). The results of this study found the conviction rate to be significantly higher whenever the mock jury members were shown visual evidence compared to when the mock jury members were shown no visual evidence (22). This study also found that the mock jury members experienced “significantly more intense emotional responses, including greater anger at the defendant” when shown gruesome photographs compared to mock jurors who were shown no photographs (22). As mentioned previously, a similar, more recent study conducted by Grady, et. al. found results consistent with the previous study: the use of gruesome visual evidence in the form of photographs increased the number of guilty verdicts made by the mock jury members (2). Another similar study conducted by Salerno and Bottoms found evidence which elicit high emotional responses may also cause a decrease in one’s cognitive processing of the evidence (26). From this, it can be concluded that the high emotional responses elicited through viewing visual evidence can have an exceedingly high influential effect on the decision making process of court actors.
More specifically to VR and AR technology, IVEs have been found to appeal to a juror’s “emotional and subconscious processes” due to IVEs creating a sense of ‘presence’ for the users (19, 27). This sense of presence has been shown to be significantly linked to users’ emotions (19). Potentially, this sense of presence could be manipulated by the creator of an IVE in order to elicit high emotional responses from the court actors using the IVE. For example, if the lighting of a specific IVE scene was slightly darkened, the emotional responses elicited from the users could be influenced to produce increased feelings of negative emotions (i.e., anxiety, anger, sadness) (19).
Tying in the issues with implicit bias mentioned previously, due to heightened emotional responses affecting one’s decision making process, one’s ability to control or be aware of the
8 influence of implicit biases is greatly limited. In other words, when one experiences
heightened emotions, one’s decision making abilities is compromised, and the influence of implicit biases is increased (8). This is highly problematic in regard to making legal
decisions. This illustrates the potential for VR and AR evidence to be wrongly understood by their users, which ultimately could be detrimental when used in a legal context.
4.1.4 Body Ownership Illusions (BOIs)
Another way in which heightened emotional responses can psychologically influence the legal actors in a courtroom is through the IVE users’ sense of ‘presence.’ More specifically, when the IVE user experiences a strong sense of ‘presence’, one could experience body ownership illusions (BOIs) (19, 28). BOIs are defined as the illusions of owning either part of a body or an entire body other than one’s own body and are created through the use of “non-bodily objects”, such as VR or AR technology (19). This concept is well illustrated through a study by Seinfield, et.al. In this study, VR technology was used on male individuals who had been convicted of domestic violence in order to simulate a body ownership illusion (BOI) which allowed the offenders to be in the body of a domestic abuse victim (28). From the results of this study, it was concluded that when one experiences a body ownership illusion, one’s recognition of fear is enhanced (28). This aspect of VR and AR technology can be psychologically influential on the court actors; If the VR or AR system is set up to have the user be placed in the virtual body of the victim, the user may perceive fear as the victim may have, causing the user to have both a bias in favour of the victim and a heightened emotional response which could affect their decision making process. In addition to the ability of BOIs to enhance the user’s recognition of fear, BOIs also increase a user’s emotions and
empathetic feelings (19). When a user experiences a BOI created by an IVE, the user’s emotions are in response to the scenario set forth in the IVE which ultimately causes the user to identify as the virtual avatar – leading to the BOI (19). This can be an issue in a courtroom as depending on the viewpoint of the virtual avatar and on the scenario set forth in the IVE, the user may sympathise with which ever party the virtual avatar belongs to. If the viewpoint of the virtual avatar in an IVE is of the victim, the court actor using the IVE may experience a BOI, which would cause them to believe as though they are that individual (the victim) for a moment. This would cause the court actor to develop a likening to the victim and become biased in favour of the victim. The alternative is true for if the viewpoint of the virtual actor in an IVE is of the suspect.
The scenario set forth in the IVE could be just as influential as the viewpoint of the avatar in the IVE. As mentioned previously, the scenarios in an IVE can intentionally be manipulated in order to elicit specific emotions. When an IVE user experiences a BOI, these emotions may be easier to elicit (19). For example, if the scenario is set with gruesome imagery, the effects of gruesome visual evidence mentioned above would apply here as well, leading to the user of the IVE to experience high emotional responses and increased empathy for a particular party. These responses would be ever so heightened if the user of the IVE was experiencing a BOI.
Ultimately, the visuomotor sensory stimulation in which the user experiences using either AR or VR technology may evoke a BOI, causing the user to become biased. Due to IVEs being man-made, there is a possibly for IVEs to be designed to intentionally elicit these BOIs and or heightened emotions from the user. Both of these aspects of IVEs present a substantial problem in terms of using VR and AR technology in a courtroom setting.
9
4.1.5 Virtual Memories Becoming Real Memories
Due to the intense feelings, one may experience when in an IVE, especially when one experiences a BOI, one’s virtual memories may become “real.” In other words, the virtual memories one makes while in an IVE may become remembered as real memories, as events that occurred in real life, not in the virtual IVE (27). This phenomenon would not be ideal if IVE technology is used for forensic reporting and legal decisions are made on the basis of possible confused memories. Additionally, this psychological influence of IVE technology could potentially be a problem when being used for aid in witness or suspect interrogations. If IVE technology is used for interrogations of witnesses or suspects, the individual may
become confused and be led to give false testimony or a false conviction on the basis of the new virtual memories.
With virtual memories being confused as real memories, if used in a legal setting, the court actor will most likely give this type of evidence a higher weight of importance and credibility when making their legal decisions (27). Again, compounding this unwanted effect of IVE technology.
4.1.6 Perception Inaccuracy
Another way in which IVE technology may cause confusion for its users deals with the problem with perception accuracy within IVEs. For the purposes of this paper, perception is defined according to the second definition listed in the Oxford dictionary: how an individual understands or interprets something. An individual using IVE technology may struggle to accurately perceive the distance of objects from themselves or from other objects, the speed of objects, or the physical location of themselves or their body parts. One study illustrating this perception inaccuracy assessed how an individual using IVE technology assessed the blame of a two-car collision at different viewpoints. Using the same simulated car collision scenario, with the cars preceding at the same speed, users at varying viewpoints assessed the blame of the car collision differently. From the ‘overhead’ viewpoint, which placed the participants behind and above car number 1, participants assessed car number 1 as having 43% culpability; From the ‘internal viewpoint’, which placed the participants inside car number 1, participants assessed car number 1 as having 34% culpability; From the ‘facing’ viewpoint, which placed the participants facing directly towards car #1, participants assessed car number 1 as having 92% culpability (19). From the results of this study, it can be
concluded that the viewpoint of the IVE user can significantly affect the way in which an individual perceives an event. This can cause issues when IVE technology is used in a courtroom as, depending on the viewpoint of the user in the IVE, the user may come to different legal decisions. Various other studies have found consistent results illustrating the inaccuracy in IVE technology user’s identifying, or perceiving, their own body or body parts (19). In one particular study, a head mount display in conjunction with a hand-tracking data glove was worn by participants. These participants were asked to identify their real right hand. However, majority of participants identified their virtual right hand instead of their real right hand. This is inaccurate as one’s virtual right hand is not the same as one’s real right hand. In the case of this one study, the participants’ real right hand was “located
approximately twenty centimetres away” from their virtual right hands (19). The findings of this study illustrate how users of IVE technology can often misperceive their own body or parts of their own body (19).
4.1.7 Psychological Conditions
Psychological conditions may influence the court actors’ decision making process when IVE technology is used in a courtroom. These psychological conditions would involve both
pre-10 existing psychological conditions as well as psychological conditions which developed due to the use of IVE technology in the courtroom. In terms of the influence of IVE technology on pre-existing psychological conditions, the effects vary depending on the type of
psychological condition one is affected by. Anxiety disorders is one general type of psychological conditions that could be affected (26). Anxiety disorders include conditions such as post-traumatic stress disorder (PTSD), generalized anxiety disorder, panic disorder, specific phobia disorders, social anxiety disorder, as well as other. Individuals who suffer from anxiety disorders have been found to experience increased levels of stress, emotion dysregulation, and intrusive memories when immersed in a ‘traumatic’ IVE using VR technology (26). Individuals with high levels of anxiety tend to be more “vulnerable to traumatic events due to less effective stress processing” (26). This would cause issues when introducing IVE technology involving possibly graphic imagery or high stress scenarios to court actors who may have anxiety disorders. These individuals would suffer great
psychological distress from their experience with the IVE technology as well have their decision making process greatly hindered due to their high levels of stress (26). Additionally, individuals with diagnosed mood disorders, which include depression and related disorders, could be at risk for experiencing an increase in negative emotions and negative thoughts (i.e., thoughts about self-harm). One study found participants who experienced a scenario through VR technology had higher reports of negative emotions to the scenario as well as more reports of negative thoughts hours after the VR experience when compared participants who experienced a scenario through a computer screen (29).
In terms of the influence of IVE technology on the development of psychological conditions, various conditions can result after an individual has had an experience with IVE technology. Evidence has shown that all court actors are vulnerable to the development of psychological health conditions after being exposed to graphical evidence in a courtroom – from members of a jury to judges and other court staff members (30, 31). These psychological health conditions include post-traumatic stress disorder, anxiety, depression, and their
accompanying symptoms such as nightmares and flashbacks (30). Additional evidence has shown that prolonged exposure to gruesome evidence has caused severe PTSD in court actors (30). IVE technology has been shown to elicit heightened emotions in its users, therefore, it can be hypothesized that IVE technology would intensify the effects of gruesome graphics, potentially causing more, and or more severe, psychological health conditions in court actors.
4.1.8 Psychophysiological Responses
Aside to psychological health conditions, IVE technology could have an influence on a user’s psychophysiological responses. One study had found that, compared to 360-degree
panoramic photographs, which tend to elicit the strongest psychological responses, IVE technology tends to elicit the strongest physiological responses (32). Along the same lines, another study found that users exposed to a stressful situation in an IVE could experience an increase in both their physiological and psychological stress response (9). These findings suggest the IVE technology is capable of eliciting psychophysiological responses in its users. IVE technology may be capable of this due to the users of IVE technology experiencing virtual synaesthesia. Virtual synaesthesia is defined as “a sensation occurring when single-modal (or multi-single-modal) stimulus sets off the simultaneous sensation over other senses – involuntarily and automatically” (9). This intense, immersive sensation can cause users to become dizzy and nauseous, a feeling similar to when one experiences motion sickness (33). This immersive sensation can also cause various other psychophysiological responses in users of IVE technology. One example of a response is illustrated through a study focusing on if AR is able to cause involuntary sensations, specifically looking at if users react to seeing and hearing their own hands burn through AR technology. This study had found that
11 all participants had a significant increase in skin conductance throughout the entire study (9). The same study had also found that participants who reported an involuntary heat sensation also experienced greater skin conductance responses when compared to participants who did not report any heat sensation (9).
The ability for IVE technology to elicit psychophysiological responses in its users has significant implications for the use of IVE technology by court actors. If a court actor experiences an adverse psychophysiological response to the use of IVE technology, their decision making may be greatly influenced in a negative way. This is due to the court actors focusing their attention on their adverse reactions rather than on the information being presented. It is also due to adverse psychophysiological responses to IVE technology having a correlation with slower cognitive processing (34). Therefore, if a court actor experiences an adverse psychophysiological response to the use of IVE technology, their ability to make an informed legal decision may be affected due to the adverse reaction causing limited
processing of the information being purveyed in the IVE.
5 Discussion
All in all, it can be said that virtual reality and augmented reality technology in legal settings have various psychological influences on the court actors. It should be noted that none of these influences happen in singularity. Each of the aforementioned psychological influences happen with conjunction with one another, only enhancing the effect of each other. With all of these confounded psychological influences resulting from the use of IVE technology, the effects on the decisions made by the court actors in a courtroom can be detrimental. It is important to be aware of all of the possible psychological influences of VR and AR technology before it is seamlessly implanted into all legal systems.
In hopes to minimize the psychological influences in which VR and AR technology can have on court actors in a legal setting, Table 2 presents a list of possible safeguards in which courts from both legal systems can enact. This list was created based on the information presented in section 3 of this paper.
Table 2: Recommendations of use of IVE technology for forensic reporting in a courtroom
General Recommendations:
1) Offer proper training to all court actors to educate all on how to be aware of the influence of implicit biases when interacting with IVE technology.
2) Offer proper training to all court actors to educate all on how to minimize adverse psychophysiological effects of IVE technology.
3) Offer proper training to all court actors in order to help minimize the psychological effects of IVE technology. In recent years, VR and AR technology has been shown to be very effective in helping to treat a multitude of psychological conditions – including all of the conditions mentioned above that may be worsened by the use of VR and AR technology. Due to the possible gruesome graphical content that may be shown through the IVE technology used in court, the use of VR and AR technology in court will most certainly not be therapeutic. However, with proper training by medical professionals, the psychological effects of IVE technology can be greatly minimized.
12 4) Provide mental health counselling available to all court actors to help minimize the
psychological effects of exposure to gruesome graphical evidence and to intense, immersive IVE technology.
5) In lines with the U.S. Federal Rules of Evidence 403, the evidence offered using IVE technology should have a “probative value” which “substantially outweighs” its “prejudicial impact” (2)
6) Conduct mental health screening on all court actors prior to introducing IVE technology into the courtroom in order to help minimize the adverse effects of pre-existing psychological conditions
Recommendations for the usage of IVE technology:
1) Display what the IVE user is seeing so that all other court actors can see in order to ensure the user is in the correct virtual orientation and viewing the proper
evidence/scenario (35, 16).
2) If using IVE technology to show a scenario in the viewpoint of one of the legal parties (i.e., suspect, victim, eyewitness), then viewpoints of all the other legal parties should be offered as well.
3) If using IVE technology to show a scenario from one geographical viewpoint, other geographical viewpoints should be offered in order to minimize the possible effect of perception inaccuracy within IVEs.
4) Allow only trained court actors should be allowed to use the IVE technology. Unless it is possible to train a jury before trial, this would mean that the members of a jury in adversarial systems would have to be excluded from using IVE
technology. This is due to the fact that a jury is typically composed of individuals from the general public. The general public may be confused, misunderstand, or be biased by the information presented in an IVE (36).
5) Before a court actor can use IVE technology, an expert must explain what information is being shared through the IVE (36).
6) Before a court actor can use IVE technology, an expert should explain the general difference between one’s physical, real location versus one’s perceived, virtual location when one uses an IVE system. The difference in perception should be specific to the IVE system in use that day.
This list of recommendations is not all inclusive but should be considered as a start for
establishing guidelines for the incorporation of VR and AR technology into forensic reporting within a courtroom.
6 Conclusions and Recommendations
The completion of this proposed literature research has provided insight into how the use of virtual reality and augmented reality within forensic reporting in legal settings may have unwanted influences on court actors’ decision making processes. With this knowledge, future research can be done in order to examine the recommended countermeasures as well as to establish new countermeasures in order to help minimize these potential unwanted influences when using IVE technology, specifically virtual reality and augmented reality in legal
13 inquisitorial legal systems to objectively incorporate the use of augmented reality and virtual reality in forensic reporting within a courtroom. Additional future research should also be conducted to more directly examine the psychological impacts and influences of IVE technology as there is limited research on this topic to date. Further thorough research
examining the direct psychological influences of IVE technology on various court actors will aid in creating new effective countermeasures for using virtual reality and augmented reality within forensic reporting in legal settings.
14
7 References
1. Jackson ARW, Jackson JM, Mountain H, Brearley D. FORENSIC SCIENCE Third edition [Internet]. 2011. Available at: www.pearsoned.co.uk
2. Bright, D.A., & Goodman-Delahunty, J. (2006). Gruesome Evidence and Emotion: Anger, Blame, and Jury Decision-Making. Law and Human Behavior, 30(2):183-202. doi: 10. 1007/s10979-006-9027-y.
3. Sieberth, T., Dobay, A., Affolter, R., & Ebert, L. (2019). A toolbox for the rapid prototyping of crime scene reconstructions in virtual reality. Forensic Science
International, 305. Available at: https://doi.org/10.1016/j.forsciint.2019.110006
4. Dremliuga, R., Dremliuga, O., & Lakovenko, A. (2020). Virtual Reality: General Issues of Legal Regulation. Journal of Politics and Law, 13(1):75-81. Available at: https://doi.org/10.5539/jpl.v13n1p75
5. Anthes, C., García-Hernandez, R.J., Wiedemann, M., & Kranzlmüller, D. (2016). State of the Art of Virtual Reality Technologies. 2016 IEEE Aerospace Conference, Big Sky, MT, 2016, 1-19. doi: 10.1109/AERO.2016.7500674.
6. “History Of Virtual Reality.” Virtual Reality Society, 2 Jan. 2020, Available at; www.vrs.org.uk/virtual-reality/history.html.
7. Flavián, C., Ibáñez-Sánchez, S., & Orús, C. (2020). Impacts of technological embodiment through virtual reality on potential guests’ emotions and engagement.
Journal of Hospitality Marketing & Management. Available at:
https://doi.org/10.1080/19368623.2020.1770146
8. Salmanowitz, N. (2016). Unconventional Methods for a Traditional Setting: The Use of Virtual Reality to Reduce Implicit Racial Bias in the Courtroom. The University of
New Hampshire Law Review, 15(1):117-160. Available at:
http://scholars.unh.edu/unh_lr/vol15/iss1/2
9. Eckhoff, D., Cassinelli, A., Liu, T., & Sandor, C. (2020). Psychophysical Effects of Experiencing Burning Hands in Augmented Reality. In: Bourdot P., Interrante V., Kopper R., Olivier AH., Saito H., Zachmann G. (eds) Virtual Reality and Augmented Reality. EuroVR 2020. Lecture Notes in Computer Science, vol 12499. Springer, Cham. Available at: https://doi.org/10.1007/978-3-030-62655-6_5
10. El Jamiy, F., & Marsh,R. (2019). Distance Estimation In Virtual Reality And Augmented Reality: A Survey. 2019 IEEE International Conference on Electro
Information Technology (EIT), Brookings, SD, USA, 2019, 063-068. doi:
10.1109/EIT.2019.8834182.
11. Jadhav, E.B., Sankhla, M.S., & Kumar, R. (2020). Artificial Intelligence: Advancing Automation in Forensic Science & Criminal Investigation. Journal of Seybold Report, 15(8):2064-2075. Available at: https://www.researchgate.net/publication/343826071 12. Sieberth, T., Dobay, A., Affolter, R., & Ebert, L. (2018). Applying virtual reality in
forensics – a virtual scene walkthrough. Forensic Science, Medicine and Pathology, 15:41–47. Available at: https://doi.org/10.1007/s12024-018-0058-8
13. Bulgakov, V., Trushchenkov, I., & Bulgakova, E. (2019). Spherical Panoramic Photo Shooting and Virtual Reality Demonstration of a Crime Scene. In: Kravets A.,
Groumpos P., Shcherbakov M., Kultsova M. (eds) Creativity in Intelligent
15
Information Science, 1084:217-225. Springer, Cham. Available at:
https://doi.org/10.1007/978-3-030-29750-3_17
14. Schofield, D. (2011). Playing with evidence: Using video games in the courtroom.
Entertainment Computing, 2(1):47-58. Available at:
https://doi.org/10.1016/j.entcom.2011.03.010
15. Singh, S. (n.d.). The Use of Virtual Reality in Courtroom Trials. Seminar Paper, Chicago-Kent College of Law. Available at:
http://blogs.kentlaw.iit.edu/perrittseminar/files/2019/05/Seminar-Paper_Singh.Sukhdeep-1.pdf
16. Schofield, D., & Fowle, K. (2013). Technology Corner: Visualising Forensic Data: Evidence Guidelines (Part 2). Journal of Digital Forensics, Security and Law, 8(2). doi: 10.15394/jdfsl.2013.1145. Available at:
https://commons.erau.edu/jdfsl/vol8/iss2/4
17. Ma, M., Zheng, H., & Lallie, H.(2010). Virtual Reality and 3D Animation in Forensic Visualization. Journal of Forensic Science, 55(5):1227-1231. doi: 10.1111/j.1556-4029.2010.01453.x.
18. Sieberth, T., Dobay, A., Affolter, R., & Ebert, L. (2019). A toolbox for the rapid prototyping of crime scene reconstructions in virtual reality. Forensic Science
International, 305. Available at: https://doi.org/10.1016/j.forsciint.2019.110006
19. Bunker, K. (2019). From Presentation to Presence: Immersive Virtual Environments and Unfair Prejudice in the Courtroom. Southern California Law Review, 92:411-440. Available at: https://southerncalifornialawreview.com/2019/01/02/from-presentation- to-presence-immersive-virtual-environments-and-unfair-prejudice-in-the-courtroom-note-by-khirin-bunker/
20. Koller, S., Ebert, L.C., Martinez, R.M., & Sieberth, T. (2019). Using virtual reality for forensic examinations of injuries. Forensic Science International, 295:30-35.
Available at: https://doi.org/10.1016/j.forsciint.2018.11.006
21. Jackson, D. (2009). Adversarial and Inquisitorial Systems. Medico-Legal Society of NSW Inc Scientific Meeting, March 2009. Available at:
https://medicolegal.org.au/wp-content/uploads/2018/04/185.pdf
22. Bright, D.A., & Goodman-Delahunty, J. (2006). Gruesome Evidence and Emotion: Anger, Blame, and Jury Decision-Making. Law and Human Behavior, 30(2):183-202. doi: 10. 1007/s10979-006-9027-y.
23. Schweitzer, K., & Nuñez, N. (2018). What Evidence Matters to Jurors? The Prevalence and Importance of Different Homicide Trial Evidence to Mock Jurors.
Psychiatry, Psychology and Law, 25(3):437-451. Available at:
https://doi.org/10.1080/13218719.2018.1437666
24. Salamanowitz, N. (2018). The impact of virtual reality on implicit racial bias and mock legal decisions. Journal of Law and the Biosciences, 174-203. doi:
10.1093/jlb/lsy005
25. Meadows, P., & Sonoda, K. (2020). Implicit Bias, Part 1 [PowerPoint slides]. Presented at: PAFP Virtual CME Conference. 13 November 2020 [cited 19 January 2021]. Available from: http://www.pafp.com/Images/112020PerryKentoPart1.pdf
16 26. Schweizer, T., Schmitz, J., Plempe, L., Sun, D., Becker-Asano, C., Leonhart, R., &
Tuschen-Caffier,B. (2017). The impact of pre-existing anxiety on affective and cognitive processing of a Virtual Reality analogue trauma. PLoS ONE, 12(12). Available at: https://doi.org/10.1371/journal.pone.0190360
27. Leonetti, C., & Bailenson, J. (2010). High-Tech View: The Use of Immersive Virtual Environments in Jury Trials. Marquette Law Review, 93(3):1073-1120. Available at: htp://scholarship.law.marquete.edu/mulr/vol93/iss3/4
28. Seinfeld, S., Arroyo-Palacios, J., Iruretagoyena, G., Hortensius, R., Zapata, L.E., Borland, D., de Gelder, B., Slater, M., & Sanchez-Vives, M.V. (2018). Offenders become the victim in virtual reality: impact of changing perspective in domestic violence. Scientific Reports, 8:2692. Available at: https://doi.org/10.1038/s41598-018-19987-7
29. Lavoie, R., Main, K., King, C., & King, D. (2020). Virtual experience, real consequences: the potential negative emotional consequences of virtual reality gameplay. Virtual Reality. Available at:
https://link.springer.com/article/10.1007/s10055-020-00440-y
30. Haragi, M., Yamaguchi, R., Okuhara, T., & Kiuchi, T. (2019). Interviewing forensic specialists regarding medical-legal illustration methods to replace gruesome graphic evidence. Journal of Visual Communication in Medicine. Available at:
https://doi.org/10.1080/17453054.2019.1687287
31. Haragi, M., Yamaguchi, R., Okuhara, T., & Kiuchi, T. (2020). Questionnaire survey of a mock jury on their impressions of medical-legal illustrations aimed at reducing trauma and PTSD of jurors. Journal of Visual Communication in Medicine. Available at: https://doi.org/10.1080/17453054.2019.1707074
32. Higuera-Trujillo, J.L., Lopez-Tarruella Maldonado, J., & Llinares Millán, MDC. (2017). Psychological and physiological human responses to simulated and real environments: A comparison between Photographs, 360˚ Panoramas, and Virtual Reality. Applied Ergonomics, 65:398-409. doi:10.1016/j.apergo.2017.05.006
33. (2018). VR in the Courtroom: Virtual reality is making its way to courtrooms but still has barriers to overcome. LEGAL TECH NEWS, 24(2):L11.
34. Szpak, A., Michalski, S.C., Saredakis, D., Chen, C.S., & Loetscher, T. (2019). Beyond Feeling Sick; The Visual and Cognitive Aftereffects of Virtual Reality. IEE
Acess. doi: 10.1109/ACCESS.2019.2940073.
35. Casey, P., Lindsay-Decusati, R., Baggili, I., & Breitinger, F. (2019). Inception: Virtual Space in Memory Space in Real Space – Memory Forensics of Immersive Virtual Reality with the HTC Vive. Digital Investigation, 29:S13-S21. Available at: https://doi.org/10.1016/j.diin.2019.04.007
36. Schofield, D., & Fowle, K. (2013). Technology Corner Visualising Forensic Data: Evidence (Part 1). Journal of Digital Forensics, Security and Law, 8(1). doi: 10.15394/jdfsl.2013.1141.
17
8 Appendix
8.1 Search Strategy
The search for the literature was conducted via Google Scholar and Science Direct. In order to gather literature sources about IVE technology, the search began by using general terms such as “virtual reality technology”, “augmented reality”, and “virtual reality and augmented reality.” Further searches were conducted using terms “forensic reporting with virtual
reality”, “forensic visualization augmented reality”, and “forensic visualization virtual reality” in order to gain information on the use of VR and AR technology in forensic reporting. Additional searches were performed to find literature on the use of VR and AR technology in a courtroom using terms such as “virtual reality in courtroom” and “augmented reality in courtroom” and “use of virtual reality in forensic cases.” In order to collect
literature on the psychological impacts and the psychophysiological impacts of VR and AR technology and their use in a courtroom, the terms “psychology”, “psychophysiology”, “virtual reality”, “augmented reality”, “forensic evidence”, and “forensic videos” were used in different variants to search for literature.
