The Memory Palace in Virtual Reality: Does immersion
improve recall?
SUBMITTED IN PARTIAL FULLFILLMENT FOR THE DEGREE OF MASTER
OF SCIENCE
HIDDE HENSEL
6379176
M
ASTER
I
NFORMATION
S
TUDIES
H
UMAN-
C
ENTERED
M
ULTIMEDIA
F
ACULTY OF
S
CIENCE
U
NIVERSITY OF
A
MSTERDAM
August 24, 2016
The Memory Palace in Virtual Reality: Does immersion
improve recall?
by Hidde Hensel
6379176
University of Amsterdam Supervisor: Frank Nack Second Reader: Dan Buzzo
ABSTRACT
The Memory Palace (MP) technique is a mnemonic device that lets the user enhance their memory by binding memo-ries, words, numbers, or images to certain (imaginary) ob-jects in certain (imaginary) places. Virtual Memory Palaces visualise these imaginary places into digital media, which can be used with the same results as the traditional Mem-ory Palace method but with an increased compliance rate. With the MP method being a combination of an associative mnemonic and a form of spatial memory, it is believed that it could be improved by enhancing either one. Immersion through virtual reality improves spatial memory. Because of this, a VR Memory Palace was built with Unity3D to research if VR has an effect on the recall of lists of words compared to NonVR with the MP method in a virtual MP environment. No significant differences were found. It seems that spatial memory plays only a small role in the MP method. And that associations between objects and infor-mation chunks in the VMP environment are far more im-portant to the effectiveness of the MP method.
Keywords
Mnemonic device, Working memory, Memory Palace, Vir-tual Memory Palace, VirVir-tual Reality
1.
INTRODUCTION
In ancient times, a good memory was important, and many techniques were developed to enhance memories [26]. These memory techniques, or mnemonic devices, would help the user to bind (abstract) data or knowledge to something more meaningful and relatable to them, which resulted in better retention of this information. One example of a popular mnemonic technique is the knuckle mnemonic, a memory technique often taught at elementary school to remember the number of days in each month of the Gregorian Cal-endar [13]. Generally, these mnemonic techniques are used for remembering lists, numerical sequences, faces and/or im-ages.
However, a paradigm shift occurred when people started us-ing other tools to replace our natural memory [7]. The de-velopment of tools like the printing press, books, computers, and the Internet all contributed to the (debatable) replace-ment or extension of our natural memory by artificial mem-ory [7]. A lot of mnemonics therefore stopped being used and developed. As a result, most mnemonics are not well known, because the majority of the population is unaware of them, or they think using it takes a lot of training and effort [16].
However, it is still important to have good memory retention for a variety of reasons. Mnemonic techniques could be help-ful in everyday problems such as remembering passwords, studying for tests and participating in them, or for remem-bering topics while giving a speech. With new technologies these ancient mnemonics can be made more accessible [14] and may be improved [12].
There is one highly effective mnemonic device, particularly for remembering lists of information, that uses environments as its primary theme: The Memory Palace (MP) or Method of Loci (MoL) [26]. The MP technique or MoL is a mnemonic that lets the user enhance their memory by binding memo-ries, words, numbers, or images to certain (imaginary) ob-jects in certain (imaginary) places. The combination of a thoroughly known environment, and association of informa-tion with the objects in the environment gives improved in-formation recall. It is even used by participants in memory contests [9].
The conventional memory palace technique needs a user to think of, and imagine, an environment with as much detail as possible. But what if there was a visualization of this environment? Research pointed out that the use of a vir-tual visualization of a memory palace (VMP) with the use of a desktop computer and monitor gives the same amount of recall and a better compliance rate than using the con-ventional memory palace method for naive participants [14]. It is indicated that with immersion through virtual reality, it might be possible to improve current virtual memory palaces [12] and thus surpass recall accuracy of the conventional MP method. Thus, it is strongly suggested that, with the use of virtual reality technology, the old memory palace mnemonic can be improved. In this thesis we assess this problem by looking at previous research on VMPs and the effect of VR on memory, and building an application to test this.
In chapter 2, we discuss previous research of memory and cognitive load. Additionally, other virtual memory palace studies are examined. In chapter 3, the main research ques-tion is formed based on related work. In chapter 4, the methods of the experiment are explained. In chapter 5, re-sults of the experiment and statistical tests are shown. In chapter 6, these results are discussed and possible future work and applications are suggested. Finally, in chapter 7, conclusions are drawn.
2.
RELATED WORK
2.1
Memory and cognitive load
Since the Memory Palace method is a mnemonic to enhance memory, some initial research into memory was done. Work-ing Memory in particular, since researchWork-ing the short term effects of the MP method on memory will be the goal of this study.
2.1.1
Working memory capacity
Working memory is described as a limited-capacity mem-ory storage for holding and processing sounds and images in active consciousness [19]. Working memory capacity is on average limited to 4 chunks, but changes with age and differs among individuals [4]. A chunk may be any piece of information, from sentences to words to letters to numbers. Young adults, when confronted with lists, have an average core working memory capacity of about 3 chunks of list con-tent, regardless of list size [3]. Another study concluded that young adults have an average working memory capacity of about 3 to 5 chunks, where chunks are person’s names in an ordered list [11]. When remembering a list, first and last items are remembered better due to a a phenomenon called the serial position effect [22]. This is due to a primacy effect that occurs because first items in a list have more time to be repeated and put into memory, and due to a recency effect that occurs because the last items of a list are remembered better since it is fresher in memory [23]. The recency effect can be countered by a period of distracting activity after remembering the list [23] [10].
2.1.2
Cognitive load
Working memory is said to consist of a central processor, which controls a number of channels that take in different kinds of stimuli [19]. Two of such channels are described when using working memory within multimedia learning. One channel is mainly for visual information, the other for speech based material [1]. When using working memory with multimedia learning, cognitive load plays a large role. The two main channels have limited capacity, each with a certain cognitive load threshold. There are numerous ways to mitigate this cognitive load between channels. The eas-iest way is to divide cognitive load evenly among channels by design. Cognitive load can be manipulated by instruc-tional design [24]. For example, if elements can be learned successively rather than simultaneously, cognitive load will be low [24]. Also, related information should be together visually to reduce cognitive load, so that persons do not have to use cognition to mentally bind them together [2]. Additionally, cognitive load can be reduced by eliminating extraneous visual material, or by providing cues for how to process this new information [20].
These fundamentals of multimedia learning are quite sim-ilar to the MP guidelines Yates described in her The Art of Memory [26]. The optimal memory palace has to be a solitary known environment with no unnecessary objects, which as described in cognitive load theory, results in re-duced cognitive load. Objects should be along a path so they can be accessed sequentially. Rooms should be distinct and not repetitive to promote binding related information together. Rooms should not be too spacious or too narrow, nor too dark or bright to further reduce cognitive load. Ad-ditionally, cognitive load is reduced by providing the user with instructions on how to process the information to be learned by creating associations.
2.1.3
Memory and Virtual Reality
According to Mania et al [17], people may find environments and events in VR more memorable. Also, spatial memory is improved when using VR [18]. However, in a study by Moreno et al [21] about learning by attending a lecture in VR, students gave higher ratings of presence when learning with HMDs, but this did not affect performance on mea-sures of retention, transfer, or program ratings [21]. VR is a relatively new and sparsely used technology, so people are not accustomed to it. This can cause an increase in cognitive load, which means a decrease in working memory resulting in a decrease in recall accuracy. Due to virtual re-ality sickness or motion sickness, participants could become distracted using virtual reality which could affect memory recall accuracy negatively.
It has to be noted that most studies about memory in VR were published over 13 years ago. Current VR technology is a lot more advanced and thus current VR technology might give different results.
2.2
Virtual Memory Palaces
Building a version of a virtual memory palace has already been the subject of previous studies. However, since previ-ous methods all used (virtual) custom variants of the original method, there are still countless of possibilities and oppor-tunities to experiment with. The custom virtual tools that are already developed only apply some elements of the tradi-tional MP technique, and have no VR support and no words spatially bound to the objects. A total of three studies were done on virtual memory palaces.
2.2.1
Building a memory palace in minutes
Legge et al [14] researched the ability to recall lists of words using the conventional method of loci (cMOL) and a cus-tom virtual method of loci (vMOL). Results from partici-pants using either one of these methods were compared to a control group without a specific assigned memory method. Participants using the vMOL could explore a virtual mem-ory palace by using a desktop with a monitor, see figure 1. These participants could navigate through the virtual envi-ronment using a keyboard and mouse. The virtual memory palaces used were pre-built virtual environments modelled after familiar and recognizable settings, namely a school, warehouse, and home setting. Participants had to recall a total of 10 lists of words, with 11 words in each list. The vMOL participants used pre-built virtual environments,
Figure 1: Virtual memory palace environment used in the study by Legge et al [14].
which they were not familiar with. These vMOL partici-pants only got 5 minutes to familiarize with the environ-ment, and only got to read a short description of the MOL method and had to apply it straight away. The same amount of recall was perceived by using the vMOL and the cMOL, which both outperformed the control group. No learning or training effect was noticed, in contrast to what previous research suggested. However, all participants were Psychol-ogy students. The study PsycholPsychol-ogy has the MOL method in its curriculum. This may explain the missing learning and training effect observed, since participants were possibly al-ready aware of how the MOL worked and may have used it before. Further conclusions of the study were that the type of environment did not matter. However, all tested environ-ments were quite similar. Also, the study concluded that the MOL is not particularly useful for remembering items in a particular order, but it is more useful for remembering lists in general.
2.2.2
Virtual Memory Palaces: Immersion aids
Re-call
Figure 2: Virtual memory palace environment used in the study by Krokos [12] during (A) and after (B) the memory user test.
To research if immersion through virtual reality with a head mounted display (HMD) and use of MOL had an effect on memory recall, Krokos et al [12] used a custom MP method in a virtual tool for use with either a monitor or a head
mounted display (HMD). In this study, memory palaces were pre-built with faces of known celebrities and cartoon figures visible, as can be seen in figure 2a. Participants were asked to first remember and later recall these faces. Participants could look around with the mouse with desktop and by ro-tating their head with a head mounted display. However, participants could not move or walk inside the virtual mem-ory palace. A route or path through the Memmem-ory Palace is a staple in the conventional MP method as described by Yates [26].
The participants using a HMD had a significant better mem-ory recall accuracy than participants using just a monitor. Additionally, the experiment had the same recall accuracy when done in an ornate palace compared to a medieval town, suggesting that the type of environments used in a VMP do not matter. Recall in this study was done while using the tool in the virtual environment, but with the faces removed, as seen in figure 2b. However, the MP method is often used for recall at tests, competitions, during speeches, or at other events where the user can only rely on his memory. This experiment used a form of cued recall, which differs from free recall which the MP method is mainly used for. The chunks of information that had to be remembered by par-ticipants were visual images in the form of faces, and not letters or words. Different chunks of information used in the MP method may lead to different results. Also, this VMP experiment did not use objects to bind with the to be recalled information, but focused more on the location of the information within the environment to remember the items. This takes out the whole associative part of the MP method, but strengthened the fact that VR does improve spatial memory.
2.2.3
The virtual memory palace
Another study about virtual memory palaces is the virtual memory palace of Fassbender et al [8]. This research pro-poses the use of an explorable pre-built Virtual Memory Palaces (VMPs). A VMP is a virtual environment that con-tains several objects that can be activated. These objects are for example parts of a wall, a table, or a curtain. These objects can be loaded and filled with an image file. Thus, this study was about using these objects in a VMP to as-sist in remembering and recalling images of certain objects. Some objects had an activation animation to get the partic-ipant’s attention. This animation also had an added sound for this reason.
Results strongly indicate a positive influence when the VMP was used for long term memory. Another result was that participants enjoyed the memory palace concept and using the VMP. However, this study was only an initial scoping study, with a total of only 15 participants and no real sta-tistical tests to conclude any significance. Also, this study did not use virtual reality to test if immersion with a HMD affected enhanced recall of information.
3.
RESEARCH QUESTION
Due to the Memory Palace mnemonic being an associative mnemonic combined with spatial memory [26], and the no-tion that VR might improve memory recall with the MP method [12], the choice was made to research the effect of Virtual Reality on recall using the MP method. Because of
young adults being frequently studied in experiments with cognitive memory capacity, the choice was also made to fo-cus on this population group. Because of previous research on Virtual Memory Palaces [14], the choice was made to use the same information to recall, namely lists of words. Thus, the following research question was formed:
Do young adults have improved memory recall of lists of words when using a Virtual Reality Memory Palace instead of a Virtual Memory Palace?
With the hypothesis:
H0: Young adults have improved memory recall of lists of words when using a Virtual Reality Memory instead of a Virtual Memory Palace.
H1: Young adults do not have improved memory recall of lists of words when using a Virtual Reality Memory instead of a Virtual Memory Palace.
Because of the debatable conclusion of Legge et al [14] that no learning effect was perceived, the following subquestions were formed: Is there a learning effect noticeable when us-ing the Virtual Memory Palace method when list number progresses? And can young adults use the same Virtual Memory Palace environment multiple times with different lists of words without noticeable loss of memory recall?
4.
METHODS
Based on these research questions, the following experiment was set up. To be able to compare the VR group with the NonVR groups, the choice was made to build a Unity3D1
application. This is because of Unity3D’s ability to com-pile the same environment to both a 3D+VR and to a 3D application. This Virtual Memory Palace experiment was mainly modeled after previous VMP research [14], with lists of words as information to be recalled. To get a varied recall accuracy in order to compare the results of both VR and NonVR groups, list length had to be significantly higher than working memory capacity [4]. The choice was made to use a list length of 11 words for the main experiment, copying list lengths of related VMP work [14] [8]. To have the ability to notice some sort of learning effect of the MP method, and to notice how often participants could use the same environment with different information, the choice was made to use multiple iterations of the main phase of the ex-periment, with multiple lists. One list is used for one run of the VMP environment. Because of time constraints of the experiment, the choice was made to create a total of 5 lists. The main VMP environment is therefore used a total of 5 times for remembering 5 different lists of words.
4.1
Participants
The choice was made to seek for a total of 40 young adults aged 20-39 to participate. Young adults were eligible to be a participant if their native language was Dutch and if they were available to do the entirety of the experiment. All participants were assigned to either the VR or NonVR group based on order of partaking, alternating both VR and NonVR groups evenly. Written informed consent was
1
unity3d.com
obtained and participants were informed at the start that (continued) participation of the experiment was voluntary.
4.2
Materials
Different materials for both groups were used. VR tests were done using a Samsung Galaxy S6 Edge2 mobile phone with the Gear VR3 Head Mounted Display. Non-VR tests were
done using an Alienware4laptop with a 13 inch screen with a 1920x1080 resolution. Recall and information gathering was done by both groups through Google Forms5on the laptop.
Audio recording was done using the audio recorder on the mobile phone.
Movement, navigation and orientation in the environment could be done with a Moga Power6 controller for the VR application and with the keyboard and mouse for the Non-VR application.
The virtual Memory Palace environment was built using the Unity3D game engine and compiled to both Android7 (VR)
and Windows 10 8 (Non-VR). Objects used for this appli-cation were gathered from the Unity Asset Store9. Because of each list having 11 words, 11 objects were chosen for the main VMP environment. Specific details about the objects can be found in Appendix 10.1. Following the rules of build-ing a Memory Palace as described by Yates [26], the choice was made to keep rooms within the VMP not too big nor too small and distinctive from each other. The resulting VMP consists of three rooms and two hallways, with each room having a certain theme. The themes are derived from familiar known environments, specifically a living room, a bedroom and an office room theme. All rooms were deco-rated based on their theme, with the goal to easily tell them apart. Each room was filled with 3 objects, due to the guide-line of objects within MPs to be noticeably apart [26]. Walls and ceilings of the rooms were also made with distinct and different colours per room to further differentiate the rooms. Each hallway had a distinct painting, thus having only one object. The words the participants had to remember were projected on each object as can be seen in figure 3. This was done to reduce cognitive load. Detailed figures of the environment can be seen in Appendix 10.3.
To improve the overall experiment, some user testing was done with the VMP application. Five initial test runs were done, which helped determining the duration of the phases, and gave insight into parts of the application which were unclear for participants. Based on these test runs, text in the application was adjusted slightly, and the decision was made to record the audio of the experiment in order to get more insight of how the participants tried to remember the list of words. The goal of the audio recording is mainly to check the compliance rate of participants for using the MP method correctly. 2samsung.com/global/galaxy/galaxys6/galaxy-s6-edge/ 3 samsung.com/global/galaxy/gear-vr/ 4alienware.com 5 google.com/intl/en/forms/about/ 6mogaanywhere.com/controllers/heropower 7 android.com 8microsoft.com/nl-nl/windows 9 assetstore.unity3d.com
Words used were taken from the high-imageability word pool used by Madan, Glaholt, and Caplan [15] and Legge, Madan, Ng, and Caplan [14]. The word pool consisted of 110 English words, each 4-6 letters in length and matched for several orthographical and phonological word properties [15]. Of these 110 translated words, 55 were randomly picked, and translated to Dutch using Google Translate10. The words
were translated to Dutch because differences in recall can be found between native language and foreign languages, even on simple working memory tasks [25]. The order in which the words were picked was the order of sequence in the lists used in the Memory Palace environment.
Recall of words, demographic information, and confidence was recorded through a Google Forms form that participants were asked to use.
4.3
Procedure
The procedure is similar to that of previous research [14], and consists of three phases. These three phases are the practice phase, the exploration phase, and the memory phase. The practice phase has the goal of familiarizing participants with the design of the further experiment, particularly the memory phase. The exploration phase is to let the partic-ipant become familiar with the VMP environment used in the memory phase. The memory phase is the main phase of the experiment, which consists of remembering and recalling lists of words with the use of the MP method in the VMP environment.
Before the experiment took place, participants were informed about the average length of 40 minutes of the experiment and a short outline of the experiment was told. Partici-pants were also informed of the audio recorder, and were then asked to sign the written informed consent. After this, the first phase of the experiment started, named the practice phase.
Figure 3: Practice exploration phase, with the prac-tice environment shown.
4.3.1
Practice phase
In the practice phase, participants were instructed to follow instructions on the screen. The practice phase was made for the participants to get them used to the further course of the experiment. They were verbally asked if they were comfort-able and if they could think out loud for the entirety of the experiment. They were also given minor instructions about
10
translate.google.com
navigation and orientation within the application, through instructions on the screen. After this, a blank test environ-ment with 5 objects was shown, as can be seen in figure 3. Participants were encouraged to move around and explore the area for 30 seconds, to familiarize with the controls and the environment.
Figure 4: Practice memory phase, with the first object-word pair visible.
Participants were then informed that words would appear on the objects in turn, following a certain route. Participants were asked to try and remember these words. Each word appeared for 5 seconds, and the following word in the list immediately appeared in turn on the next object. After 30 seconds, participants were taken out of the environment and asked to do some mental calculations in the form of a quick calculation online game11 for 2 minutes. This was done to
counter a recency effect and to combat elaborative rehearsal of the words, which affects remembering [6]. The game el-ement is chosen so participants are encouraged to focus on the calculations and not on repetition of the words, though participants are explicitly told the result of the calculations was not part of the test. After this, they were asked to recall the words. They were asked to take into account the order that the words appeared, and the confidence they had that it was the right word and the right place. The time par-ticipants had for this practice recall phase of 5 words was 1 minute. Additionally, participants of the VR group were asked if they developed motion sickness. The practice phase took approximately 5 minutes to complete.
Figure 5: Exploration phase. The starting location of the first room within the main VMP environment.
4.3.2
Exploration Phase
11
During the exploration phase, participants were put into the main VMP environment, as shown in figure 5. The par-ticipant had 2 minutes to explore all three rooms and two hallways, and was urged to familiarize with the entirety of the environment. After this exploration phase, the partic-ipant was asked to read a small introductory text about the MP method adapted from Yates. This text is shown in the Appendix. The Exploration phase took approximately 4 minutes to complete.
Figure 6: The living room of the Memory Palace environment, with the first word of the list showing.
4.3.3
Memory Phase
This phase consisted of the main experiment and is alike the practice phase. However, it consisted of 5 rounds of each approximately 5 minutes. In every round, participants were asked to think out loud, and to use the Memory Palace method as explained to try and remember 11 words. These 11 words were showed sequentially, each for 7 seconds, on objects along a route through the environment. Participants were encouraged to bind the word to its object by use of some sort of association. The 7 seconds per word were chosen after user testing, it being enough time to create an association and to move to the object. After this memorization round, participants were asked to do some mental calculations in the form of a quick calculation online game12for 2 minutes. This was done to counter a recency effect and to combat elaborative rehearsal of the words, which affects remember-ing [6]. The game element was chosen so participants were encouraged to focus on the calculations and not on repeti-tion of the words, though participants were explicitly told the result of the calculations was not part of the experiment. After this mental calculation round, participants were asked to recall the 11 words, taking into account the position in the list, using Google Forms. As in the practice phase, partici-pants could indicate their confidence in their answer with a 5 item Likert scale, for it being the right word and the right list position. The total memory phase took approximately 25 minutes to complete.
4.4
Scoring
When scoring lists of words, multiple factors can be taken into account. In this study, particular focus was put on the position of the word within the list, correct spelling of the word, and use of synonyms. Based on these factors, the choice was made to score results in four different ways.
12
mathplayground.com/quick calculate.html
These are strict scoring, lenient scoring, strict no position scoring, and lenient no position scoring. Strict scoring means that the correct word is filled in at the correct position. Le-nient scoring means that synonyms, misspelled and/or sin-gular/plural mistakes of the word also count as a correct answer. No position scoring implies that exact position in the list does not matter.
This being a study of the effect of the VMP method on recall of lists of words, it is necessary to check results for compli-ance and proper use of the MP technique. Complicompli-ance will be perceived by studying the audio recordings of the experi-ment. A list result is compliant if the participant tried to use the method, by binding a word to its object, for more than 50 percent of the list. A participant is compliant if he or she has had three or more compliant lists. All inclusive results are calculated as well. These consist of both compliant and non compliant list results.
5.
RESULTS
For the main experiment, a total of 40 20-38 year old Dutch native speakers participated. As seen in table 1, participants of the VR group consisted of 12 males and 8 females, with an age between 20 and 36 years old (N = 20, µ = 25.8, σ = 4.1). Participants of the NonVR group also consisted of 12 males and 8 females, with an age between 23 and 38 years old (N = 20, µ = 27.2, σ = 3.6). Compliance rates of the participants using the MP method were measured through the audio recording files and were found to be 12.5 percent of the people and 19 percent of lists. Compliant only participants were 9 males and 8 females in the VR group (N = 17, µ = 25.5, σ = 4.0), and 12 males and 6 females in the NonVR group (N = 17, µ = 26.8, σ = 3.6). Results for compliant only groups can be seen in figure 7 (B,D), figure 9 (A-D), and figure 10 (B,D).
Data Analysis was done using Microsoft Excel 2016. First, normality tests were done on the word recall accuracy re-sults, which concluded that the data was not normally dis-tributed. The data is highly skewed which suggest a ceiling effect. Therefore, in order to compare two groups of non parametric data, Mann Whitney U tests were performed. Because of the last three lists showing increased recall when compared to the first two lists, as seen in figure 8, Mann Whitney U tests between these two groups were also per-formed. Thus, based on the scoring system explained in the methods, the following results were calculated.
5.1
Fixed-position scoring
5.1.1
All-inclusive Strict-scoring
For All inclusive, fixed position, strict scoring results, a Mann Whitney U test indicated that the recall accuracy of lists of words was not different for the VR group (Mdn 8, N = 100) compared to the NonVR group (Mdn 8, N = 100), U = 5014, N = 200, p = 0.946104. For all inclusive, fixed position, strict scoring results, a Mann Whitney U test indi-cated that the recall accuracy was greater for the last three lists (Mdn 8, N = 120) compared to the first two lists (Mdn 8, N = 80), U = 6286.5, N = 200, p = 0.00021.
Table 1: Age and gender statistics for both VR, NonVR, All-inclusive, and compliant only groups. All-inclusive Compliant Only
N Males Females M age SD age N Males Females M age SD age VR 20 12 8 25.8 4.1 17 9 8 25.5 4.0 NonVR 20 12 8 27.2 3.6 18 12 6 26.8 3.6
Figure 7: Recall accuracy based on both all-inclusive and compliant-only for both (a–b) fixed-position and(c–d) no-position accuracy.
For All inclusive, fixed position, lenient scoring results, a Mann Whitney U test indicated that the recall accuracy of lists of words was not different for the VR group (Mdn 9, N = 100) compared to the NonVR group (Mdn 8, N = 100), U = 5008, N = 200, p = 0.984442. For all inclusive, fixed position, lenient scoring results, a Mann Whitney U test indicated that the recall accuracy was greater for the last three lists (Mdn 8, N = 120) compared to the first two lists (Mdn 8, N = 80), U = 6422.5, N = 200, p = 0.000052.
5.1.3
Compliant-only Strict-scoring
For Compliant only, fixed position, strict scoring results, a Mann Whitney U test indicated that the recall accuracy of lists of words was not different for the VR group (Mdn 9, N = 80) compared to the NonVR group (Mdn 8, N = 82), U = 3243, N = 162, p = 0.901315. For compliant only, fixed position, strict scoring results, a Mann Whitney U test indicated that the recall accuracy was greater for the last three lists (Mdn 9, N = 106) compared to the first two lists (Mdn 8 N = 56), U = 3591.5, N = 162, p = 0.028106.
5.1.4
Compliant-only Lenient-scoring
For compliant only, fixed position, lenient scoring results, a Mann Whitney U test indicated that the recall accuracy of lists of words was not different for the VR group (Mdn 9, N = 80) compared to the NonVR group (Mdn 9, N = 82), U = 3264, N = 162, p = 0.957732. For compliant only, fixed position, lenient scoring results, a Mann Whitney U test indicated that the recall accuracy was greater for the last three lists (Mdn 9, N = 106) compared to the first two lists (Mdn 8, N = 56), U = 3686.5, N = 162, p = 0.011397.
5.2
No-position scoring
5.2.1
All-inclusive Strict-scoring
For all inclusive, no position, strict scoring results, a Mann Whitney U test indicated that the recall accuracy of lists of words was not different for the VR group (Mdn 8, N = 100) compared to the NonVR group (Mdn 8, N = 100), U = 5245, N = 200, p = 0.54944. For all inclusive, no position, strict scoring results, a Mann Whitney U test indicated that the recall accuracy was greater for the last three lists (Mdn 8, N = 120) compared to the first two lists (Mdn 8, N = 80), U = 6016.5, N = 200, p = 0.002416.
Figure 8: Recall accuracy as a function of List Number for the all-inclusive analyses, separately for fixed and no-position accuracy. Accuracy analyses were conducted based on both (a–b) strict-scoring and (c–d) lenient-scoring.
For all inclusive, no position, lenient scoring results, a Mann Whitney U test indicated that the recall accuracy of lists of words was not different for the VR group (Mdn 9, N = 100) compared to the NonVR group (Mdn 9, N = 100), U = 5300.5, N = 200, p = 0.462827. For all inclusive, no position, strict scoring results, a Mann Whitney U test indicated that the recall accuracy was greater for the last three lists (Mdn 9, N = 120) compared to the first two lists (Mdn 9, N = 80), U = 6165.5, N = 200, p = 0.000661.
5.2.3
Compliant-only Strict-scoring
For compliant only, no position, strict scoring results, a Mann Whitney U test indicated that the recall accuracy of lists of words was not different for the VR group (Mdn 9, N = 80) compared to the NonVR group (Mdn 9, N = 82), U = 3375.5, N = 162, p = 0.749044. For compliant only, no position, strict scoring results, a Mann Whitney U test indicated that the recall accuracy was not different for the last three lists (Mdn 9) compared to the first two lists (Mdn 8, N = 56), U = 3408, N = 162, p = 0.121262.
5.2.4
Compliant-only Lenient-scoring
For compliant only, no position, lenient scoring results, a Mann Whitney U test indicated that the recall accuracy of lists of words was not different for the VR group (Mdn 9, N = 80) compared to the NonVR group (Mdn 10, N = 82), U = 3464.5, N = 162, p = 0.536509. For compliant only, no position, lenient scoring results, a Mann Whitney U test
indicated that the recall accuracy was not different for the last three lists (Mdn 10, N = 106) compared to the first two lists (Mdn 9, N = 56), U = 3485, N = 162, p = 0.068652.
6.
DISCUSSION
In this paper we asked one main question: Do young adults have improved memory recall of lists of words when using a Virtual Reality Memory Palace instead of a Virtual Memory Palace? Two subquestions were also asked: Firstly, is there a learning effect noticeable when looking at list number pro-gression? And secondly, how often can the MP environment be used for working memory tasks with lists of words? The results demonstrate that no significant difference could be found in memory recall of lists of words when using a Virtual Reality Memory Palace instead of a Virtual Memory Palace. However, when looking at figure 8 and 9, a small difference in accuracy can be seen in the VR and NonVR groups when looking at recall as list numbers progress. At the first lists, NonVR performs slightly better. At the last lists, VR performs slightly better. This can be seen in all graphs A-D of both figures. However, it is not known if and how this trend continues when list number increases beyond 5. The small difference in memory recall accuracy might be because of the fact that the VR group possibly has a higher cognitive load induced by the use of new technology. This needs time to adjust to or get accustomed to.
Figure 9: Recall accuracy as a function of List Number for the compliant-only analyses, separately for fixed and no-position accuracy. Accuracy analyses were conducted based on both (a–b) strict-scoring and (c–d) lenient-scoring.
A learning effect is noticeable when looking at list number progression, and statistically significant when comparing the first two lists with the last three lists in all but compliant only no position results, as can be seen in figure 8 and 9. This suggests that, when first using the method, either par-ticipants need some practice to use the MP method effi-ciently, or have trouble remembering the specific position of the objects within the environment. Participants might need more than 2 minutes to explore the new environment for it to be used as a MP environment. The previous Vir-tual MP study of Legge et al used 5 minutes [14]. Since the learning effect discontinues after the first two lists, which sums up the total time of the participant in the MP envi-ronment to 5 minutes, this might be the correct time for getting accustomed to a MP environment. However, it has to be noted that most participants felt bored exploring the environment for 2 minutes, and might not have familiarized with the environment a great deal more if this time was in-creased. A certain stimulus or incentive might be needed, such as adding a game element to the exploration phase. For example, adding puzzles to each room by finding and interacting with the objects in a particular sequence can help familiarize with the environment more and make it less boring to explore. Also, walking a certain route within the environment with a narrative active might help familiariz-ing, since it helps alleviate cognitive load by using multiple cognitive channels.
The MP method is excellent for remembering lists of words in order, but is slightly better for remembering lists of words without order. Compliant lists with fixed position had an average memory recall accuracy of 8.14 words, compliant lists without fixed position had an average recall memory accuracy of 8.4 words. This is far greater than the 3-5 chunks of average working memory capacity. This might be because of mnemonics ordering data in more manageable chunks, and binding it to already known (long term) mem-ories. One thing to note is that the data was skewed and a ceiling effect was perceived at 11 words, suggesting that participants could have remembered even more words if the list sizes of the experiment would have been larger. How-ever, the working capacity limit of each participant was not measured. Participants were all known to the researcher, and they may not be an accurate portrayal of the popula-tion of young adults. It would be better if working memory capacity was first measured per participant, so that the dif-ference in working memory capacity when using the MP can be studied.
The Memory Palace method is not usable for everyone. Out of all participants, 12.5 percent were non compliant, and 19 percent of the total lists were non compliant. In previous re-search by Legge et al [14], the percentage of non compliant lists was 60 percent. This is a significant difference, which could be caused by a decrease in cognitive load in this ex-periment by automatically visually binding the words to the
Figure 10: Recall accuracy as a function of Word Position, separately for all-inclusive and compliant-only methods. Accuracy analyses were conducted based on both (a–b) strict-scoring and (c–d) lenient-scoring.
objects, making it easier to apply the method. The cogni-tive load might have been decreased further by encouraging the participant to think out loud and thus using the added auditory cognitive channel of the working memory.
When studying the audio files, it was found that the non compliant participants were either poor at thinking of asso-ciations to link words with objects, or simply did not want to use the MP method. However, most participants got better and faster at creating these associations when list numbers progressed. The skill to quickly create meaningful associa-tions between words and objects might be crucial for this custom MP method to work efficiently, especially since ob-ject and word pairs were randomly chosen and participants only had a short amount of time linking them. This skill some participants had naturally, some had to learn and very little did not possess. Research to the effect of good asso-ciations to improved memory recall with lists of words with the MP method should be researched.
When looking at figure 10, participants recall on average the first and last words of lists best, and the words in the middle of the list worse. This means that the serial position effect occurs when using the MP method with free recall of lists of words. This can be due to the serial position effect being there naturally when remembering list of words. However, participants also expressed having problems remembering the middle room of the environment, which means that the serial position effect might be perceivable when remembering routes or environments [23], which can negatively affect the MP method. With a more familiar and thoroughly known
environment, this may be alleviated. However, some par-ticipants may have cheated by trying to repeat remembered words during the calculation game in the memory phase of this experiment. This may have affected the recall accuracy, especially the recall accuracy of words at the beginning or end of the list, and may be why the serial position effect is noticeable.
Translating the English words from previous research [14] to Dutch with the use of Google Translate might have re-sulted in the words having slightly different orthographical and phonological word properties. This may have affected recall accuracy slightly and may have caused some word lists to be remembered better or worse. Also, the choice of assign-ing random words to random objects might have resulted in certain word-object pairs being easier to associate with each other, possibly affecting the recall accuracy of some word lists and word positions.
Additionally, some participants in the VR group had very little experience with use of a controller to navigate within the environment. The same can be said for the NonVR group with using the keyboard and mouse to navigate in-side the environment. This may have increased cognitive load, and reduced the time the participant saw a particular word-object pair, which could have affected recall accuracy negatively. Also, the HMD used was the Samsung Gear VR, a budget VR headset. A high end HMD could have given different results, by possibly reducing cognitive load.
A Virtual Memory Palace is a good digital mnemonic tool for use when learning lists of words in working memory. Virtual Reality in combination with the MP method does not seem to significantly contribute to improved memory recall, and thus in future work, MP tools could just as well be devel-oped in NonVR, reducing precious time and resources. VR does not seem beneficial for this technique with free recall. With cued recall, allowing participants to use the environ-ment as a cue when recalling information, immersion with VR might affect recall with lists of words positively [12]. With cued recall, a MP could be used as an environment in which the user can return to further amplify retention of information, for example when remembering a password at home. However, for the VMP to be used with long term in-formation such as passwords, future research has to be done looking at the long term memory effects of using a VMP. A VMP environment can perhaps be used multiple times as well for use with long term memory, which can reduce time and resources when creating multiple different ones. For use in applications, automatically generated Memory Palaces could be used. Even with unknown environments, with random words and random objects, most people have the ability to think of usable associations and effectively use the MP method. This can be used to quickly learn all sorts of lists of words. However, custom virtual mem-ory palaces can also be used. Participants in every VMP study could not create their own VMP environment but were forced to use pre-built ones. However, familiar, self built, or deeply known environments are staples in the conventional MP method as described by Yates [26]. Most participants expressed the need to customize the VMP, particularly the ability to choose a specific object per word. They felt that it would benefit memorization and recall. Custom virtual memory palaces, built by its users for specific information, may increase memory recall even more and offer a promising study for future research.
Another promising study for future research on Virtual Mem-ory Palaces is the effect of different fictitious environments compared to the more natural and recognizable environ-ments used in previous research. Especially due to a re-cent study by Coxon et al [5] concluding that less ‘real’ or ‘naturalistic’ environments can lead to more vivid memorial experiences and remember responses. These less real or nat-uralistic environments can perhaps be realised by the use of different kinds of objects.
Some initial research into the effect of the (lack of) objects in the VMP environment has been done alongside this ex-periment. Due to time constraints and lack of participants, this could not be successfully completed. The VMP envi-ronment used for this additional experiment was the same as in the main experiment, but the objects were removed. It seems that participants are less compliant and need more training to use the environment with no objects present. It seems that only the route in an environment from informa-tion chunk to informainforma-tion chunk in itself is not enough to give superior recall with little to no training. Thus it seems from this brief research that (imaginary) objects are highly paramount to the success of the MP method.
The effect of objects in (virtual) MPs can also be studied in
a different way. It is said that in roman times, people used active animated objects with specific sounds as loci within their mental MP [26]. Effect on recall when using objects with an animation and/or sounds might be promising fu-ture work. This is especially useful because of the fact that the working memory has multiple channels, and the audi-tory channel can be used more with added sounds. Active objects could also portray the information associated with it better, making it easier for users to create a link. This is in particular useful when using a MP for events, as events may be ”played out” by the objects. However, the effect of remembering other types of information, like events, with a VMP has not been studied. If effective, this may provide a good learning method for chronological events or certain (biological) mechanisms.
7.
CONCLUSION
In sum, a Virtual Memory Palace is highly effective when using it to learn lists of words, giving a high recall accuracy and a high compliance rate of the Memory Palace method. However, improved immersion through Virtual Reality does not give improved memory recall when compared to the Vir-tual Memory Palace. It seems that spatial memory plays only a small role in the MP method. And that associations between objects and information chunks in the VMP envi-ronment are far more important to the effectiveness of the MP method. Also, it is essential for the user to fully remem-ber the VMP environment used, including (the position of) all objects. It helps if the user is already familiar with the environment, otherwise the user must become familiar with it before using the MP method. Another conclusion is that a learning effect of the MP method is perceivable, suggest-ing that most participants need trainsuggest-ing to use the method effectively, as compliance rate and recall accuracy rose when the method was used more often as list numbers progressed. Also, Virtual Memory Palace environments can be used for at least up to five times, using different lists of words each time, without losing noticeable memory recall accuracy in short term working memory.
8.
ACKNOWLEDGEMENTS
I would like to thank all the participants of the experiment for their time and effort.
9.
REFERENCES
[1] Alan Baddeley. Working memory. Science, 255(5044):556, 1992.
[2] Paul Chandler and John Sweller. Cognitive load theory and the format of instruction. Cognition and instruction, 8(4):293–332, 1991.
[3] Zhijian Chen and Nelson Cowan. Core verbal
working-memory capacity: The limit in words retained without covert articulation. The Quarterly Journal of Experimental Psychology, 62(7):1420–1429, 2009. [4] Nelson Cowan. Working Memory Capacity: Classic
Edition. Psychology Press, 2016.
[5] Matthew Coxon and Katerina Mania. Measuring memories for objects and their locations in immersive virtual environments: The subjective component of memorial experience. In Handbook of Human Centric Visualization, pages 453–471. Springer, 2014.
[6] Fergus IM Craik and Michael J Watkins. The role of rehearsal in short-term memory. Journal of verbal learning and verbal behavior, 12(6):599–607, 1973.
[7] Douwe Draaisma. Metaphors of memory: A history of ideas about the mind. Cambridge University Press, 2000. [8] E Fassbender and W Heiden. The virtual memory palace.
Journal of Computational Information Systems, 2(1):457–464, 2006.
[9] Joshua Foer. Moonwalking with Einstein: The Art and Science of Remembering Everything. Penguin Press, 2011. [10] Murray Glanzer. Encoding in the perceptual (visual) serial
position effect. Journal of Verbal Learning and Verbal Behavior, 5(1):92–97, 1966.
[11] Graeme S Halford, Nelson Cowan, and Glenda Andrews. Separating cognitive capacity from knowledge: A new hypothesis. Trends in cognitive sciences, 11(6):236–242, 2007.
[12] Eric Krokos and Amitabh Varshney. Virtual memory palaces: Immersion aids recall.
[13] Dionysius Lardner. The Museum of Science and Art, volume 5-6. Walton and Maberly, 1855.
[14] Eric LG Legge, Christopher R Madan, Enoch T Ng, and Jeremy B Caplan. Building a memory palace in minutes: Equivalent memory performance using virtual versus conventional environments with the method of loci. Acta psychologica, 141(3):380–390, 2012.
[15] Christopher R Madan, Mackenzie G Glaholt, and Jeremy B Caplan. The influence of item properties on
association-memory. Journal of Memory and Language, 63(1):46–63, 2010.
[16] Eleanor A Maguire, Elizabeth R Valentine, John M Wilding, and Narinder Kapur. Routes to remembering: the brains behind superior memory. Nature neuroscience, 6(1):90–95, 2003.
[17] Katerina Mania and Alan Chalmers. The effects of levels of immersion on memory and presence in virtual
environments: A reality centered approach. CyberPsychology & Behavior, 4(2):247–264, 2001. [18] Katerina Mania, Tom Troscianko, Rycharde Hawkes, and
Alan Chalmers. Fidelity metrics for virtual environment simulations based on spatial memory awareness states. Presence: Teleoperators and Virtual Environments, 12(3):296–310, 2003.
[19] Richard E Mayer. The Cambridge handbook of multimedia learning. Cambridge University Press.
[20] Richard E Mayer and Roxana Moreno. Nine ways to reduce cognitive load in multimedia learning. Educational
psychologist, 38(1):43–52, 2003.
[21] Roxana Moreno and Richard E Mayer. Learning science in virtual reality multimedia environments: Role of methods and media. Journal of educational psychology, 94(3):598, 2002.
[22] Bennet B Murdock Jr. The serial position effect of free recall. Journal of experimental psychology, 64(5):482, 1962. [23] Ian Neath. Contextual and distinctive processes and the
serial position function. Journal of Memory and Language, 32(6):820, Dec 01 1993. Last updated - 2013-02-23. [24] John Sweller. Cognitive load theory, learning difficulty, and
instructional design. Learning and instruction, 4(4):295–312, 1994.
[25] Maurits WML Van den Noort, Peggy Bosch, and Kenneth Hugdahl. Foreign language proficiency and working memory capacity. European Psychologist, 11(4):289–296, 2006. [26] Frances Amelia Yates. The art of memory, volume 64.
Random House, 1992.
10.
APPENDIX
10.1
Unity3D Objects
All Objects used within the Unity3D environment were taken from the unity3d assetstore13.
10.1.1
Test Environment
The barrel is from 10 Low-poly medieval models by Bradley Heszelgraves14. The bookshelf is from Worn Bookshelf by Jason Wong15. The snack machine is from Snack Machines
by VIS Games16. The pallet jack is from Industrial Objects Pack by Arkham Interactive17. And the lamp is from Retro Lamps v.1 by Artur G.18.
10.1.2
Main Environment
Room 1 has three objects. The first object consists of a sofa from Furni - Modern Sofa 01 Starter by Skipper Research & Development19, a medieval carpet from 10 Low-poly
me-dieval models by Bradley Heszelgraves20, and a plant from Small Plants by Keilbaum21. The second object consists of a TV from Flatscreen TV by Rutger Klunder22and TV
fur-niture from TV Furfur-niture by Enozone23. The third object is a piano from Piano by Miroslav Uhlir24.
Hallway 1 has one object. This object consists of the paint-ing Korenveld met kraaien by Van Gogh25and a frame from Classic Picture Frame by Vertex Studio26.
Room 2 has three objects. The first object consists of a chest from Classic Treasure Box by MobileDesignLabo27,
and a carpet from Round Carpet by Olof Hagelin28. The
second object is a bed from Bed collection by Gooseman’s
13assetstore.unity3d.com 14 https://www.assetstore.unity3d.com/en/#!/content/ 30639 15 https://www.assetstore.unity3d.com/en/#!/content/ 8458 16 https://www.assetstore.unity3d.com/en/#!/content/ 3517 17 https://www.assetstore.unity3d.com/en/#!/content/ 10996 18 https://www.assetstore.unity3d.com/en/#!/content/ 19601 19 https://www.assetstore.unity3d.com/en/#!/content/ 39616 20 https://www.assetstore.unity3d.com/en/#!/content/ 30639 21 https://www.assetstore.unity3d.com/en/#!/content/ 6930 22 https://www.assetstore.unity3d.com/en/#!/content/ 9721 23 https://www.assetstore.unity3d.com/en/#!/content/ 60122 24 https://www.assetstore.unity3d.com/en/#!/content/ 154 25https://www.vangoghmuseum.nl/nl/collectie/ s0149V1962 26https://www.assetstore.unity3d.com/en/#!/content/ 59038 27https://www.assetstore.unity3d.com/en/#!/content/ 8952 28https://www.assetstore.unity3d.com/en/#!/content/ 15171
Graphics29. The third object is a wooden wardrobe from 10
Low-poly medieval models by Bradley Heszelgraves30 Hallway 2 has one object. This object consists of the paint-ing The Battle of Alexander at Issus by Albrecht Altdorfer31, which was Napoleon Bonaparte’s favorite painting.
Room 3 has three objects. The first object is a customisable filing cabinet from Old Office Props Free by Jake Sullivan32. The second object consists of crates and shelves from In-dustrial Objects Pack by Arkham Interactive33. The third
object consists of a table, chair, and PC from Props for the Classroom by VR34.
10.2
MP Instruction Text
Bij de Memory Palace methode worden dingen onthouden door gebruik te maken van afbeeldingen, locaties en routes. Als we iets willen onthouden volgens deze methode, bijvoor-beeld een woord, dan moeten we eerst dat woord voorstellen (als een afbeelding) en dan plaatsen in een bepaalde locatie. Willen we een lijst met objecten onthouden, dan moeten we een route uitstippelen tussen deze locaties. Een pad door de lijst als het ware.
De eenvoudigste manier om dit te doen zou zijn om een vertrouwde omgeving te gebruiken en de afbeeldingen/woorden erin te plaatsen. Vervolgens kunnen de woorden later herin-nerd worden door je, in je hoofd, deze omgeving voor te stellen en om er doorheen te lopen, langs de woorden, waar-door het mogelijk is zelfs de volgorde van een lijst objecten te herinneren.
Je wordt zometeen in de eerder verkende omgeving geplaatst. Deze omgeving heeft objecten in een bepaalde route. Er verschijnt om de beurt even een woord op een object op de route. Probeer deze te onthouden door het woord ook echt aan het object (en de route) te koppelen. Dit kan bijvoor-beeld door er zelf een ezelsbruggetje mee te maken.
10.3
Images of the main VMP Environment
29 https://www.assetstore.unity3d.com/en/#!/content/ 25256 30 https://www.assetstore.unity3d.com/en/#!/content/ 30639 31https://commons.wikimedia.org/wiki/File:Albrecht_ Altdorfer,_The_Battle_of_Alexander_at_Issus.jpg 32https://www.assetstore.unity3d.com/en/#!/content/ 53735 33https://www.assetstore.unity3d.com/en/#!/content/ 10996 34https://www.assetstore.unity3d.com/en/#!/content/ 5977
Figure 11: The main virtual memory palace environment used in the experiment. The whole environment consists of three rooms and 2 hallways, each filled with objects. Participants could navigate freely through this environment.
Figure 12: The main VMP environment used in the experiment. The whole environment consists of three rooms and 2 hallways, each filled with objects. Participants could navigate freely through this environment. 11 total objects were placed within the environment. The number of the object represents the order of the object within the environment. This corresponds with word position of the word lists. One environment contained one list of 11 words, sequentially shown on the 11 objects.
