Impact of musical tempo and mode on mood, arousal, and performance in a virtual reality game: A preliminary study with healthy subjects
Olivier J. Teerling
Faviola Brugger-Dadis – Tilo Hartman – Ashley de Burgoyne Universiteit van Amsterdam / VU / NeuroReality
2 Abstract
In the Netherlands alone, stroke currently accounts for at least 320,000 patients. Stroke can have a major impact on cognition and thus efficient and diverse cognitive rehabilitation is required. Virtual reality (VR) has been proposed as a valuable tool in stroke rehabilitation due to its novelty and high entertainment value. One of the most common usages of VR is videogaming. Music has been proven to enhance the enjoyment of videogaming. So, in order to create an immersive and multi-modal VR-game design, the current research investigated the effects of different musical tempi and modes of Mozart’s Sonata 433 in D on the experience of a VR selective attention game. Mood and arousal scores, measured with the POMS and Affect Grid, and the in-game performance scores were collected. Results show that playing the game didn’t affect the in-game performance scores. Neither was there an interaction effect of music and condition. However music did lead to a general higher arousal and improved mood over all conditions. This indicates that music can have a motivational role in VR-game play and that VR-gameplay in general raises mood and arousal. VR-gaming is being developed in order to create more entertaining rehabilitation methods for stroke patients. This study shows that the incorporation of music in VR-gaming can contribute to the motivation to play these games and therefore to the potential success of this new state-of-the art design to ameliorate stroke rehabilitation.
3 Impact of musical tempo and mode on mood, arousal, and performance on a virtual reality
game: A preliminary study with healthy subjects
Worldwide, stroke is recognized as the second most common cause of death (Baylan, Swann-Price, Peryer, & Quinn, 2016). Stroke rates in the Western Europe population aged 65+ are expected to increase from 20% in 2000 to 35% in 2050 (Truelsen et al., 2006). Accessible and easy to use rehabilitation programs are essential for a good recovery (Ada, Dorsch, & Canning, 2006; Langhorne, Bernhardt, & Kwakkel, 2011; Minyoung et al., 2016). VR headsets (head mounted displays; HMDs) are new, easy to use devices which have fairly recently been introduced in the gaming- but also in the medical industry. Consequently, VR has been studied as a possible tool to bridge the gap between professional stroke rehabilitation and qualitative rehabilitation at home (Kong et al., 2016; Lee et al., 2016; Yates et al., 2016).
Following occupational therapy, which entails therapy in the home environment, for at least one year post-stroke, leads to better recovery in the completion of day-to-day activities, like event planning or grocery shopping (Langhorne et al., 2011; Walker et al., 2004). However, stroke-survivors often don’t succeed in finishing a proposed rehabilitation program because it is perceived as boring or ineffective (Jaume-i-Capó, A., Moyà-Alcover, B., Varona, J., Martinez-Bueso, P., & Chiong, A. M., 2013; Jaume-i-Capó, A., Moyà-Alcover, B., & Varona, J., 2014). One of the pillars of (commercial) VR-gaming is that it is entertaining, which enhances the motivation to use the system more frequently and for a longer duration (Yates, Kelemen, & Sik Lanyi, 2016a). This makes it an effective tool for occupational therapy (Lee et al., 2016). The current project investigates this new state-of-the-art approach of stroke rehabilitation. VR-based rehabilitation can be used both in collaboration with stroke recovery units as on an individual level for long-term recovery at home. The user-friendly design of VR devices, the possibility to get direct rewarding feedback and the increase in mood and arousal
4 are among the benefits stroke-survivors report when using VR (Lee et al., 2016). In order to be successful with VR, the simplicity to handle the device and the difficulty of the games should be tailored to the patient’s situation (Minyoung et al., 2016). The broader project behind the current study uses VR-games designed to train several cognitive domains. The games have a range of difficulties, making sure users don’t lose interest when the games are too difficult or too easy. In general, these games follow the idea of Flow theory (Csikszentmihalyi, 1990), where there is a fine balance between the skill of the user against the difficulty of the game. The current study specifically uses the selective attention game. Except for occasional headache or dizziness, no major downsides have been reported when using VR, making it safe for use with stroke survivors (Yates, Kelemen, & Sik Lanyi, 2016b). In order to enhance the motivation to follow this form of therapy, the current study investigates a single, but important part of the multisensory VR-game, namely the auditory part. As multi-sensory stimulation has been proven effective for cognition and cognitive recovery, introducing music as a form of auditory stimulation to the game could enhance the immersive effect of VR-game play. This can hold great potential for improvement of mood, arousal and cognitive performance of the user when playing the VR game (Baka, Kentros, Papagiannakis, & Magnenat-Thalmann, 2018).
Listening to music has many positive effects on both the brain and general wellbeing. It appears to reduce depression (Aalbers et al., 2017; Talwar et al., 2006), enhance cognitive abilities such as focused and visual attention (Baylan et al., 2016; Sarkamo et al., 2008; Schellenberg, 2005) and enhance general mood and arousal (in stroke-survivors) (Husain, Thompson, & Schellenberg, 2002; Thompson, Schellenberg, & Husain, 2001). In a long-lasting Finnish study, 75% of the stroke-survivors reported that listening to music during their recovery improved their mood, depressive symptoms and poor concentration (Forsblom, Laitinen, Särkämö, & Tervaniemi, 2009).
5 Within music one of the most studied genres is classical music (Aalbers et al., 2017; Thompson et al., 2001; Tsai et al., 2013). Music listening is an extremely personal experience and therefore even more interesting to study. The theory on which classical music studies are based is that the effects of different properties of music, like: tempo, pitch and mode are universal and can be generalized over genre. Continuing on the study done by Husain (2002), the current study looks at the effects of tempo and mode on participant’s cognition, mood and arousal while playing a selective attention task in a Virtual Reality environment. The theory behind this approach is that music has an indirect effect on cognition, through increased arousal and improved mood. This is known as the Arousal-Mood Hypothesis, shown in Figure 1 below (Husain et al., 2002; Thompson et al., 2001; Tsai et al., 2013). In particular, tempo affects arousal while mode has an effect on mood. Specifically, higher tempo leads to higher arousal which leads to improved cognitive performance in an inverted U-shape, meaning that medium arousal leads to better cognitive performance than maximum and minimum arousal. Music in a major mode leads to improvements in mood which leads to better cognitive performance (Husain et al., 2002). Next to these musical properties it seems that complexity of a song negatively affects cognitive performance (Balkwill, Thompson, & Matsunaga, 2004) and familiarity of a song positively affects cognitive performance (Chmiel & Schubert, 2017; Pereira et al., 2011) . In conclusion, the literature so far states that (familiar) fast-paced and technically easy music pieces in a major mode lead to higher enjoyment and positively affects cognitive performance, while (unfamiliar) slow and technically complex music pieces in a minor mode lead to less enjoyment and even anger, which negatively affects cognitive performance (Balkwill, Thompson, & Matsunaga, 2004).
6 Figure 1. Arousal and mood hypothesis. A diagram that represents how song properties have an indirect effect on cognitive performance. Tempo and mode influence cognitive performance through the domains of arousal, mood and enjoyment. (Husain et al., 2002).
This study will continue on the study of Husain (2002) and Thompson (2001) where the authors showed an improvement in focused attention using the first movement of Mozart Sonata K 448 with different tempi and modes. This study however, combines VR and music to create a multi-sensory cognitive training program. As this design is eventually meant for stroke rehabilitation, finding specific effects of music on mood, arousal and/or selective attention can have great implications for the usage of music and VR for stroke rehabilitation. So primarily this study will try to replicate the findings of Husain (2002) on mood and arousal. It is expected that tempo influences arousal and mode influences mood. Next, this study will see if the effects of listening to music on mood and arousal parallel the effects of listening to music on selective attention, measured with accuracy and reaction time of the participant in the current VR-game design. It is predicted that a higher tempo leads to a higher arousal and a major mode leads to a more positive mood (Husain et al., 2002). It is expected that listening to an up-tempo version of Mozart’s Sonata K448 in a major mode will improve mood and increase arousal which in
7 turn will increase accuracy and reduce reaction time. A down-tempo version of Mozart’s Sonata K448 in a minor mode is expected to decrease mood, reduce arousal and in turn have no effect on selective attention.
Finding effects of music on mood, arousal and cognition can hold great potential for the application of music in any kind of rehabilitation form, including VR stroke-rehabilitation as tested in the current design. This study can also further strengthen the research on which aspects of music actually affect health and cognition.
Method Participants
A total of 49 participants (71% female, 94% university degree), recruited mostly around the universities in Amsterdam, from 18 to 57 years old (M = 24.2 , SD = 5.43) took part in this study. Participants were either English speaking or Dutch speaking. Because of the time limit of this study and the number of participants needed, the eventual participant pool is considered a convenient sample. Participants did the study voluntarily with two gift-cards to be distributed at the end of the study. None of the participants were excluded in the data analysis. Most students followed a Science, Technology, Engineering or Mathematics (STEM) subject (94%) and had no prior experience with VR (82%).
Intervention
The VR game was developed by Neuroreality. The system made use of a Google Daydream headset and the participants were seated. To minimize physical effort which is important for stroke-patients who might suffer from physical inabilities, only head movements were needed to play this game. For the current study a Samsung Galaxy S10 was inserted in the Google Daydream. The music was presented through over-ear headphones, maximizing full emergence of the participant in the music and the game.
8 For the current study participants had to play a selective attention game. The selective attention game is a combination of two different selective attention paradigms. One type is the Speed of Processing (SOP training), which has shown to improve cognitive functioning (Ball, Edwards, & Ross, 2007; Vance et al., 2007). The other type is the Cancellation Attention test (CatT) (Bates & Lemay, 2004). The game consisted of three phases. In phase 1 the participants needed to find a target that corresponded in color with the given example. In phase 2, the participants needed to find a target that corresponded in shape with the given example. In phase 3, the participants needed to find a target that corresponded both in color and shape with the given example. The difficulty of the game was in finding the correct target amongst distractors and finding it within the designated time. The targets dis- and reappeared, making it more challenging for the participants. The amount of distractors defined the difficulty of the game and for the current study this was assessed doing a pilot study (unpublished data).
Materials
The music used in this study is a modification of the first movement of Mozart’s Sonata K. 448. The piece was played by a skilled pianist and the tempo and mode modifications were artificially done by a MIDI-controller. The exact same music was used in the study of Husain (2002). The music was kindly provided to me by Glenn Schellenberg. For this study only the fast (165 BPM), D-major and slow (60 BPM), D-minor versions were used as those versions proved to have the highest effect on mood, arousal and cognition (Husain et al., 2002). The game lasted 9 minutes. Therefore, the fast version, which lasted 4 minutes and 48 seconds, was played twice in almost it’s entirety. The slow version was cut off at 4 min 48 and also repeated. Therefore the participants in the slow version never heard the full song. However, previous studies showed that the perception of a song is different when hearing it for the second time (Iwanaga, Ikeda, & Iwaki, 1996, newer ref). A fade in and fade out was used to begin and end the songs. First, the participants listened to the music without playing the game and then the
9 music was also played while playing the game. It was made sure that the quality and volume of the music were similar in both phases. During the game, there were game-related sounds next to the music. These sounds were used to indicate participants if they had the right answer.
Two questionnaires were used that scored for Mood and Arousal. The Profile of Mood States (POMS) consisted of 32 adjectives (Dutch version; (Ark, Marburger, & Mellenbergh, n.d.)) or 24 adjectives (English version; (Terry, Lane, Lane, & Keohane, 1999; Yeun & Shin-Park, 2006)). The adjectives described personal states or emotions and participants had to score how much they identified with that emotion on a 5-point Likert scale ranging from 0 (strongly disagree) to 4 (strongly agree). The POMS took approximately five minutes to administer. To follow the same procedure as Husain’s (2002) article and to validate the scores on the POMS, participants also filled in the Affect Grid (AG). The AG is an alternative, single item scale used to score for Mood and Arousal (Russell, Weiss, & Mendelsohn, 1989). This grid follows the same rules as a 9-point Likert scale. Mood is scored on the horizontal axis ranging from 1(depressed) to 9 (frantic excitement). Arousal is scored on the vertical axis ranging from 1 (tired) to 9 (excited). Internal consistency can not be assessed with a single-item scale, but for a reliability study, see Russell & Mendelsohn, 1989.
Procedure
The participants were given an informed consent form prior to testing. Participants were randomly divided in three groups by a random number generator. The participants got acquainted with the VR-headset, the headphones and were shown how to use the VR game by a desktop tutorial. Two groups listened to either the fast-major version or the slow minor version of Mozart’s Sonata. The control group listened to silence. This listening experience was around 5 minutes for all groups. Participants were encouraged to close their eyes in order to elicit a more emerged listening experience. Next, the participants filled in the POMS, either English or Dutch, to score for mood and arousal prior to exposure to the VR game (Ark, Marburger, &
10 Mellenbergh, 1995). The answers on a subset of questions that predicted mood and arousal were used in the final analysis. At the same time, participants also filled in the AG. Participants then played the game which was backtracked with the same version of Mozart’s Sonata or silence. They played the game for approximately 9 minutes. After finishing the game, they filled in the POMS and AG again and got a debriefing asking for the familiarity and experienced pleasure of the sound.
Data analysis – Questionnaire liability
As questionnaires with a different number of questions were used in two languages, the internal consistency of these questionnaires was checked using the current data. The POMS made use of subscales with a different amount of questions per subscale in Dutch compared to English. The questions of the subscales eventually loaded on the same factor. Cronbach’s alpha scores, which represents the liability scores were calculated. In the next line the Cronbach alpha’s scores are presented for the pre- and post mood & arousal scores, respectively. The Dutch depression (mood) subscale consisted of 8 items (α = .71, α = .58). The English depression subscale consisted of 4 items (α = .91, α = .85). The Dutch vigor (arousal) subscale consisted of 5 items (α = .82, α = .89). The English vigor subscale consisted of 4 items (α = .73, α = .68). This shows moderate to excellent internal consistency of all questionnaires.
Statistical analysis
To test whether listening to music with different musical features had an effect on selective attention, both the accuracy and the reaction time of the gameplay were measured and compared between groups. Mood and arousal of the participants were measured by comparing the difference score on the POMS questionnaire and AG between groups. The pre-score was subtracted from the post-score. A positive score in the AG illustrated a rise in Arousal and Mood. A positive score in the POMS reflected a rise of arousal, but a decrease of mood. This was taken into account in the analysis. Next to this, the participants filled in a questionnaire to
11 see if they particularly liked the music and if they were familiar with the song. In this questionnaire, general questions about the game were asked to see if there were any unclarities in the design. Covariates that were tested to rule out any interference with the data were language, gender, age and ethnicity.
To test the hypotheses, an experimental 3 (fast-major vs. slow-minor vs. silence) x 2 (pre- vs. post-manipulation) between-subjects design was applied. A mixed ANOVA design was used to asses the effects of music on mood and arousal using the scores of the POMS and the AG. Preliminary differences between groups were ruled out. The effects of music listening on in-game performance was tested with a mixed ANOVA design. Subsequently, covariates were tested to rule out confounding factors. As a questionnaire was used with different factors predicting the same variable, Cronbach’s alpha was calculated to determine the internal validity of the questions. Besides, Mahalanobis distance calculation was used to find outliers, but no outliers were detected.
Results
Mood and arousal scores were collected with the POMS and AG. One-way ANOVA’s ruled out that there were significant pre-existing differences for mood and arousal between the three groups (data not shown). A normality test, examining the Shapiro-Wilks test and standardized skewedness, showed a normal distribution of all below mentioned data. Subsequently, the assumption of homogeneity was met for all below data, tested with the Levene’s test. An alpha level of .05 was used for all below mentioned analyses.
Reaction time & Accuracy
To test for the hypothesis that listening to music affects your cognitive performance, a repeated measures design was used. Reaction time and accuracy of the selective attention task were measured across all three conditions. No main effect of condition (fast-major vs.
slow-12 minor vs. silence) was found for accuracy, F(2, 46) = 1.32, p > .28. Besides, no main effect of condition was found for reaction time, F(2, 46) = .16, p > .85 (see Table 1).
Table 1
Mean accuracy and reaction time of the three conditions (Fast-major, Slow-minor and Silence) measured while playing the selective attention game.
Scores Accuracy Reaction time
Mean (SD) Mean (SD) n Fast-Maj 0.93 (0.03) 3.60 (0.27) 17 Slow-Min 0.91 (0.04) 3.64 (0.34) 17 Silence 0.92 (0.04) 3.57 (0.35) 15
Note: n = number of participants; SD = standard deviation.
Mood & Arousal
To test for the hypothesis that listening to music affects mood and arousal, difference scores (post-manipulation minus pre-manipulation) were calculated for the POMS and AG. These differences scores are shown in table 2. For a color-coded grid visualization of the AG results, see Appendix 1.
13 Table 2
Mean difference score (post minus pre) of mood and arousal measured with the POMS and AG of the three conditions (Fast-major, Slow-minor and Silence)
Difference scores POMS AG Mood M(SD) Arousal M(SD) Mood M(SD) Arousal M(SD) n Fast-Maj 1.5(2.1) 0.3(3.8) 0.4(2.2) 0.8(1.7) 17 Slow-Min 1.2(1.5) -0.5(2.6) -0.3(1.4) 0.6(2.0) 17 Silence 0.8(1.4) -0.9(4.1) 0.3(0.9) 0.3(1.9) 15
Note: n = number of participants; M = mean; SD = standard deviation. A positive number on the mood scale represents a decrease in mood, while a positive score on the POMS-arousal scale represents an increase in POMS-arousal. Positive scores on the AG- mood & POMS-arousal score represents an increase in mood and arousal.
Previous research found that tempo positively affected arousal and mode positively affected mood. It was therefore predicted that a higher tempo would lead to higher arousal levels and a major mode would lead to an improved mood. However, the current study found that the POMS and AG not always predicted arousal and mood in the same way, as can be seen in Table 2. Mood increased for the fast-major condition in both the POMS and AG. However, for the slow-minor version, mood increased in the POMS and decreased in the AG. Besides, while the POMS revealed a decrease in arousal in the slow-minor condition, the AG revealed an increase. This unalignment of POMS and AG scores are in contrast to earlier studies that found a coherent effect of tempo and mode on mood and arousal using the same questionnaires (Husain et al., 2002).
14 Mood
In order to test if the effects of the manipulation on the participants changes in mood were significant, a mixed-ANOVA design was used. The means for the difference scores were used as a dependent variable. The three conditions were used as independent variables. There was a main effect of the manipulation on mood-POMS score, F(1, 46) = 16,3, p < 0.001. This main effect of manipulation was not confirmed by the main mood-AG results, F(1, 46) = 0.78, p > 0.63. There was no interaction effect of condition and mood-POMS score, F(2, 46) = 0.91, p > 0.41. Nor was there an interaction effect of AG-mood and condition, F(2, 46) = 0.78, p > 0.46. Figure 2 shows the mean pre- and post-manipulation POMS-mood scores per condition. A significant lower mood score was found post-manipulation compared to pre-manipulation for all conditions. This indicates an improvement in mood, but without there being an effect of condition.
Figure 2. A bar graph of the mean pre- and post-manipulation POMS-mood scores of the three different conditions.
Arousal
Examining arousal, no main effect of the manipulation on the arousal-POMS score was found, F(1, 46) = 1,27, p > 0.27. There was, however, a main effect of the manipulation on the
0 0,5 1 1,5 2 2,5 3 3,5 4
Fast-major Slow-minor Silence
Score
Condition
15 AG score, F(1,46) = 4,74, p < 0.05. No interaction effect of condition and arousal-POMS score was found, F(2, 46) = 0.39, p > 0.68. Neither was there an interaction effect of the manipulation on the arousal-AG score, F(2, 46) = 0.22, p > 0.80. Figure 3 shows the mean pre- and post-manipulation AG-arousal scores per condition. A significant higher arousal score was found post-manipulation compared to pre-manipulation for all conditions. This indicates an overall rise in arousal, but without there being an effect of condition.
Figure 3. A bar graph of the mean pre- and post-manipulation AG-arousal scores of the three different conditions.
In the next section, subsequent interesting results will be shown. Because of difficulties finding enough participants, the current study used both English- and Dutch speaking students. Both the POMS and the AG exist in a validated English and Dutch version. To rule out that the difference in language led to significant differences in either the POMS or AG score a repeated-measures ANOVA was performed.
Language - Arousal
There was no main effect of the manipulation on POMS-arousal scores in both language groups, F(1,47) = 0.13, p > 0.72. However, there was an interaction effect of language and
0 1 2 3 4 5 6 7
Fast-major Slow-minor Silence
Score
Condition
16 POMS-arousal score, F(1, 47) = 8,64, p < 0.005. Interestingly, the average score of the English participants went up from 8,78 (SD = 3.30) to 9,83 (SD = 2,71), while the average score of the Dutch participants went down from 12,23 (SD = 3,41) to 10,87 (SD = 4,06). Besides, a significant difference in the pre-scores of both language groups was found, F(1,47) = 0,150, p < 0.001. This suggests that the Dutch participants scored higher in general on the POMS questionnaire than English participants and that the difference score between English and Dutch participants is significant.
Figure 4 & 5: A bar graph of the mean pre- and post-manipulation POMS-arousal scores of the two different languages (left). A bar graph showing a significant higher difference score for Dutch compared to English respondents (right).
Looking at arousal, there was a main effect of the manipulation on the AG-arousal scores, F(1,47) = 5,18, p < 0.05. An interaction effect of language and AG-arousal scores was also found, F(1, 47) = 5,03, p < 0.05. There was a significant difference in the arousal score of the AG for English people (M = 6,28, SD = 0.36) and in the arousal score of the AG for Dutch people (M = 5.24, SD = 0.27); F(1, 47) = 5.34, p <0.05. Next, the post-scores of the two language groups were significantly different, F(1,47) = 9,81, p < 0.001. This suggests that
0 2 4 6 8 10 12 14 Pre Post Score Time English Dutch -2,5 -2 -1,5 -1 -0,5 0 0,5 1 1,5 2 Score (Pos t-p re ) Language English Dutch
17 English people had a significant higher score post manipulation than Dutch people and that the manipulation significantly increased the AG-arousal score in the English participants.
Figure 6 & 7: A bar graph of the mean pre- and post-manipulation AG-arousal scores of the two different languages (left). A bar graph showing a significant higher difference score for the English compared to Dutch respondents (right).
Language – mood
There was a significant main effect of language on POMS-mood score, F(1,47) = 22,93, p < 0.001. Besides, there was an interaction effect of language and time on POMS-mood score, F(1, 47) = 5,77, p < 0.020. The pre-score of the English participants was significantly higher than the pre-score of the Dutch participants, F(1,47) = 16,79, p < 0.05. This suggests that the English participants scored higher in general on the POMS-mood questionnaire than Dutch participants and that the difference score in the English group is significantly higher than the difference score in the Dutch group (figure 8).
0 1 2 3 4 5 6 7 8 Pre Post Score Time English Dutch 0 0,2 0,4 0,6 0,8 1 1,2 1,4 1,6 1,8 Score (Pos t-p re ) Language English Dutch
18 Figure 8 & 9: A bar graph of the mean pre- and post-manipulation POMS-mood scores of the two different languages (left). A bar graph showing a significant higher difference score for English compared to Dutch respondents (right).
Looking at the AG-mood scores there was a main effect of language, F(1,47) = 7,48, p < 0.001. This indicates that there was one language that scored significantly higher than the other. Indeed, looking at the averages, the Dutch language scored a general higher score than the English language. However there was no interaction effect F(1,47) = 0.789, p > 0.789. The lack of an interaction effect means nothing can be said about the effect of language on the difference score. 0 1 2 3 4 5 Pre Post Score Time English Dutch 0 0,5 1 1,5 2 2,5 Sc ore (Pos t-p re ) Language English Dutch
19 Figure 10 & 11: A bar graph of the mean pre- and post-manipulation AG-mood scores of the two different languages (left). A bar graph of the mean difference AG-mood scores of the two languages (right).
Discussion
The participants in the current study listened to one of the two versions of the Sonata 433 in D by Mozart or to silence, while playing a selective attention game presented in a VR environment. The goal of the current study was to find out how musical tempo and mode affects mood, arousal and (cognitive) performance while playing a VR selective attention game. The addition of gameplay in this study contributed in the found changes in mood and arousal. The effect of gameplay on mood & arousal scores will be discussed further down. Besides, it was studied if changes in mood and arousal parallel changes in performance. Firstly, the effects on performance will be discussed, followed by the mood & arousal scores. Finally, the effect of using questionnaires in different languages will be discussed.
Performance, measured with reaction time and accuracy, did not differ among participants in either of the three conditions. This is not in line with earlier findings that state that tempo and mode can play a role on cognitive abilities (Husain et al., 2002; Sloboda & Juslin., 2001; Yerkes & Dodson., 1908). However a number of studies also challenge the view
0 1 2 3 4 5 6 7 8 Pre Post Score Time English Dutch -0,8 -0,6 -0,4 -0,2 0 0,2 0,4 0,6 English Dutch
20 that music, or musical tempo and mode have an effect on cognitive abilities (Črnčec, Wilson, & Prior, 2006; McCutcheon, 2000; Steele, Bass, & Crook, 1999; Steele, Bella, et al., 1999). This can be explained by the fact that the current selective attention game is a new selective attention game, specifically designed for the current VR-environment. Besides, the difficulty of the game was moderate to low, leading to a low chance of insecurities and therefore low variability in the accuracy score. The participants only played one trial of the game and as this specific game has not been validated yet, it is unclear if this led to a sufficient amount of data to detect any significant differences in accuracy and reaction time.
Looking at mood and arousal, the current study did find a main effect of time. After playing the game, participants showed an increase in mood, measured with the POMS, and an increase in arousal, measured with the AG. Unfortunately, the significant rise in mood found with the POMS was not found with the AG. Neither was the significant rise in arousal, measured with the AG, found with the POMS. Nevertheless, the found results suggests that the gameplay, possibly in combination with the music, increased mood and raised arousal. This is an effect that is sought for as the virtual world and the game serve to increase motivation to play the game and therefore improve rehabilitation possibilities for stroke survivors. Having increased arousal and improved mood as an effect of the gameplay implicates that this desired effect is reached. Having a rehabilitation method that clearly increases mood and arousal can be a huge improvement to the current available stroke rehabilitation methods.
Further, no significant interaction effect of time and condition was found on mood and arousal in the current study. Where previous studies found that higher and lower tempo led to respectively higher and lower arousal, and a major and minor mode led to respectively a better and worse mood, the current study did not find this effect. One of the reasons for this finding could be that the current study had VR game play as an additional manipulation next to music listening. Obviously, gameplay can have a huge effect on mood and arousal. The feeling of
21 achievement and competition are well known for raising arousal and, depending on the success, effecting mood as well (Kaye & Bryce, 2014; Rieger, Wulf, Kneer, Frischlich, & Bente, 2014). With VR-gameplay these effects can be even stronger as the game is experienced in a way more immersed way than 2-D gameplay. In order to control for this variable, the results of the two music conditions were compared to the silent condition, where the same gameplay occurred but without the music. This however still doesn’t test the effect of the 3D-gameplay by itself. Therefore in future studies, it may be even better to include a condition where the game is played in 2D on a computer-screen, to check for the 3D effect or to let participants do an alternative task that doesn’t elicit a competitive feeling, like reading a text. In this way the effects of 3D-gameplay on mood and arousal can be better tested. This was thought of before starting this research, but because of time constraints and the high number of needed participants, this condition was left out. This can be considered a limitation of the current study. Although the POMS-mood score seems to be higher in the music conditions (see Table 2), this result was not significant. The same counts for the AG-arousal score. The power analysis showed that 66 participants were needed, while only 49 people participated. Increasing the number of participants to the desired amount is one obvious way to make the results more reliable. However, as described above, even with the current number of participants, some interesting results were found.
In the main effects, the rise in mood and arousal was not found in both the questionnaires. The rise in mood was only found with the POMS and the rise in arousal was only found with the AG. This discrepancy in POMS and AG results can possibly be explained by the fact that the questionnaires were presented in two different languages (Dutch and English). The results showed that the way participants filled in the Dutch and English questionnaires differed. In summary, the POMS-arousal score went up for English participants, while it went down for Dutch participants. Besides, the pre-score was higher for the Dutch
22 participants compared to English participants (see figure 4 & 5). Next, the POMS-mood score went down for both English and Dutch students, but the pre- and post-score were higher for English participants (see figure 8 & 9). The mood-AG scores went up for Dutch but not for English participants. Also, the pre- and post mood-AG score for English was lower than Dutch (see figure 10).
A possible explanation for this is that during the experiment participants might have understood the meaning of the adjectives of the POMS differently in the two languages. This claim is strengthened by the fact that the Dutch participants claimed that the Dutch words were ‘old-fashioned’ and asked more often the exact meaning of the word than the English participants. Admitted, the translated Dutch version stems from 1993 (Ark et al., 1993), while the used English version is from 1999 ((Terry et al., 1999). With the AG, it were the Dutch participants who asked more often what the explanation exactly entailed. This was a flaw in the current study. It might have been better to let Dutch participants fill in the English questionnaires. In hindsight this probably wouldn’t have caused too many problems as most participants were Dutch students following a study with English as it’s root language.
However, a cultural difference shouldn’t be ruled out too quickly. Common issues with translated questionnaires are that most questionnaires are translated from English. This gives rise to the issue of cultural hegemony and Anglo-centrism of the questionnaires (Sperber, 2004). Only translating questionnaires is not enough. It needs to be made sure that, especially with the POMS, words elicit the same feeling in the reader in different languages. One can imagine that if the adjectives of the depression subscale sound more negative in one language, that the response in that language is also more tailored to the extremes (either completely disagree or completely agree).
One last possible explanation of the discrepancy of the results between the two languages is the difference in the number of participants per language group. There were 31
23 Dutch participants and 18 English participants. A new power analysis confirmed, 66 participants were needed for this study. A bigger number and a more equal distribution of English and Dutch participants could lead to different results.
For a future study it would be wise to include a control condition that only does music listening and no gameplay. This will give more clarity about the exact effect of gameplay. Games often are supported with typical ‘gameplay’ music. It is interesting to know as well if listening to typical gameplay music would have the same effect on mood, arousal and motivation to play the game. Giving the user the possibility to listen to regular music like Mozart or (self-selected) music from pop and rock genres might enhance motivation more than gameplay music. This could be a very interesting continuation of the current study. Finally, testing the current VR game-design in combination with music in a population of stroke-survivors is a crucial next step in order to develop novel ways of stroke-rehabilitation.
24 References
Aalbers, S., Fusar-Poli, L., Freeman, R. E., Spreen, M., Ket, J. C., Vink, A. C., … Gold, C. (2017). Music therapy for depression. Cochrane Database of Systematic Reviews, (11). https://doi.org/10.1002/14651858.CD004517.pub3
Ada, L., Dorsch, S., & Canning, C. G. (2006). Strengthening interventions increase strength and improve activity after stroke: a systematic review. Australian Journal of Physiotherapy, 52(4), 241–248. https://doi.org/10.1016/S0004-9514(06)70003-4
Ark, L. A. Van Der, Marburger, D., & Mellenbergh, G. J. (n.d.). de AANGEPASTE PROFILE OF MOODS STATES ( Aangepaste POMS ) Handleiding en verantwoording.
Baka, E., Kentros, M., Papagiannakis, G., & Magnenat-Thalmann, N. (2018). Virtual Reality Rehabilitation Based on Neurologic Music Therapy: A Qualitative Preliminary Clinical Study (pp. 113–127). Springer, Cham. https://doi.org/10.1007/978-3-319-91152-6_9 Balkwill, L.-L., Thompson, W. F., & Matsunaga, R. (2004). Recognition of emotion in
Japanese, Western, and Hindustani music by Japanese listeners 1. Japanese Psychological
Research, 46(4), 337–349. https://doi.org/10.1111/j.1468-5584.2004.00265.x
Ball, K., Edwards, J. D., & Ross, L. A. (2007). The Impact of Speed of Processing Training on Cognitive and Everyday Functions. The Journals of Gerontology: Series B, 62(Special_Issue_1), 19–31. https://doi.org/10.1093/geronb/62.special_issue_1.19
Bates, M. E., & Lemay, E. P. (2004). The d2 Test of Attention: Construct validity and extensions in scoring techniques. Journal of the International Neuropsychological Society, 10(03), 392–400. https://doi.org/10.1017/S135561770410307X
Baylan, S., Swann-Price, R., Peryer, G., & Quinn, T. (2016). The effects of music listening interventions on cognition and mood post-stroke: a systematic review. Expert Review of Neurotherapeutics, 16(11), 1241–1249. https://doi.org/10.1080/14737175.2016.1227241
25 Chmiel, A., & Schubert, E. (2017). Back to the inverted-U for music preference: A review of the literature. Psychology of Music, 45(6), 886–909. https://doi.org/10.1177/0305735617697507
Črnčec, R., Wilson, S. J., & Prior, M. (2006). No Evidence for the Mozart Effect in Children. Music Perception, 23(4), 305–318. https://doi.org/10.1525/mp.2006.23.4.305
Forsblom, A., Laitinen, S., Särkämö, T., & Tervaniemi, M. (2009). Therapeutic Role of Music Listening in Stroke Rehabilitation. Annals of the New York Academy of Sciences, 1169(1), 426–430. https://doi.org/10.1111/j.1749-6632.2009.04776.x
Husain, G., Thompson, W. F., & Schellenberg, E. G. (2002). Effects of Musical Tempo and Mode on Arousal, Mood, and Spatial Abilities. Music Perception, 20(2), 151–171. https://doi.org/10.1525/mp.2002.20.2.151
Iwanaga, M., Ikeda, M., & Iwaki, T. (1996). The Effects of Repetitive Exposure to Music on Subjective and Physiological Responses. Journal of Music Therapy, 33(3), 219–230. https://doi.org/10.1093/jmt/33.3.219
Kaye, L. K., & Bryce, J. (2014). Go with the flow: The experience and affective outcomes of solo versus social gameplay. Journal of Gaming & Virtual Worlds, 6(1), 49–60. https://doi.org/10.1386/jgvw.6.1.49_1
Langhorne, P., Bernhardt, J., & Kwakkel, G. (2011). Stroke rehabilitation. The Lancet, 377(9778), 1693–1702. https://doi.org/10.1016/S0140-6736(11)60325-5
Lee, M., Pyun, S.-B., Chung, J., Kim, J., Eun, S.-D., & Yoon, B. (2016). A Further Step to Develop Patient-Friendly Implementation Strategies for Virtual Reality-Based Rehabilitation in Patients With Acute Stroke. Physical Therapy, 96(10), 1554–1564. https://doi.org/10.2522/ptj.20150271
McCutcheon, L. E. (2000). Another Failure to Generalize the Mozart Effect. Psychological Reports, 87(1), 325–330. https://doi.org/10.2466/pr0.2000.87.1.325
26 Minyoung, L., Sung-Bom, P., Jinjoo, C., Jungjin, K., Seon-Deok, E., & BumChul, Y. (2016). A Further Step to Develop Patient-Friendly Implementation Strategies for Virtual Reality-Based Rehabilitation in Patients With Acute Stroke. Physical Therapy. https://doi.org/10.2522/ptj.20150271
Pereira, C. S., Teixeira, J., Figueiredo, P., Xavier, J., Castro, S. L., & Brattico, E. (2011). Music and emotions in the brain: familiarity matters. PloS One, 6(11), e27241. https://doi.org/10.1371/journal.pone.0027241
Rieger, D., Wulf, T., Kneer, J., Frischlich, L., & Bente, G. (2014). The winner takes it all: The effect of in-game success and need satisfaction on mood repair and enjoyment. Computers in Human Behavior, 39, 281–286. https://doi.org/10.1016/J.CHB.2014.07.037
Russell, J. A., Weiss, A., & Mendelsohn, G. A. (1989). Affect Grid: A Single-Item Scale of Pleasure and Arousal, 57(3), 493–502. https://doi.org/10.1037/0022-3514.57.3.493 Sarkamo, T., Tervaniemi, M., Laitinen, S., Forsblom, A., Soinila, S., Mikkonen, M., …
Hietanen, M. (2008). Music listening enhances cognitive recovery and mood after middle cerebral artery stroke. Brain, 131(3), 866–876. https://doi.org/10.1093/brain/awn013 Schellenberg, E. G. (2005). Music and Cognitive Abilities. Current Directions in Psychological
Science, 14(6), 317–320. https://doi.org/10.1111/j.0963-7214.2005.00389.x
Sperber, A. D. (2004). Translation and validation of study instruments for cross-cultural
research. Gastroenterology, 126, S124–S128.
https://doi.org/10.1053/J.GASTRO.2003.10.016
Steele, K. M., Bass, K. E., & Crook, M. D. (1999). The Mystery of the Mozart Effect: Failure to Replicate. Psychological Science, 10(4), 366–369. https://doi.org/10.1111/1467-9280.00169
Steele, K. M., Bella, S. D., Peretz, I., Dunlop, T., Dawe, L. A., Humphrey, G. K., … Olmstead, C. G. (1999). Prelude or requiem for the ‘Mozart effect’? Nature, 400(6747), 827–827.
27 https://doi.org/10.1038/23611
Talwar, N., Crawford, M. J., Maratos, A., Nur, U., McDermott, O., & Procter, S. (2006). Music therapy for in-patients with schizophrenia. British Journal of Psychiatry, 189(05), 405– 409. https://doi.org/10.1192/bjp.bp.105.015073
Terry, P. C., Lane, A. M., Lane, H. J., & Keohane, L. (1999). Development and validation of a mood measure for adolescents. Journal of Sports Sciences, 17(11), 861–872. https://doi.org/10.1080/026404199365425
Thompson, W. F., Schellenberg, E. G., & Husain, G. (2001). Arousal, Mood, and The Mozart Effect. Psychological Science, 12(3), 248–251. https://doi.org/10.1111/1467-9280.00345 Truelsen, T., Piechowski-Jozwiak, B., Bonita, R., Mathers, C., Bogousslavsky, J., & Boysen,
G. (2006). Stroke incidence and prevalence in Europe: a review of available data. European Journal of Neurology, 13(6), 581–598. https://doi.org/10.1111/j.1468-1331.2006.01138.x
Tsai, P.-L., Chen, M.-C., Huang, Y.-T., Lin, K.-C., Chen, K.-L., & Hsu, Y.-W. (2013). Listening to Classical Music Ameliorates Unilateral Neglect After Stroke. American
Journal of Occupational Therapy, 67(3), 328–335.
https://doi.org/10.5014/ajot.2013.006312
Vance, D., Dawson, J., Wadley, V., Edwards, J., Roenker, D., Rizzo, M., & Ball, K. (2007). The accelerate study: The longitudinal effect of speed of processing training on cognitive performance of older adults. Rehabilitation Psychology, 52(1), 89–96. https://doi.org/10.1037/0090-5550.52.1.89
Walker, M. F., Leonardi-Bee, J., Bath, P., Langhorne, P., Dewey, M., Corr, S., … Parker, C. (2004). Individual patient data meta-analysis of randomized controlled trials of community occupational therapy for stroke patients. Stroke, 35(9), 2226–2232. https://doi.org/10.1161/01.STR.0000137766.17092.fb
28 Yates, M., Kelemen, A., & Sik Lanyi, C. (2016a). Virtual reality gaming in the rehabilitation of the upper extremities post-stroke. Brain Injury, 30(7), 855–863. https://doi.org/10.3109/02699052.2016.1144146
Yates, M., Kelemen, A., & Sik Lanyi, C. (2016b). Virtual reality gaming in the rehabilitation of the upper extremities post-stroke. Brain Injury, 30(7), 855–863. https://doi.org/10.3109/02699052.2016.1144146
Yeun, E. J., & Shin-Park, K. K. (2006). Verification of the profile of mood states-brief: Cross-cultural analysis. Journal of Clinical Psychology, 62(9), 1173–1180. https://doi.org/10.1002/jclp.20269
