Graduate School of Communication University of Amsterdam
2021—2022
Master’s programme entertainment communication
1
Offbeat type beat:
Music-movement synchrony, enjoyment, and perceived exertion in virtual reality rhythm gaming
Completion date: July 1st, 2022 Word count: 7386
Erika Herrmann Student number: 12231584 Dr. Jeroen Lemmens
2 Abstract
The present study investigated the effect of synchronous music on enjoyment, performance, and perceived exertion in VR rhythm gaming. Synchronous music was predicted to have a positive effect on game performance, increase enjoyment, decrease perceived physical exertion and HR, and increase HRV. Hypotheses predicted song enjoyment and song familiarity to have positive effects on enjoyment of VR rhythm gameplay. In a 2-x-3 experimental design, participants recruited through the University of Amsterdam (N = 201) were assigned to conditions of synchronous or asynchronous play in difficulty levels that were either above, below, or matching their skillset. Game performance, heart rate, and HRV were recorded, and participants afterwards reported their levels of song enjoyment, song familiarity, game enjoyment, and perceived exertion through a post-
experiment survey. Enjoyment of songs was positively associated with game enjoyment.
Synchronous music may have positively impacted game performance when the level of game difficulty was too easy or matching the skillset of the player. Synchronous music did not have a significant impact on the enjoyment of the game, nor did it significantly impact HR, HRV, and perceived exertion.
3 Introduction
The current number of commercially available virtual reality headsets make it evident that virtual reality (VR) has evolved beyond its original reputation of a nausea-inducing novelty. Its increasing accessibility and advancements have created new paths for gaming to continue to grow through. VR gaming is a natural extension of traditional video gaming, building on extant gameplay mechanics combined with basic movement intuition. When games are successfully adapted from their original screen-and-console format into a VR version, players experience an enhanced version of the game (Pallavicini et al., 2019).
Perhaps the most apparent, if not the foremost difference between VR gaming and traditional gaming is the difference in visual interface. The primary feature and function is to encompass the player in the game, a complete 360-degree world which the user finds themself a part of, without being restricted to a monitor. Beyond visual aesthetics, players’ ability to see their own hands in virtual reality creates the feeling of being inside the game, allowing a more direct and enhanced cognitive connection with their capabilities of avatar control and movement (Bollmer & Suddarth, 2022).
While the visual aspects of VR have been extensively researched, the auditory
elements are more often overlooked when comparing the medium to a traditional console. VR offers and often employs binaural audio—a feed of sound that mimics the way hearing works in reality and tracks individuals’ head movements to create more realistic audio (Bible, 2016). Gamers give feedback that in traditional media, audio elements are more noticeable and more important to their playing, whereas in virtual reality, sounds and music contribute more subtly to the overall immersion and experience, and therefore tend to go more
unnoticed, (Rogers at al., 2018). The differences in VR audio are subtle but meaningful, as they work with the mechanics of the headset and the movement of the player to create a feed of sound that contributes to a more immersive experience. In VR games, music and sound
4 design have the possibility of playing more prominent roles when the immersive sound capabilities are utilized to their full extent.
Music’s presence in a video game is not to fill in an empty space, but to meaningfully contribute to the overall experience of playing the game. Music provides a connection
between player and game, an emotional bridge between narrative and reality that should create a world more immersive than silence and sound effects (Berndt & Hartmann, 2008).
For instance, music’s role in the enjoyment of a rhythm game has been previously studied with its impact on fan communities and cultures (Demers, 2006). In examining its effects on actual play, the type of music used in a video game can differently impact an individual’s affective response depending on the content of the game (Zhang & Gao, 2014). The interactions between genres and game mechanics therefore go beyond the significance of music’s inclusion at a basic level—the type of music and the type of game must work together to create a cohesive experience.
In certain game genres, such as rhythm games, music goes beyond a supplementary factor and becomes the focus of gameplay. Primarily taking the form of non-narrative games, rhythm games have found prominence and popularity long before the advent of VR, in arcade and console games such as Dance Dance Revolution and Guitar Hero, which notably employ music as one of their main features and appeals. Rhythm games rely on visual cues and movement coordinated to the music of the game, wherein actions are prompted in time with the music’s rhythm (Charbonneau et al., 2009). One of the most successful rhythm games in VR format is Beat Saber, having sold over four million copies worldwide since its release (Baker, 2021). Rhythm gaming’s popularity is in part due to its ease of accessibility, whether via app or console. In a rhythm-based video game, music plays a large role in facilitating gameplay, with the speed and beats of the song coordinating with the visual cues on screen in order to create a prompt for the player to move with the music. There is more physical
5 movement involved in VR gameplay than there is when playing with a console alternative.
However, in VR rhythm games, the extent of effects of music on performance remain unknown.
Both auditory and visual cues are essential to movement-music synchrony, which is when physical, real-life actions made by the player align with the music they are listening to (Takehana et al., 2019). In general, the synchrony of a VR game’s sound to the movement of the player helps to create a more realistic VR environment (Kern & Ellermeier, 2020). In an immersive VR environment, instinctive movements become more important and more reliant on a sense of presence and immersion. While music’s presence can help mitigate negative emotional and physical effects of exercise, in addition to adding to the overall enjoyment of a game, it is unknown how much of these effects are reliant on synchrony between movement and music. While the study of dance-based exercise games reveals that interactivity between screen and player helps stimulate a moderate and enjoyable level of exercise (Lin, 2015), there is still little research into this effect in a fully immersive VR environment. Therefore, this study seeks to answer the following research question:
What is the influence of synchronous music on enjoyment, performance, and perceived exertion in rhythm-based VR games?
Theoretical framework
VR’s selling point and most salient source of appeal are its immersive qualities.
Immersion describes the feeling of absorption an individual feels and occurs when the player is fully engaged in the game, without elements of play and elements of the real world
distracting them and detracting from their focus on the game experience (Moore & Green, 2020). In movement-based VR games, higher immersion levels may suggest higher levels of
6 motivation towards exertion, leading to higher enjoyment of the game and its physical
demands (Ijaz et al., 2020). This feeling of immersion can extend to spatial presence, which describes the feeling of being physically present within a medium that an individual may have (Biocca, 1997). The individual feels actions are possible within the environment of the medium, and not the real world. VR games create a higher sense of presence for the user (Pallavicini et al., 2019). Generally, VR users’ heightened senses of presence can lead to higher levels of enjoyment (Tussyadiah, 2018). In VR gaming, a higher sense of presence can precede a stronger intention to continue to play, alongside enjoyment (Jang & Park, 2019).
Spatial presence and enjoyment are intertwined in the overall experience of the game. The effect is not entirely based upon visual aspects of the game. Music in video games can have a positive effect on spatial presence (Klimmt et al., 2018). Therefore, the music in a VR rhythm game has the potential to build and enhanced sense of presence to elicit emotional and
physical responses that are more pronounced than in a traditional game format.
Music in video games has the potential to elicit physical as well as emotional
responses. An individual’s level of physical excitement can be increased by both the music in the video game as well as the video game itself (Zhang & Gao, 2014). For instance, up- tempo, stressful music in video games elicits a physiological stress response (Hébert et al., 2005). Emotional responses can also be affected by video game music—games with a
soundtrack that players enjoy can also increase their enjoyment of the game, whereas disliked music can detract from overall game enjoyment (Klimmt et al., 2019; Faric et al., 2019).
Beyond soundtracks, popular music also has a place in gaming. Familiar music can increase game enjoyment and performance (Cassidy & MacDonald, 2009). Thus, the following hypotheses are proposed:
H1a: Song familiarity has a positive effect on enjoyment of VR rhythm gameplay.
H1b: Song enjoyment has a positive effect on enjoyment of VR rhythm gameplay.
7 Perceptual load theory illustrates that an individual’s selective attention ability is dependent on the load of information that is to be processed (Lavie, 1995). There is a
capacity limit for high levels of both audio and visual modalities which affect an individual’s ability to process stimulus loads (Pashler, 1998, p. 162). Task performance is dependent on the quality of the data that an individual must process, in addition to which cognitive
resources they must use to do so. If there is competition between processes, such as audio and visual, then performance may be limited (Norman & Bobrow, 1975). Conflicting congruent sounds distract more strongly than congruent ones, thus capturing more of an individual’s attention, however, distracting audio cues activate an increased focus on visual stimuli as a counter to the distraction (Zimmer et al., 2010). When presented with incongruent
audiovisual stimuli, individuals are more likely to prioritize the visual cues over the audio (Collignon et al., 2008). When individuals choose which stimuli to focus on and which to ignore, they typically notice only the overt and obvious properties of the “ignored” stimuli (Pashler, 1998, p. 95). Thus, when the game audio is distracting as it is asynchronous, more focus may be directed to what players then aim to focus on: the visuals of the game. Presence of music alone does not affect performance (Hébert et al., 2005). When gameplay is paired with noncontingent music, that is, music with pacing and mood that does not match that of the game, players score higher than gaming with the game’s original score (Tan et al., 2010).
Therefore, it is predicted that over the course of gameplay, players may be able to discern asynchronous music. Therefore, the second hypothesis is introduced:
H2: Synchronous music has a positive effect on VR rhythm game performance.
When gameplay is not matched to its background music, players become more concentrated on the incongruity. Their performance level is lower when the music is inappropriate for the game, as attention is diverted from gameplay and away to the dissent between the audio and visual stimuli (Zhang & Gao, 2014). When the visual stimuli are more
8 difficult to process when individuals are presented with audiovisual cues, then auditory modality is preferred, indicating a general preference of the path of least ambiguity
(Collignon et al., 2008). Distracting, incongruent music may only succeed in detracting from player scoring ability when the visuals of the game are too difficult to focus on as a counter measure.
In rhythm games, music’s role expands beyond enhancing the experience, to rather being the foundation of the game as a whole. Rhythm games command actions to be made in time with music, based on a perception of musical synchrony. Perception of synchrony links movement to different audiovisual cues (Takehana et al., 2019). Individuals feel the need to move with the beat of the music they hear, to whichever game prompt for action they may be presented with. However, the audio stimulus of music prompting a movement does not overtake the visual prompts, nor their resulting activation of movement instincts (Sevdalis &
Keller, 2009). A rhythm game can still be played asynchronously to the music, though the difference in movement-music synchrony is noticeable enough to potentially draw players out of the game and back into reality. Players of VR exergames seek intuitive movements, but also enjoy games with music (Faric et al., 2019). Exercise with music can increase positive emotional response level (Lim et al., 2014). When music is included but actively detracts from the intuition of controlling the game, there is the possibility of a negative effect on immersion and presence, as attention is being directed towards reality, and away from gameplay. The implication here is the possibility that the game is less enjoyable. A third hypothesis is proposed:
H3: Synchronous music has a positive effect on enjoyment of VR rhythm games.
More taxing levels of gameplay difficulty result in higher levels of exertion, while low gameplay difficulty requires little exertion. In both of these conditions, level of
9 enjoyment is lower than when gameplay requires a moderate level of exertion (Yoo et al., 2018; Pallavicini & Pepe, 2020). The relationship between enjoyment and exertion may differ when perceived exertion and actual exertion are simultaneously analyzed. High levels of immersion can distract from and mitigate feelings of exertion in virtual reality exercise games (Faric et al., 2019). Immersion is created by the nature of virtual reality; however, it is
possible that other elements of the gameplay distract from it. In the case of exertion, gameplay that requires too high a level of physical effort can detract from the world of the game and shift the focus of the player to reality. The distraction provided by off-beat music could also lead to a similar consequence. Exercise with synchronous music can significantly reduce perceived levels of exertion (Lim et al., 2014). With synchronous music, individuals are experiencing a higher level of immersion as the sound does not detract from the
experience of the game. An implied consequence of exercise that goes against the beat of the music—that is asynchronous—is that exertion is possibly perceived accurately, or that the individual feels they are putting forth more exertion than they would if they were moving synchronously with the music. The fourth hypothesis is suggested:
H4a: Synchronous music has a negative effect on perceived physical exertion in VR rhythm gameplay.
For the purpose of comparison of perception of physical strain to a more objective measure, two additional hypotheses are proposed:
H4b: Synchronous music has a negative effect on HR in VR rhythm gameplay.
H4c: Synchronous music has a positive effect on HRV in VR rhythm gameplay.
10 Method
Sample
In order to examine the effects of synchronous versus asynchronous music against easy, difficult, and skill-matched levels of gameplay difficulty, a 2-x-3 experiment was developed. Power analysis indicated a required sample size of 180, with 30 participants per condition. A total of 201 English-speaking participants between the ages of 18 and 60 were recruited (M = 21.31, SD = 3.89). Participants were assigned across the six conditions, which were divided between asynchronous and synchronous play, and again according to level of gameplay difficulty. The gameplay difficulty conditions were part of a larger study
measuring flow, though the measures are not relevant for the present study. The final sample included three asynchronous conditions: an asynchronous easy condition (n = 33) that was 27.3% male (n = 9) and 72.7% female (n = 24), an asynchronous hard condition (n = 33) that was 21.2% male (n = 7), 75.8% female (n = 25), 3.0% nonbinary (n = 1), and an
asynchronous match condition (n = 32) that was 21.2% male (n = 6), 78.1% female (n = 25), and 3.1% nonbinary (n = 1). Similarly, the three synchronous control conditions included a synchronous easy condition (n = 34) that was 23.5% male (n = 8), 73.5% female (n = 25), and 2.9% nonbinary (n = 1), a synchronous hard condition (n = 36) that was 25.0% male (n = 9) and 75.0% female (n = 27), and a synchronous match condition (n = 33) that was 27.3%
male (n = 9) and 72.7% female (n = 24). 74.6% of participants were female (n = 150), with the remaining 23.9% of participants identifying as male (n = 48) and 1.5% as nonbinary (n = 3) with participants’ ages ranging from 18 to 60 (M = 21.31, SD = 3.89). Participants were mainly undergraduate Communication Science students at the University of Amsterdam—
122 of which chose research credits as a participation reward. Other participants received the monetary reward—those individuals being the remaining Communication Science students, students of other disciplines and universities, and the non-students.
11 Procedure
Participants were informed that taking part in the study would require them to play the game Beat Saber, and that they be at least 16 years of age. After participant consent was given, participants were asked to wear a Polar H10 Heart Rate Monitor. All participants played the rhythm VR game Beat Saber. Regardless of the condition assigned, the study involved playing through eight songs in Beat Saber, including a tutorial, a song used to assign gameplay difficulty, and six more songs that were manipulated based on condition assignment. The six songs and levels were sourced from the free Beat Saber community- made website BeastSaber, on the basis of levels had a high on-site ranking and included the difficulty levels easy, normal, difficult, and expert, and songs of various genres, languages, BPMs, and release dates (ranging from 1997 to 2019), in order to account for possible music preference and familiarity. Six songs were included in the experiment: Blow by Ke$ha, Hollaback Girl by Gwen Stefani, a shortened version of Black and Yellow by Wiz Khalifa, Toxic by Britney Spears, Du Hast by Rammstein, and a shortened version of Say So by Doja Cat.
For the experiment, each participant was asked to wear a Polar H10 Heart Rate Monitor connected to the Elite HRV mobile application in order to measure their heart rate (HR) and heart rate variability (HRV) throughout the course of playing the game. They were informed about the general length of play and tasks, and then assigned to either the
synchronous or asynchronous condition. Participants first played through a tutorial, during which their initial HR and HRV were measured. Afterwards, they played Elixia by Mord Fustang to prepare them for actual gameplay and to measure their level of gameplay adeptness for difficulty condition placement. The “good cuts,” or successful in-game movements that were recorded after a playthrough of Elixia on normal difficulty, were then
12 used as a basis to which actual participants would be assigned to conditions in the
experiment, the division of which is shown in Table 1.
Table 1
Experiment condition distribution as based on pretest scores
Condition
Required cuts in Elixia
Level of gameplay
difficulty Song 1 Song 2 Song 3 Song 4 Song 5 Song 6
Too easy
> 259 Easy
Toxic Blow Say So Du Hast
Black and Yellow
Hollaback Girl
Match
0 - 259 Easy
Toxic Blow Say So Du Hast
Black and Yellow
Hollaback Girl 260 - 269 Normal
Du Hast
Black and
Yellow Blow Say So Toxic
Hollaback Girl 291 – 300 Hard
Du Hast
Black and
Yellow Say So
Hollaback
Girl Blow Toxic
Too hard
< 280 Hard
Du Hast
Black and
Yellow Say So
Hollaback
Girl Blow Toxic
> 280 Expert Hollaback
Girl Du Hast Toxic
Black and
Yellow Blow Say So
For the manipulated and non-manipulated songs, participants played through two at a time in three blocks, for the six total songs that constituted the main experiment. In between every two songs, players were asked for their estimation of the amount of time passed for the two songs they had just played in each block, as a part of the larger study involving flow.
Afterwards, participants took a 10-minute survey about their experience, for a 40-minute total experiment duration. Participants were rewarded with either university research credits or
€7.50 upon completion.
Manipulation
For pilot testing, gameplay sessions with several participants were conducted a priori in order to categorize the difficulty levels of each of the six songs. This was done across every game-assigned level (easy, normal, difficult, and expert) in order to reach a general
13 consistency and account for differences in level design and to create an order of play that would gradually increase in difficulty. The feedback and scores on each song was assessed and used to create a ranking, resulting in a different song order per difficulty level, as shown in Table 1.
Musical synchrony indicates an alignment between the beat of the music and an individual’s movement (Bacon et al., 2012). Rhythm-based games use musical synchrony as the primary game mechanic, wherein the controls of the game trigger game events in synch to the music of the game (Richardson & Kim, 2011). In Beat Saber, musical synchrony was manipulated based on condition, and players’ concept of musical synchrony (M = 3.78, SD = 1.05) was afterwards measured with the question “Did you feel like the rhythm of the songs matched the movement rhythm indicated by the boxes” on a 7-point Likert scale as part of a manipulation check. In order to manipulate synchrony, levels were uploaded to the web application Beatmapper, wherein the placement of blocks was randomly offset throughout each level in variations of beats or fractions of beats, thus either arriving too early or too late at random and without consistency. In order to achieve asynchronousness that was more unpredictable and would not be perceived as a consistent delay or time offset, blocks were divided into groups of 20 to 30 and were misplaced either forwards or backwards, by as little as 370 milliseconds and as much as 1640 milliseconds, for an average offbeat displacement of 950 milliseconds. The song order per difficulty was kept in both the synchronous and asynchronous conditions. An independent samples t-test was conducted to determine the effectiveness of music-movement synchrony manipulation. Levene’s test for equality of variances was significant, F = 7.71, p = .006. The manipulation check indicated that the synchronous group had significantly higher scores when rating their perception of movement synchrony (M = 4.07, SD = 0.95) in comparison to the asynchronous group (M = 3.47, SD =
14 1.07), t(194) = 4.13, p < .001, d = 0.59, 95% CI0.31, 0.88]. Thus, the intended synchrony worked as expected.
Measures
Enjoyment. As the nature of Beat Saber calls for physical involvement in gameplay, enjoyment was measured using the 18-item PACES Physical Activity Enjoyment Scale (Kendzierski & DeCarlo, 1991) modified into 7-point Likert scale questions. A principal axis factor with Direct Oblimin rotation was conducted with the 18 items that measure enjoyment.
Both the Eigenvalue-criterion and the Scree Plot show there are four factors, 8 items measuring Factor 1 (gameplay enjoyment) and 7 items measuring Factor 2 (physical enjoyment), with the remaining 3 items loading onto Factors 3 and 4, which were excluded from the final analysis. In total, the factors explained 53.91% of the variance in the 18 items, with Factor 1 accounting for 46.79% of the variance explained, and Factor 2 adding 7.12% of explained variance. Reliability of the factors is good, Cronbach’s alpha = .88 for Factor 1 and .88 for Factor 2. Enjoyment was therefore measured as a two-dimensional construct:
gameplay enjoyment and physical enjoyment. Gameplay enjoyment (M = 5.83, SD = 0.92) was measured with an 8-item scale and included the items “It was very exciting” and the recoded negative “It was no fun at all.” Physical enjoyment (M = 5.48, SD = 0.96) used a 7- item scale and included the items “I felt good physically” and “I found it energizing.”
Song familiarity and song enjoyment. Song familiarity refers to familiarity with a song played prior to the experiment. Song enjoyment refers to how much a participant enjoyed hearing each song, regardless of if they had heard it before participation in the experiment. For each song, participants were also asked to grade their enjoyment (M = 4.01, SD = 0.67) and familiarity (M = 3.98, SD = 0.93) in a single-question Likert scale for each concept: “I enjoy this song” and “I am familiar with this song.” These aspects were measured as a possible control for total game enjoyment.
15 Game performance. Players’ in-game performance refers to the level of success the participant has at the end of a played level. In Beat Saber, players receive a number of good cuts (out of total possible cuts), referring to blocks that were slashed in the correct direction.
This numeric value is different across song and level of game difficulty, but consistent across asynchronous and synchronous conditions. Game performance (M = 0.81, SD = 0.17) was calculated based on an overall percentage score, from total possible cuts over the six songs (M = 2014.73, SD = 864.75) and total cuts successfully made (M = 1496.50, SD = 413.53).
Perceived exertion. Psychological perception of exertion was measured as an individual’s perceived degree of use of their musculature as well as their heart rate (Watt &
Grove, 1993). Perceived exertion was measured using a 9-point adapted version of the 10- point version of Borg’s 1982 perceived exertion scale (as cited by Williams, 2017), with levels including rest; very slight; moderate; somewhat severe; severe; very severe; almost maximal; & maximal (M = 8.78, SD = 2.23).
Heart rate and heart rate variability. Heart rate is the general number of heart beats per minute, while heart rate variability, a related measure, refers to the changes in time
between heart beats. In most cases, a rest state will correspond with a lower HR and a higher HRV, while a state of high exertion will correspond with a higher HR and a lower HRV (Elite HRV, 2021). Participants’ HR and HRV were measured using the Polar H10 Heart Rate Monitor and the Elite HRV app, as Polar HR straps have been applied successfully in previous VR research (Lemmens et al., 2022). HR and HRV ratings were taken first at the tutorial level to establish a baseline measure from minimal movement, and then later every two songs in order to create the mean change in HR (M = 117.06, SD = 18.96) and HRV (M = 37.57, SD = 10.31) for the whole experiment, to later be compared to perceived exertion.
16 Results
Song Enjoyment and Song Familiarity on Game Enjoyment
In order to determine the relations between relevant constructs, a bivariate correlation analysis was conducted to measure the correlations between gender (male and female), age, song familiarity, song enjoyment, gameplay enjoyment, and physical enjoyment. results of the analysis are demonstrated in Table 2. A significant but weak negative correlation was found between song familiarity and age (r = -0.17, p = .021). Older participants were less likely to be familiar with the songs in game. Significant but weak positive correlations were found between song enjoyment and gender (r = 0.19, p = .009). Female participants had higher levels of song enjoyment than men. Song enjoyment and song familiarity were positively but weakly correlated (r = 0.17, p = .018). Significant, positive correlations of moderate strength were found between gameplay enjoyment and song enjoyment (r = 0.25, p
< .001), and physical enjoyment and song enjoyment (r = 0.42, p < .001). A strong
significant correlation was found between physical enjoyment and gameplay enjoyment (r = 0.77, p < .001). Expected correlations with song familiarity were not present: no significant correlations were found between song familiarity and gameplay enjoyment (r = 0.05, p = .530) nor were they found with physical enjoyment (r = 0.04, p = .578).
To supplement the significant correlations found between song enjoyment and gender, an independent samples t-test was conducted to examine whether males and females differed in their levels of song enjoyment. Levene’s test for equality of variances was significant, F = 4.49, p = .036, equal variances are not assumed. Results revealed that males had significantly lower scores for song enjoyment (M = 3.76, SD = 0.76) compared to females (M = 4.07, SD = 0.63), t(58.88) = -2.38, p = .021, d = -0.62, 95% CI[-0.56, -0.05]. Females enjoyed the songs played more than males.
17 To test whether song enjoyment positively affects game enjoyment (H1a), and
whether song familiarity positively affects game enjoyment (H1b), two regression analyses were conducted using SPSS. The first regression model with gameplay enjoyment as
dependent variable and song enjoyment, song familiarity, age, and gender (male and female) as independent variables is significant, F(4, 175) = 3.63, p = .007. The regression model can therefore be used to predict level of gameplay enjoyment, but the prediction strength is weak:
8 percent of the variation in level of gameplay enjoyment can be predicted on the basis of song enjoyment and song familiarity (R2 = .08). Song enjoyment is a significant, moderate predictor of gameplay enjoyment, b* = 0.27, t = 3.58, p < .001, 95% CI[0.17, 0.60]. For every unit increase in level of song enjoyment, gameplay enjoyment increases by .38. Song familiarity, b* = -0.02, t = -0.27, p = .790, 95% CI[-0.17, 0.13], age, b* = -0.06, t = -0.77, p = .443, 95% CI[-0.05, 0.02], and gender, b* = -0.14, t = -1.90, p = .060, all showed
nonsignificant, weak associations with gameplay enjoyment. A second regression model with physical enjoyment as the dependent variable and song enjoyment, song familiarity, age, and binary gender as the independent variables is significant, F(4, 175) = 10.89, p < .001. The regression model can therefore be used to predict level of physical enjoyment with reasonable strength: 20 percent of the variation in level of physical enjoyment can be predicted on the basis of song enjoyment and song familiarity (R2 = .20). Song enjoyment is a significant, moderate predictor of physical enjoyment, b* = 0.45, t = 6.35, p < .001, 95% CI[0.45, 0.86].
Gender is a significant negative predictor of physical enjoyment, though the association strength is weak, b* = -0.16, t = -2.25, p = .026, 95% CI[-0.69, -0.04]. Female participants felt less physical enjoyment during gameplay. Physical enjoyment had nonsignificant, weak associations with song familiarity, b* = -0.03, t = -0.44, p = .661, 95% CI[-0.18, 0.11], and age, b* = 0.05, t = 0.73, p = .469, 95% CI[-0.02, 0.05]. For every unit increase in level of song enjoyment, physical enjoyment increases by .66. For females in comparison to males,
18 physical enjoyment decreases by .36. Therefore, support is found for H1a: song enjoyment is positively associated with game enjoyment. No support is found for H1b’s assumption that song familiarity has a positive effect on game enjoyment.
Table 2
Descriptive statistics and correlations for study variables.
Variable n M SD 1 2 3 4 5
1. Gender (male and female) 201 1.77 0.43 —
2. Age 201 21.31 3.89 -0.11 —
3. Song familiarity 183 3.98 0.93 -0.06 -0.17* —
4. Song enjoyment 183 4.01 0.67 0.19** 0.04 0.17* —
5. Gameplay enjoyment 201 5.83 0.92 -0.08 -0.03 0.05 0.25** — 6. Physical enjoyment 201 5.48 0.96 -0.08 0.08 0.04 0.42** 0.77**
*p < .05, **p < .005
Musical Synchrony on Gameplay Performance
To test whether synchronous music has a positive effect on gameplay performance (H2), a two-way analysis of variance was conducted. Music synchrony was included as the independent variable and gameplay performance was included as the dependent variable. In addition, level of gameplay difficulty was included as a second independent variable for the purpose of examining potential differences between groups. The results demonstrated a significant but weak main effect of music synchrony on game performance, F(1, 195) = 4.80, p = 0.030, ƞ² = 0.00, as well as a significant, strong main effect of difficulty condition on game performance, F(2, 195) = 483.06, ƞ² = 0.83. Music-movement synchrony explained 0%
of differences in game performance scores, but difficulty of play explained 83% of differences in game performance scores. Estimated marginal means indicated that in play conditions that were too easy for players’ skillsets, playing with synchronous music resulted in slightly higher scores (M = 0.97, SD = 0.01) than playing with asynchronous music (M = 0.94, SD = 0.10). In play conditions that were too difficult for players, playing with
19 synchronous music resulted in the same score (M = 0.59, SD = 0.09) compared to when playing with asynchronous music (M = 0.59, SD = 0.08). When game difficulty matched players’ skillsets, playing with synchronous music (M = 0.90, SD = 0.05) resulted in slightly higher scores than when playing with asynchronous music (M = 0.88, SD = 0.06). The analysis additionally revealed a non-significant, weak interaction between the effect of music synchrony on level of gameplay difficulty, F(2, 195) = 0.44, p = .644, ƞ² = 0.00. The effect of synchronous music on game performance is not different for those who played at a level of difficulty that was too easy, too hard, or matching their level of skills. The effect of
synchronous music on game performance is not different for those who played at a level of difficulty that was too easy, too hard, or matching their level of skills. H2 cannot be fully supported as, while synchronous music has a positive effect on gameplay performance, this effect is very slight and only occurs in conditions where difficulty is either below or matching the skillset of the player.
Musical Synchrony on Game Enjoyment
In order to test whether synchronous music positively affects game enjoyment (H3), an independent samples t-test was conducted to determine the effect of music
synchronousness on enjoyment. Results revealed no significant difference in level of gameplay enjoyment between the asynchronous and synchronous conditions, t(181.71) = - 1.51, p = .133, d = -0.22, 95% CI[-0.45, 0.06]. Estimated marginal means revealed that those who played with synchronous music (M = 5.74, SD = 1.06) and those who played with asynchronous music (M = 5.94, SD = 0.73) did not significantly differ in level of gameplay enjoyment. For physical enjoyment, Levene’s test for equality of variances was significant, F
= 6.28, p = .013, equal variances are not assumed. Results revealed no significant difference between the asynchronous and synchronous conditions, t(191.92) = -1.32, p = .190, d = -0.19, 95% CI[-0.44, 0.09]. Those who played with synchronous music did not significantly differ
20 in level of physical enjoyment (M = 5.40, SD = 1.05) compared to those who played with asynchronous music (M = 5.57, SD = 0.84). Therefore, support cannot be found for H3’s assumption that synchronous music positively effect game enjoyment.
Musical Synchrony on Perceived Exertion, HR, and HRV
To determine the relations between constructs relevant to H4, a bivariate correlation analysis was conducted to reveal the correlation between HR, HRV, perceived exertion, age, and gender (male and female). Full results of the analysis are demonstrated in Table 3. Age and perceived exertion had a weak, significant negative correlation (r = -0.18, p = .028).
There is a slight correlation between being younger and feeling higher levels of exertion while playing. Perceived exertion and HR were significantly but weakly correlated, (r = 0.19, p = .016). There is a small correlation between having a higher HR and feeling higher levels of exertion while playing. HR and HRV had a moderate, significant negative correlation (r = -0.47 p < .001). The higher a participant’s HR was, the lower the HRV: variability in heart rate decreases when heart rate increases. While a relationship was expected, perceived exertion was not significantly correlated with HRV (r = -0.43, p = .595).
Table 3
Descriptive statistics and correlations for study variables.
Variable n M SD 1 2 3 4
1. Gender (male and female) 201 1.76 0.43 —
2. Age 201 21.31 3.89 -0.11 —
3. Perceived exertion 158 8.78 0.45 0.01 -0.17* —
4. HRV 201 37.57 10.31 -0.05 -0.04 -0.04 —
5. HR 201 117.06 18.96 0.00 -0.07 0.19* -0.47**
*p < .05, **p < .005.
21 In order to test whether synchronous music has a negative effect on perceived
exertion (H4a), a univariate analysis of variance was conducted demonstrating the effect of synchronous music on perceived exertion, when controlling for the effect of gameplay difficulty. There was a non-significant, weak main effect of music synchronousness on
perceived exertion, F(1, 155) = 1.19, p = .276, ƞ² = 0.01. Those who played with synchronous music (M = 8.63, SD = 2.42) did not have significant differences in perceived exertion
compared to those who played with asynchronous music (M = 8.78, SD = 2.23). The assumption that synchronous music has a negative effect on perceived exertion is not supported.
To test whether synchronous music has a negative effect on HR (H4b), A second univariate analysis of variance was conducted. The analysis demonstrated the effect of synchronous music on HR, when controlling for the effect of gameplay difficulty. There was a non-significant effect of music synchronousness on perceived exertion, F(1, 199) = 3.45, p
= .065, ƞ² = 0.02. Those who played with synchronous music (M = 114.73, SD = 19.19) did not have significant differences in HR compared to those who played with asynchronous music (M = 119.49, SD = 18.51). No support is found for the assumption that synchronous music has a negative effect on HR.
Finally, a third univariate analysis of variance was conducted to test whether
synchronous music has a positive effect on HRV (H4c). The analysis demonstrated the effect of synchronous music on HRV, when controlling for the effect of gameplay difficulty. There was a nonsignificant, weak main effect of music synchronousness on HRV, F(1, 198) = 1.05, p = .307, ƞ² = 0.00. Those who played with synchronous music (M = 38.30, SD = 11.16) did not have significant differences in HRV compared to those who played with asynchronous music (M = 37.57, SD = 10.31). Support is not found for the assumption that synchronous music has a positive effect on HRV.
22 Discussion
The immersive play environment of VR lends itself to new dimensions of music- based gaming that have the potential to affect physiological and psychological responses through exploration of audiovisual and movement-based gameplay mechanics. The main purpose of this study was to gain a better understanding of the effects of music-movement synchrony on enjoyment and perceived exertion in virtual reality rhythm gaming. There are two key findings to highlight: first, the results of this study indicate song enjoyment is positively associated with overall game enjoyment. Second, synchronous music may
positively impact game performance when the level of game difficulty is manageable for the player.
Song Enjoyment and Song Familiarity on Game Enjoyment
The results of this research provide supporting evidence that the enjoyability of music correlates with the enjoyability of rhythm gameplay in VR. In line with previous research, enjoyment of the music in a video game can increase enjoyment of the game itself (Klimmt et al., 2019; Faric et al., 2019). Whereas Cassidy and MacDonald (2009) previously found that song familiarity can increase game enjoyment, the present study did not find significant effects of song familiarity on game enjoyment in neither the gameplay nor the physical dimensions of measurement. A possible explanation of this result is the correlation between song familiarity and song enjoyment. As the songs are a part of the game, separating the enjoyment of the songs and the enjoyment of the game could simply indicate familiarity and enjoyment work in tandem to create higher levels of game enjoyment, but familiarity itself does not have an effect of its own. These findings highlight the impact of music choice and implementation in rhythm video games. As another point of consideration, there is also the possibility that the direction of the effects was not as predicted—game enjoyment could have also been a predictor of song enjoyment. To account for this possibility, future research
23 should seek to measure song enjoyment before and after a round of play. As familiarity was not found to have a significant effect on overall game enjoyment, what can be implied is the important of emphasizing a soundtrack or selection of music that players will enjoy. Video game music does not necessarily have to be recognized or popular in order to facilitate an enjoyable gameplay experience, or, alternatively, if the experience of playing the game is enjoyable, the familiarity of the music is of less importance.
Musical Synchrony on Gameplay Performance
Partial support was found for the hypothesis that synchronous music has a positive effect on game performance. In difficulty conditions that were matching or below
participants’ skillsets, game performance was slightly higher with synchronous music than with asynchronous music. However, in the condition where game difficulty was above participants’ skillsets, there was no significant difference in performance. This pattern of results is somewhat consistent with previous literature stating music that is inappropriate or mismatched for the game results in lower performance (Tan et al., 2010; Zhang & Gao, 2014). However, in the present study’s results, differences in performance level were still minimal—the hypothesis that asynchronous music reduces performance level cannot be fully confirmed with reasonable confidence. Previous research has found a significant positive impact of music-movement synchrony on video game performance (Hufschmitt et al., 2021;
Tan et al., 2010). The results from the present research would imply that these results are more game-specific, as they could not be replicated in a rhythm VR game environment. In line with arguments put forth Collignon and colleagues (2014), the lack of significant results could be explained by individuals generally preferring the path of least ambiguity when they are presented with both auditory and visual cues. Thus, it is a possibility that the visual cues were profound and enough to overtake any uncertainties caused by music asynchronousness, so that performance remained relatively similar in both conditions. Even being arrhythmic, it
24 would still be easier for players to anticipate the arrival of the blocks based on visual cues rather than audio cues, whether that decision is conscious or not. Implications from the significant results regarding performance in easy and matching levels of difficulty would indicate the importance of contingent and synchronous music to be applied in general. As the experiment forced participants into levels of play difficulty rather than allowing them to select for themselves based on their capabilities, there is also the possibility that these effects are not generalizable to real-life gameplay scenarios.
Musical Synchrony on Game Enjoyment
No evidence was found to support the hypothesis that synchronous music positively affects game enjoyment. Previous research has suggested the music in general can increase positive responses to games in general (Klimmt et al., 2019). The lack of significant results may indicate a ceiling effect wherein the game overall was still enjoyable simply due to the presence of music and the nature of the game itself, and that participants generally enjoyed the playing the game as well as enjoyed themselves physically, in both synchronous and asynchronous conditions. As many participants indicated a lack of experience with VR games, the novelty effect could have positively affected enjoyment in both conditions (Bardo et al., 1996). These findings could indicate the possibility that the importance of gameplay enjoyability outweighs music enjoyability.
Musical Synchrony on Perceived Exertion, HR, and HRV
There was no negative effect of musical synchrony on perceived exertion and HR, nor was there a positive effect on HRV found. This contradicts the findings of Lim and
colleague’s previous research indicating exercise with synchronous music can reduce perceived levels of exertion and potentially slightly reduce HR (Lim et al., 2014). However, differences between results could be due to the nature of the activity not being pure exercise but a VR game—and not even an exergame at that. As HR and HRV are correlated measures
25 (Elite HRV, 2021), the connection would explain the further lack of significant change in HRV in the asynchronous and synchronous conditions. As perceived exertion and HR were also significantly correlated a potential implication could be that the enjoyment of gameplay as well enjoyment of the music are not enough to mitigate actual feelings of exertion.
Evidence to support the hypotheses predicting the negative effect of synchronous music effect on HR and perceived exertion and the positive effect on HRV was also not found through this research. This is partially in contrast to previous research which found that synchronous music can decrease feelings of exertion (Lim et al., 2014). However, these differences in results could be as Beat Saber, while using full-body movement as a primary mechanic of gameplay, does not necessitate that that said movement is high in intensity as exercise may be.
Limitations and suggestions for future research
There are several potential limitations concerning the results of this study. One of which is the lack of control for first time VR or Beat Saber players, which could have accounted for the novelty of the experience. Previous research has indicated that the novelty of a movement-based video game can account for greater intensity of activity and higher levels of enjoyment (Lyons et al., 2014). In the present research, such an explanation could be the determining factor of several instances of nonsignificant differences. Potential follow- up research could overcome such limitations by controlling for the amount of experience participants could have with VR, Beat Saber, and videogames in general. In addition, it is important to consider the exclusion of nonbinary participants in gender-related measures, as the sample size was comparatively insufficient to provide meaningful analysis.
If, as the present study suggests, there is an effect of song enjoyment on game enjoyment, then future research should seek to explore the effect of music in other VR gaming applications. The immersive nature of VR creates an environment that prioritizes
26 visual aspects and their impact of game experience; however, music’s presence still has a significant effect on overall game enjoyment. Future research could additionally explore the relationship between game genre and music genre, music with and without lyrics, and distracting music in order to determine further effects of audio aspects in a primarily visual medium.
Conclusion
This thesis can be seen as a first step towards research on the different effects of music in VR rhythm gaming. The present study has served to enhance understanding of the effects of song enjoyability on overall game enjoyability when music is one of the foremost characteristics of an immersive game experience. Song enjoyment and song familiarity were predicted to increase enjoyment. Hypotheses predicted that synchronous music would increase enjoyment, decrease perceived physical exertion and HR, and increase HRV.
Musical synchrony seemed plausibly relevant to enjoyment based on previous research indicating the effects of music-movement synchrony in exercise (Lim et al., 2014). However, the present study found that in a VR rhythm setting, gameplay is still enjoyable regardless of the synchrony of the music, and that perceived exertion, HR, and HRV are not significantly affected. The research adds onto previous findings about enjoyment (Cassady & MacDonald, 2009; Klimmt et al., 2019, Faric et al., 2019), contributing to a growing body of evidence towards the importance of music choice and implementation in immersive movement-based video games.
27 References
Bacon, C. J., Myers, T. R., & Karageorghis, C. I. (2012). Effect of music-movement
synchrony on exercise oxygen consumption. Journal of Sports Medicine and Physical Fitness, 52(4).
Baker, H. (2021, October 28). Beat Saber Passes $100 Million In Revenue on Quest Platform. UploadVR. https://uploadvr.com/beat-saber-100-million-
revenue/#:%7E:text=This%20puts%20the%20milestone%20into,million%20copies%
20across%20all%20platforms.
Bible, T. (2016, May 31). Binaural Audio for Narrative VR. MetaQuest.
https://www.oculus.com/story-studio/blog/binaural-audio-for-narrative-vr/
Biocca, F. (1997). The cyborg's dilemma: Progressive embodiment in virtual environments. Journal of computer-mediated communication, 3(2).
Bardo, M. T., Donohew, R. L., & Harrington, N. G. (1996). Psychobiology of novelty seeking and drug seeking behavior. Behavioural brain research, 77(1-2), 23-43.
Cassidy, G., & MacDonald, R. (2009). The effects of music choice on task performance: A study of the impact of self-selected and experimenter-selected music on driving game performance and experience. Musicae Scientiae, 13(2), 357-386.
Charbonneau, E., Miller, A., Wingrave, C., & LaViola Jr, J. J. (2009, August). Understanding visual interfaces for the next generation of dance-based rhythm video games.
In Proceedings of the 2009 ACM SIGGRAPH Symposium on Video Games (pp. 119- 126).
Collignon, O., Girard, S., Gosselin, F., Roy, S., Saint-Amour, D., Lassonde, M., & Lepore, F.
(2008). Audio-visual integration of emotion expression. Brain research, 1242, 126- 135.
Demers, J. (2006). Dancing machines: ‘Dance Dance Revolution’, cybernetic dance, and musical taste. Popular Music, 25(3), 401-414.
Donohue, S. E., Woldorff, M. G., & Mitroff, S. R. (2010). Video game players show more precise multisensory temporal processing abilities. Attention, perception, &
psychophysics, 72(4), 1120-1129.
Elite HRV. (2021, March 10). Heart Rate Variability vs. Heart Rate.
https://elitehrv.com/heart-rate-variability-vs-heart-rate
Faric, Potts, H. W. W., Hon, A., Smith, L., Newby, K., Steptoe, A., & Fisher, A. (2019).
What players of virtual reality exercise games want: Thematic analysis of web-based reviews. Journal of Medical Internet Research, 21(9).
https://doi.org/10.2196/13833
28 Hébert, S., Béland, R., Dionne-Fournelle, O., Crête, M., & Lupien, S. J. (2005). Physiological
stress response to video-game playing: The contribution of built-in music. Life sciences, 76(20), 2371-2380.
Ijaz, K., Ahmadpour, N., Wang, Y. & Calvo, R. A. (2020) Player experience of needs satisfaction (PENS) in an immersive virtual reality exercise platform describes motivation and enjoyment. International Journal of Human–Computer
Interaction, 36(13), 1195-1204.
Kendzierski, D., & DeCarlo, K. J. (1991). Physical activity enjoyment scale: Two validation studies. Journal of Sport & Exercise Psychology, 13(1).
Kern, A. C., & Ellermeier, W. (2020). Audio in VR: effects of a soundscape and movement- triggered step sounds on presence. Frontiers in Robotics and AI, 7, 20.
Klimmt, C., Possler, D., May, N., Auge, H., Wanjek, L., & Wolf, A. L. (2019). Effects of soundtrack music on the video game experience. Media Psychology, 22(5), 689-713.
Lavie, N. (1995). Perceptual load as a necessary condition for selective attention. Journal of Experimental Psychology: Human perception and performance, 21(3), 451.
Lemmens, J. S., Simon, M., & Sumter, S. R. (2022). Fear and loathing in VR: The emotional and physiological effects of immersive games. Virtual Reality, 26(1), 223-234.
Lim, H., Karageoghis, C. I., Romer, L. M., & Bishop, D. T. (2014). Psychophysiological effects of synchronous versus asynchronous music during cycling. Medicine and Science in Sports and Exercise, 46(2), 407–413.
https://doi.org/10.1249/MSS.0b013e3182a6378c
Norman, D. A., & Bobrow, D. G. (1975). On data-limited and resource-limited processes. Cognitive Psychology, 7(1), 44-64.
Pallavicini, F., Pepe, A., & Minissi, M. E. (2019). Gaming in virtual reality: What changes in terms of usability, emotional response and sense of presence compared to non-
immersive video games? Simulation & Gaming, 50(2), 136-159.
Pallavicini, F., & Pepe, A. (2020). Virtual reality games and the role of body involvement in enhancing positive emotions and decreasing anxiety: Within-subjects pilot study.
JMIR serious games, 8(2).
Pashler, H. (1998). The psychology of attention. The MIT Press.
Richardson, P., & Kim, Y. (2011). Beyond fun and games: A framework for quantifying music skill developments from video game play. Journal of New Music Research, 40(4), 277-291.
29 Rogers, K., Ribeiro, G., Wehbe, R., Weber, M., & Nacke, L. (2018). Vanishing importance:
Studying immersive effects of game audio perception on player experiences in virtual reality. Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, USA, Paper 328, 1–13. DOI:https://doi.org/10.1145/3173574.3173902
Sevdalis, V., & Keller, P. E. (2009). Self‐recognition in the perception of actions performed in synchrony with music. Annals of the New York Academy of Sciences, 1169(1), 499-502.
Takehana, A., Uehara, T., & Sakaguchi, Y. (2019). Audiovisual synchrony perception in observing human motion to music. PLOS one, 14(8).
Tan, S.-L., Baxa, J., & Spackman, M. P. (2010). Effects of built-in audio versus unrelated background music on performance in an adventure role-playing game.
International Journal of Gaming and Computer-Mediated Simulations, 2(3).
Tussyadiah, I. P., Wang, D., Jung, T. H., & Tom Dieck, M. (2018). Virtual reality, presence, and attitude change: Empirical evidence from tourism. Tourism Management, 66, 140–154. https://doi.org/10.1016/j.tourman.2017.12.003
Watt, B., & Grove, R. (1993). Perceived exertion. Sports Medicine, 15(4), 225-241.
Williams, N. (2017). The Borg rating of perceived exertion (RPE) scale. Occupational Medicine, 67(5), 404-405.
Yoo, S., Carter, M., & Kay, J. (2018, October). VRmove: Design framework for balancing enjoyment, movement and exertion in VR games. In Proceedings of the 2018 Annual Symposium on Computer-Human Interaction in Play Companion Extended Abstracts (pp. 295-307).
Zhang, J., & Gao, X. (2014). Background music matters: Why video games lead to increased aggressive behavior?. Entertainment Computing, 5(2), 91-100.
Zimmer, U., Roberts, K. C., Harshbarger, T. B., & Woldorff, M. G. (2010). Multisensory conflict modulates the spread of visual attention across a multisensory
object. Neuroimage, 52(2), 606-616.
