March 28, 2018 Research Project 2 Peter Saalbrink, 6102794 Supervisor: Fleur Bouwer, PhD Co-assessor: dr. Heleen Slagter
2
Timing Mechanisms and Temporal Expectations in Music:
Rhythmic Predictability Is Dissociated from Periodicity and
Promotes Delta Phase Entrainment
Peter Saalbrink
Abstract: Studying expectations of musical rhythm provides insight in how the brain processes temporal information. Neural oscillators are found to entrain to external rhythm, providing a substrate for the implementation of temporal expectations. However, the different effects of predictability and periodicity of rhythmic patterns are often ignored in timing research. Therefore, we aimed to validate the dissociation between two auditory timing mechanisms, duration-based and beat-based timing, in the context of delta-band (0.5–3 Hz) phase entrainment. We manipulated predictability and periodicity independently, and recorded behavioral responses to unexpected soft sounds and EEG while participants (N=31) listened to the rhythms. We found a performance increase for predictability and periodicity on-the-beat, but with a larger behavioral cost for deviance in periodic rhythms. Furthermore, we found increased delta phase entrainment for predictable rhythms, while we could not exclude that delta phase entrainment also occurred for periodic rhythms. While a full dissociation in auditory timing was not validated, we concluded that the processing of rhythmic predictability is facilitated by repetition-based timing as part of a unified timing model.
Keywords: beat-based timing; beat perception; delta phase entrainment; duration-based timing; dynamic resource allocation; EEG; neural resonance; phase coherence; repetition-based timing; rhythmic periodicity; rhythmic predictability; temporal expectations; unified timing model.
3 Contents
Introduction ... 5
Temporal Expectations ... 5
Temporal Expectations Help Shape Perception ... 5
Rhythm as a Framework for Studying Temporal Expectations ... 5
Beat-Based Timing ... 6
What Do We Need to Hear a Beat?... 6
Beat Perception ... 6
Dynamic Attending ... 7
Neural Resonance ... 9
Delta Phase Entrainment in Beat-Based Timing ... 9
Brain Network of Beat-Based Timing ... 10
Dissociation from Duration-Based Timing ... 11
Auditory Timing Dissociation ... 11
Duration-Based Timing ... 12
Double Dissociation ... 12
Delta Phase Entrainment in Duration-Based Timing ... 13
Dissociating Predictability from Periodicity ... 14
Methods... 16 Participants ... 16 Stimulus Design ... 16 Stimuli ... 16 Periodicity of Patterns ... 16 Predictability of Sequences ... 17 Deviant Stimuli ... 17 Procedure ... 18
4
Behavioral Analysis ... 19
EEG Recording and Preprocessing... 19
Delta-Band Phase Analysis and Inter-Trial Phase Clustering ... 20
Figures & Data... 22
Results ... 25
Behavioral Results ... 25
Delta-Band Phase Results ... 26
Figures & Data... 28
Discussion ... 35
Predictability and Periodicity on-the-Beat Improve Performance ... 35
Dynamic Resource Allocation ... 35
Unified Timing Model ... 36
Increased Delta Phase Coherence for High Predictable Rhythms ... 36
Individual Differences in Beat Perception ... 37
Evoked Neuronal Activity ... 37
Neural Resonance and Predictability ... 38
Updating Neural Resonance Theory ... 39
Repetition-Based Timing and the Unified Model ... 40
Suggestions for Future Research ... 42
Suggestions for Improvement ... 42
Conclusion ... 44
Figures ... 45
5
Introduction
Temporal Expectations
Temporal Expectations Help Shape Perception
Timing processes in the brain are crucial in perception and motor function, two systems that together enable a multitude of cognitive functions. For example, rhythm perception relies on temporal processes that involve the synchronization of motor and auditory areas (Grahn, 2012; Merchant & de Lafuente, 2014). Timing research shows that perception is not merely a bottom-up process, but partly relies on feedback of top-down information (Grahn, 2012). Timing processes in the brain exploit temporal patterns from the dynamic external world to create expectations of incoming stimuli, based on context and experience (Large, 2008; Large & Snyder, 2009). The generation of temporal expectations is an essential step in perceptual processing and helps improve performance (Leow & Grahn, 2014). The generation of temporal expectations might be achieved by the entrainment of neural networks (i.e., the synchronization of internal oscillations) to certain features of temporal patterns (Breska & Deouell, 2017; Large, 2008; Large & Snyder, 2009; Leow & Grahn, 2014; Nozaradan, Peretz, Missal, & Mouraux, 2011; Schwartze & Kotz, 2013), although contradictory findings exist on the nature of those features and their effects on phase entrainment of neural oscillatory activity. This research study aims to uncover whether separate, independent timing mechanisms are involved in the generation of temporal expectations, in the context of phase entrainment in the delta band (0.5– 3 Hz). Furthermore, an integral mechanism for the implementation of timing is proposed.
Rhythm as a Framework for Studying Temporal Expectations
Music is often used as a model for studying temporal expectations. Humans are unique in their ability to perceive musical rhythms, which relies on the processing of temporal patterns (Large & Snyder, 2009). Temporal expectations are generated at multiple levels, forming a hierarchical neural organization that influences perceptual processing (Clark, 2013). Rhythm is also hierarchically organized (Lerdahl & Jackendoff, 1983; Povel & Essens, 1985; Vuust & Witek, 2014), and thus provides a useful framework for studying the implementation of temporal expectations in the brain.
6 But much remains unknown about the generation of temporal expectations of musical rhythm. Temporal expectations might be generated based on either one of two distinct features of rhythmic patterns (Bouwer & Honing, 2015; Bouwer, Werner, Knetemann, & Honing, 2016). First, expectations can be based on a periodic beat: a salient stimulus occurring at regular time intervals in the pattern. Second, predictability of a repetitive pattern, with recurring sequences of stimuli, provides additional cues for temporal expectations.
However, predictability and periodicity may have different effects on the generation of temporal expectations. This will be discussed in more detail in section three of this chapter. First, temporal expectations of periodicity, a well-researched topic, will be discussed.
Beat-Based Timing
What Do We Need to Hear a Beat?
A musical rhythm is defined as a temporal structure of auditory stimuli, often accompanied by a salient and periodic (i.e., regular) component called the beat (i.e., pulse) that arises from the hierarchical nature of the temporal structure. Perception of periodic rhythms depends on beat-based timing (Grahn & Brett, 2007; Teki, Grube, Kumar, & Griffiths, 2011). Beat-beat-based timing entails that temporal intervals in the rhythmic structure are encoded relative to the beat (i.e., ΔTi/Tbeat).
Periodic rhythms usually consist of temporal intervals with integer ratios, although it is unknown whether this is essential (Grahn & Brett, 2007). Temporal intervals in a periodic rhythm are then encoded as multiples or subdivisions of the beat interval. A beat interval between 440 to 1080 ms is most commonly salient (Grahn & Brett, 2007), and most humans have a preferred tempo of 100 bpm or 1.67 Hz (i.e., a beat at every 600 ms; Fraisse, 1978). This frequency range corresponds with neural oscillatory activity in the delta band (Large, Herrera & Velasco, 2015). Indeed, Povel & Essens (1985) already predicted in that rhythms are easiest perceived when a hierarchical internal clock is induced.
Beat Perception
The cognitive process that extracts a regular pattern from a rhythm is called beat perception. This is a psychological response, because the beat is not an innate property of a rhythm (Lerdahl & Jackendoff, 1983). Humans may possess a genetic predisposition for beat perception
7 (Honing, Bouwer, & Háden, 2014), and even newborns display the ability of beat perception (Winkler, Háden, Ladinig, Sziller, & Honing, 2009).
Beat perception is bidirectional in the sense that it constitutes an interaction between bottom-up and top-down processing: on the one hand, when a regular beat interval is detected, predictions of the beat interval are generated and sustained; and on the other hand, predictions of future beat intervals are error-corrected when incoming auditory information mismatches the internal representation of the beat interval (Grahn & Rowe, 2013; Leow & Grahn, 2014). Beat perception enables an improvement in task-related performance (Leow & Grahn, 2014). This can be achieved because periodic rhythm—or more specifically, the beat interval—is encoded automatically: timing of intervals in periodic rhythms can be performed relative to the beat interval (i.e., ΔTi/Tbeat), thus enabling an improvement in performance. It is not the case
that periodic rhythms draw attention to temporal aspects of the rhythm more than aperiodic rhythms do (Grahn & Rowe, 2009); instead; for aperiodic rhythms, intervals must be encoded separately, which demands higher processing resources (Leow & Grahn, 2014).
Concluding, the temporal structure of a periodic rhythm enables the generation of temporal expectations (Large, 2008; Large & Snyder, 2009, Vuust, Ostergaard, Pallesen, Bailey, & Roepstorff, 2009). Temporal expectations enable high vigilance for when the internal representation of the beat is altered, for example through deviances in the rhythm. Deviance in a periodic rhythm is easiest to detect at the more salient beat positions, and causes re-evaluation of the internal representation of the beat (Ladinig, Honing, Háden, & Winkler, 2009; Vuust et al., 2009).
Dynamic Attending
Beat perception even occurs when the moment of the beat coincides with a silence (i.e., syncopation; Tal et al., 2017). This shows that a physical stimulus at the time of the beat does not need to be present for beat perception, which is possible because beat perception intrinsically arises from the physics of neural oscillations (Large et al., 2015). The ability to extract a beat from a rhythmic structure indicates the existence of endogenous periodicity (McAuley, 2010). Beat perception can occur even for frequencies not physically present in the periodic stimulus. This rules out the possibility that synchronization of neural oscillations is a mere consequence of listening to periodic stimuli (Large et al., 2015; Tal et al., 2017).
8 Beat-based timing is therefore often described using entrainment models, which are based on internal oscillators. These neural oscillators have a flexible period and phase, which are predicted to entrain to the beat in a periodic rhythm. Neural oscillators consist of populations of excitatory and inhibitory neurons, that, through their interactions, create oscillations of activity at different periods that can be entrained to a periodic stimulus, and facilitate perceptual processing of stimuli (Large & Snyder, 2009). Entrainment of oscillators entails a modulation of both their period and phase. Thus, the phase of naturally occurring spontaneous neural oscillations is affected when a rhythmical stimulus is presented. This is predicted to result in entrainment of the oscillators to the rhythm, measured as increased phase coherence across trials in research experiments. Temporal expectations of the stimulus are generated when the frequency of the oscillator is greater than the frequency of the stimulus and their relative phase is negative (Large & Snyder, 2009).
It was first hypothesized by Jones (1976) that the generation of temporal expectations is facilitated by internal oscillations, although they were described as attentional oscillations. Jones’ dynamic attending theory (DAT) poses that periodic rhythms entrain these attentional oscillations so that they are in phase, to facilitate perceptive processing with those rhythms (Jones, 1976; Large & Jones, 1999). Attentional oscillations were thought to represent internal fluctuations in attententional energy and enable coordination with the dynamic external world. The predictions of DAT correspond to findings from electroencephalography (EEG) studies that peaks in attentional energy are entrained to the beat in a periodic rhythm (Bouwer & Honing, 2015). It has been found that activity in the brain areas of the beat-based timing network, which will be discussed in more detail at the end of this section, is most enhanced when the beat interval is close to the preferred tempo (i.e., 600 ms; Grahn & Brett, 2007; Leow & Grahn, 2014; Schwartze, Rothermich, Schmidt-Kassow, & Kotz, 2011). Furthermore, it has been found that when neuronal groups entrain to periodic stimuli, neurons are periodically activated at the entrained interval, and even continue to fire periodically when the periodic stimulus has ceased (Sumbre, Muto, Baier, & Poo, 2008). Thus, DAT has been proven to describe how neural oscillations can be exploited to act like these attentional oscillations to generate temporal expectations about future stimuli (Large et al., 2015).
9
Neural Resonance
Neural resonance theory (NRT) expands on the idea that humans possess an internal periodic representation of the beat. Specifically, NRT is a computational model that shows how neural oscillators resonate to external rhythms and how an internal representation of the beat is generated through nonlinear coupling of those oscillators (Large, 2008; Large, 2011; Large et al., 2015; Large & Kolen, 1994; Large & Snyder, 2009). NRT hypothesizes that neural activity around the frequency of the beat is internally generated and not stimulus-driven. The finding that spontaneous emergence of neural oscillations in the EEG signal at frequencies other than either frequencies found in the stimulus patterns (i.e., subharmonics) or the frequency of the beat when listening to periodic rhythm is also explained by this prediction of NRT (Tal et al., 2017).
Findings from EEG studies show that maximal entrainment to periodic stimuli is indeed accomplished when the beat frequency matches the internally generated optimal frequency (Grahn & Brett, 2007; Stefanics et al., 2010; Will & Berg, 2007). Entrainment is usually expressed as inter-trial phase coherence (ITPC). This entails that the phase of, in this case, low-frequency neural oscillations at the stimulus onset (e.g., of the beat) in a rhythmic pattern is consistent across trials (Busch, Dubois, & VanRullen, 2009; Busch & vanRullen, 2012; Henry & Obleser, 2012; Mathewson, Gratton, Fabiani, Beck, & Ro, 2009; Stefanics et al., 2010; VanRullen, Busch, Drewes, & Dubois, 2011). Additionally, bursts (i.e., peaks in oscillatory power) in higher-frequency neural activity coincide with on-beat stimuli (Fujioka, Trainor, Large, & Ross, 2009, 2012; Zanto, Large, Fuchs, & Kelso, 2005). Induced oscillatory activity is even increased when on-beat stimuli are omitted (Snyder & Large, 2005) or imagined (Iversen, Repp, & Patel, 2009). Thus, NRT provides an explanation for the measurable EEG responses to periodic stimuli. Supported by the findings from EEG studies, NRT proves that neural oscillations reflect a representation of rhythm in the brain, and that temporal expectations have a neural substrate in oscillatory brain activity.
Delta Phase Entrainment in Beat-Based Timing
It has been highlighted in the previous paragraph that entrainment of neural oscillations to periodic rhythm occurs. One frequency range of neural activity, the delta band, stands out as a likely candidate for entrainment, as it corresponds with the optimal beat frequency. Indeed, it
10 has been found that delta-band activity synchronizes to the beat when listening to periodic rhythms. Delta oscillations entrain their phase to periodic stimuli so that neuronal excitability is temporarily enhanced and processing performance at the time of the beat is optimized (Breska & Deouell, 2017; Henry & Obleser, 2012; Lakatos, Chen, O’Connell, Mills, & Schroeder, 2007; Lakatos, Karmos, Mehta, Ulbert, & Schroeder, 2008; Lakatos et al., 2013, 2009, 2005; Merchant, Grahn, Trainor, Rohrmeier, & Fitch, 2015; Nozaradan et al., 2011; Nozaradan, Peretz, & Mouraux, 2012; Schroeder & Lakatos, 2009; Schroeder, Lakatos, Kajikawa, Partan, & Puce, 2008; Stefanics et al., 2010; Will & Berg, 2007), even in the absence of physical stimuli at the time of the beat (Tal et al., 2017). The phenomenon of delta phase entrainment makes the function of oscillatory activity apparent, and reflects the importance of being able to make temporal expectations.
Delta phase entrainment affects perceptive processing and helps extracting temporal structure of musical rhythms. This occurs through a reset of delta phase in response to external stimuli, so that task-relevant stimulus events will coincide with the phase at which neurons are best excitable (Henry & Herrmann, 2014; Schroeder & Lakatos, 2009). Additionally, the power of higher-frequency oscillations fluctuates in concordance with delta phase, coupling brain areas of the beat-based timing network (Cameron & Grahn, 2014; Merchant et al., 2015; Nozaradan, Zerouali, Peretz, & Mouraux, 2013; Will & Berg, 2007). Thus, the entrainment of the phase of delta band oscillations when listening to periodic rhythm is a mechanism through which stimulus processing and perceptual sensitivity is enhanced, and functions as a neural substrate for the generation of temporal expectations in a beat-based timing system, as predicted by DAT and NRT (Cameron & Grahn, 2014; Henry & Herrmann, 2014; Merchant et al., 2015; Nozaradan, 2014).
Brain Network of Beat-Based Timing
The timing system of the brain includes motor areas (i.e., the cerebellum, the basal ganglia, the premotor cortex, the supplementary motor area [SMA], and the pre-SMA). Motor areas, in addition to auditory regions, become activated when listening to periodic rhythms that induce a beat (Grahn & Brett, 2007; Grahn & Rowe, 2009). Moreover, motor and auditory areas show functional connectivity. The functional coupling of auditory areas to motor areas is essential for beat perception. The degree of functional connectivity is a strong indicator for task-related performance (Grahn & McAuley, 2009; Grahn & Rowe, 2009; Leow & Grahn, 2014; Vuust et
11 al., 2009). To answer the question why the motor system is involved in auditory perception, it has been hypothesized that the motor system uses patterns of simulations of periodic body movement in order to synchronize timing and generate temporal expectations of future auditory events (Grahn & Brett, 2007; Patel & Iversen, 2014).
A beat-based timing network, that includes the basal ganglia and the SMA, becomes activated for the perceptual processing of temporal periodicity (Teki et al., 2011). This network is only used when a rhythm is periodic (either isochronous or temporally regular; i.e., occurring at regular time intervals). The basal ganglia and the SMA are thought to be essential for creating and maintaining an internal representation of the beat interval. The network of basal ganglia and SMA makes feed-forward predictions about future beats, evaluating them and correcting whenever necessary. Thus, the beat-based timing network facilitates perceptual processing of temporal information and improves performance (Leow & Grahn, 2014).
Recently, the question has arisen whether temporal expectations of periodicity and the brain network of beat-based timing are mutually exclusive. But first, dissociation from another timing mechanism will be discussed.
Dissociation from Duration-Based Timing
Auditory Timing Dissociation
The auditory timing dissociation hypothesis poses that a clear distinction can be made between two separate timing systems for perceiving musical rhythm (Honing et al., 2014; Keele, Nicoletti, Ivry, & Pokorny, 1989; Leow & Grahn, 2014; Vuust & Witek, 2014). As discussed, a specialized beat-based timing network, consisting of the basal ganglia and the SMA, enables beat perception. Another, distinctive, component of the timing system of the brain is the duration-based timing network (Teki et al., 2011), described in more detail in the next paragraph. It has indeed been found that two different brain networks function as these dissociated timing mechanisms (Grube, Lee, Griffiths, Barker, & Woodruff, 2010; Morillon, Schroeder, Wyart, & Arnal, 2016; Teki et al., 2011). Beat-based and duration-based timing both can have a behavioral benefit (Morillon et al., 2016), but, with the right experimental design, no behavioral benefit necessarily arises from the use of either the beat-based timing network or the duration-based timing network (Grahn & Brett, 2007).
12 However, a full dissociation of the independent effects of predictability and periodicity in the auditory modality on behavioral performance has not yet been made. Many research studies did not incorporate a periodic but unpredictable rhythm in their experimental design (e.g., Morillon et al., 2016). Therefore, we aim to definitively and categorically dissociate the effects of predictability and periodicity.
Duration-Based Timing
The existence of a duration-based timing network has been well established. Duration-based timing relies on the encoding of absolute durations of time intervals (i.e., ΔTi), and is primarily used when a stimulus is predictable (i.e., a rhythm with recurring sequences of stimuli). A network comprising the cerebellum and the inferior olive is activated for the perceptual processing of absolute temporal intervals used in duration-based timing (Teki et al., 2011). The function of the cerebellar network is that of a stopwatch-like timing mechanism, that processes absolute timing of short intervals. This network is not able to entrain to a regular beat.
Models that describe duration-based timing are usually based on an internal clock that reacts to external stimulus onsets and stores the duration of each interval as a reference in memory, to compare its timing to new stimulus onsets (Teki et al., 2011). This distincts duration-based timing from beat-based timing, which rather compares intervals relative to a periodic beat. While beat-based timing is more precise and automatic, duration-based timing employs a more flexible allocation of resources and a prolonged state of readiness (Breska & Deouell, 2017; Schwartze, Farrugia, & Kotz, 2013). Therefore, the existence of the dissociated beat-based and duration-based networks reflects an integral system for the dynamic allocation of resources.
Double Dissociation
If two distinct networks exist that facilitate two separate timing systems, it would be expected that an impairment in either one of those networks would only affect the associated timing system, but not the other. Indeed, such a double dissociation has been found. Patients with Parkinson’s disease, which affects the basal ganglia, show deficits in beat perception (Grahn & Brett, 2009); while patients with cerebellar damage perform worse in interval timing tasks (Grube, Cooper, Chinnery, & Griffiths, 2010). Beat perception was not affected in patients with cerebellar damage, so it has been ruled out that beat-based timing relies on the timing of
13 intervals by the cerebellum; while it has been confirmed that the basal ganglia are necessary specifically for beat-based timing.
However, patients with Huntington’s disease, which also affects the basal ganglia, performed worse on both beat-based and duration-based timing tasks (Cope, Grube, Singh, Burn, & Griffiths, 2014). Indeed, it has been hypothesized that a unified rather than a modular system of auditory timing exists, in which the striatum has a central role (Teki, Grube, & Griffiths, 2012). Although these findings do not rule out the possibility that the basal ganglia are merely necessary to perform the tasks in these research studies, this is highly unlikely. Notwithstanding, our observation that the basal ganglia are strongly implicated in beat-based timing still holds. The current research study therefore aims to establish in healthy individuals whether beat-based and duration-based timing are separate (i.e., supplementary) or additional (i.e., complementary) systems.
Delta Phase Entrainment in Duration-Based Timing
As highlighted above, delta phase entrainment has been associated with beat-based timing. However, recent literature contradicts this. It has been found that temporal expectations based on predictability but not periodicity are reflected by delta phase entrainment as well (Breska & Deouell, 2017; Herbst & Obleser, 2017). In these studies, periodicity had been removed from the rhythms and evoked responses that could contaminate phase entrainment measures were eliminated. Delta phase entrainment in predictable conditions did not differ in magnitude to the periodic condition, indicating that periodicity is not necessarily the only indicator for delta phase entrainment (Breska & Deouell, 2017). A suggested explanation is that duration-based timing relies on climbing neuronal activity (i.e., increasing neural activity that peaks for temporal predictions), which led to a pre-stimulus phase reset and, therefore, higher phase coherence (Breska & Deouell, 2017; Herbst & Obleser, 2017).
However, Breska & Deouell (2017) performed their experiment using visual stimuli, whereas Herbst & Obleser (2017) removed periodicity from their experimental design entirely. Thus, a full dissociation of the independent effects of predictability and periodicity in the auditory modality on delta phase entrainment has not yet been made.
14
Dissociating Predictability from Periodicity
Looking further into the generation of temporal expectations in the brain is essential to our understanding of perception of rhythmic patterns. The separate contributions of the duration-based and beat-duration-based timing networks to the generation of temporal expectations of musical rhythms remain unclear. Specifically, the dissociation between predictability and periodicity of rhythmic patterns and their separate effects on the selection of duration-based and beat-based timing systems are often ignored in research studies. Furthermore, much remains unanswered about the connection between temporal expectations and low-frequency neural oscillations. Duration-based and beat-based timing mechanisms may independently affect the phase of delta band oscillations. The current study therefore aims to validate the dissociation between duration-based and beat-based timing in the auditory modality, in the context of temporal expectations based on predictability and periodicity of rhythmic patterns.
We used EEG to record neural activity while subjects were listening to rhythmic patterns and performed a behavioral task in which their responses to unexpected soft sounds were recorded. We manipulated predictability and periodicity of the rhythmic patterns independently, and used both on-beat and off-beat sounds, thus creating four types of rhythms and eight experimental conditions. This allowed us to assess the different effects of temporal expectations generated by duration-based and beat-based timing mechanisms, at both regularly recurring and random positions in the rhythm, on perceptive processing and delta phase entrainment. We tested subjects’ button responses to deviants (i.e., intensity decrements in an auditory oddball paradigm; Ladinig et al., 2009; Schwartze et al., 2013) in the rhythmic patterns and analyzed their hit rates and mean reaction times; and we analyzed whether delta phase entrainment occurred when listening to the rhythmic patterns.
The data were analyzed to test our general hypothesis of a dissociation in performance and neural activity supporting duration-based and beat-based timing. Specifically, for button responses, we expected that rhythmic patterns with high predictability improved performance level, whereas high periodicity reduced response latencies, as found in previous studies (Morillon et al., 2016). Additionally, we expected that the effect of periodicity would only occur on the beat, resulting in a periodicity by position interaction, with reduced relative performance for off-beat sounds in the high periodic patterns. Furthermore, for the neural oscillatory activity data, we expected that delta phase entrainment would occur when a
beat-15 based timing mechanism could be used (i.e., on the beat of high periodic patterns); but not for duration-based timing, which is prompted by high predictable patterns. More specifically, we expected that phase coherence may occur for trials in any condition, but phase coherence was expected to be highest in the high periodicity conditions for on-beat trials, reflecting that subjects had the same phase of delta oscillations on every trial in these conditions. Additionally, we expected to find no subject-specificity of delta phase locking to the beat, as measured by mean-phase coherence (MPC), when comparing the between-subjects mean-phases for on-beat and off-beat trials in the high periodicity conditions.
Discovering the similarities and differences of the generation of temporal expectations in the auditory modality between predictability, relying on duration-based timing, and periodicity, relying on beat-based timing, and connecting these mechanisms to the possible neural substrate of delta phase entrainment, will ultimately help us better understand our brains. The findings of this study can be used in future timing research.
16
Methods
Participants
32 subjects (22 female, 10 male; age: 23.4 ± 4.86 years) were recruited through the recruitment system for students and other volunteers of the Psychology department of the University of Amsterdam. Participants received either 4.0 research credits or 40 euro in cash when they completed the full experiment, which took four hours maximum. All participants signed a consent form. The experiment was approved by the University of Amsterdam Faculty of Social and Behavioural Sciences Ethics Review Board. No participants reported to suffer from hearing problems or neurological or psychiatric disorders.
Stimulus Design
Stimuli
Stimuli were woodblock sounds played at 70 dB over a mono channel speaker, and were created using MatLab 8.5.0 (MathWorks, USA) at a sampling rate of 44.1 kHz and 32 bit resolution. Each auditory stimulus possessed three properties: high or low periodicity, high or low predictability, and on an on-beat or off-beat position; and was either a standard or deviant stimulus (see Figure 1).
Periodicity of Patterns
We created patterns of either five or six stimuli. Each stimulus was either on an on-beat or an off-beat position. The equivalent positions in the low periodic patterns to on-beat and off-beat stimuli in the high periodic patterns were labeled as on-beat and off-beat positions in the analyses.
High periodic patterns, corresponding to three bars in three-four meter, used integer inter-onset intervals. Several patterns for the low periodicity condition were created through the use of non-integer inter-onset intervals, and only those were selected which in a pilot study no subjects reported to have a perceivable regular beat, even in the low periodicity × high predictability condition; the patterns that were used in the low periodicity × low predictability condition were reported as completely random in a pilot study. The inter-onset intervals between stimuli were always totaling 12 interval units per pattern and were related by ratios of 1:2:2:3:4 for patterns
17 with five stimuli and 1:1:1:2:3:4 for patterns with six stimuli in the high periodic conditions, and 1:1.4:1.4:3:5.2 and 1:1:1:1.4:3:4.6 in the low periodic conditions (Grahn & Brett, 2007). In the high periodicity conditions, a beat was present at the onset of every four interval units (i.e., at accumulated interval units 1, 5, 9; the 13th interval unit was also the 1st interval unit of the next pattern in the sequence). Beats arose spontaneously from the temporal structure of the patterns (Grube & Griffiths, 2009; Povel & Essens, 1985).
The inter-onset interval (i.e., the length of one interval unit) was established in a pilot study at 150 ms. At this rate, a beat sounded every 600 ms, corresponding to the preferred tempo of 1.67 Hz (Fraise, 1982). Each interval unit was multiplied by this duration, so that each pattern always had a total duration of 1.8 s. All timing information can also be found in Table 1.
Predictability of Sequences
We created sequences of repeated patterns. Each sequence, across all conditions, always consisted of patterns of either five or six stimuli, but never both. Each sequence consisted of 128 patterns and had a total duration of 230.4 s (see Table 1). High predictable sequences consisted of one of the patterns displayed in the upper part of Table 2, which was then constantly looped for the duration of the sequence. The patterns in the lower part of Table 2
were not used in the high predictable conditions, as they would result in sequences that are identical to those created from the first two patterns when patterns are looped. In low predictable sequences, different patterns were semi-randomly concatenated from all six patterns of the corresponding high/low periodicity condition. The order of the six different patterns, as displayed vertically in Table 2, was randomized before each subsequent concatenation. This way, each different pattern occurred the same number of times within low predictability sequences. Importantly, one pattern could not be repeated for more than three consecutive patterns, so that the low predictable character of the sequence was preserved. In the high periodicity × low predictability condition, this means that the surface structure of the sequence varied, but the metrical structure was left intact.
Deviant Stimuli
In addition to standard patterns, we generated patterns containing deviants. A quarter of all patterns within each sequence (i.e., 32 patterns) contained a deviant stimulus. This was a
18 standard stimulus with an intensity decrement of 6 dB. In each sequence, 26 deviants followed interval units 1 and 3, with half at an on-beat position and half at an off-beat position. These interval units were selected because they were fixed across conditions. The remaining six deviants followed one of the remaining interval units in an evenly distributed manner. These deviants served as decoys to retain unpredictability, and were discarded in the analyses. Patterns containing a deviant were always separated by at least two standard patterns, and no pattern contained more than one deviant. The first six patterns of a sequence never contained a deviant.
Procedure
The experiment consisted of 16 sequences, for a total duration of 61.44 min (see Table 1). Each sequence consisted of patterns that belonged to one periodicity × predictability condition; so, for each condition, subjects listened to a total of four sequences. Sequences were presented in random order, but number of conditions and number of sequences per condition were fixed for each subject. The first experimental sequence was preceded by two shorter practice sequences, which were not used in analyses. A minimum hit rate of 50 percent and a maximum of two errors in the practice sequences were required to continue with the experiment. Subjects had to listen carefully and attentively to each of the sequences. Furthermore, subjects were instructed not to move during the experiment, or tap to the rhythm. In between sequences, subjects were allowed a short break of less than a minute whilst they remained seated.
When a deviant stimulus was presented, subjects had to press a button on the armrest of their chair (an operation that required minimal movement). Subjects were instructed to press the button as soon as possible after hearing the deviant stimulus, but only if they indeed perceived a softer sound. Subjects received on-screen feedback on their performance at the end of each sequence, as expressed in hit rate and number of errors.
We obtained subjects’ written informed consent and answered any questions before introducing the experiment and setting up for EEG. The experiment consisted of two comparable sessions, which were used to manipulate attention; only the second session, which was the attentive condition, was used in the current research study. The experiment was concluded by the Beat Alignment Test to assess beat perception (Iversen & Patel, 2008), which subjects completed on the computer, and the Goldsmiths Musical Sophistication Index to assess musicality
19 (Müllensiefen, Gingras, Musil, & Stewart, 2014), which subjects filled out on paper. The total duration of participation was four hours at maximum.
The experiment was conducted in a dedicated soundproof EEG lab at the University of Amsterdam. Auditory stimuli were presented over a mono channel Logitech speaker, positioned at a fixed location at 90 cm from the middle of subjects’ heads. Instructions were presented at the center of a 21” CRT monitor. Stimulus presentation and response acquisition were handled using Presentation software 19.0 (Neurobehavioral Systems, USA). A possible delay of stimulus scheduling within the software was taken into account in programming and was minimized at 0 ms. A playback latency of 26 ms was discovered within our setup, and was corrected for in analyses.
Behavioral Analysis
The collected behavioral data (i.e., hit rates and reaction times from the deviant detection task) were subjected to statistical analyses using SPSS Statistics 24 (IBM). Hit rates and mean reaction times of subjects’ button responses were compared across conditions. Hit rates were used as a measure of performance level, reflecting quality of auditory processing; and mean reaction times were used as a measure of performance speed, reflecting response readiness. For each subject, all hit rates were discarded if the mean hit rate for all conditions was below 50 percent. Furthermore, trials were discarded if the reaction time was larger than three standard deviations from the mean reaction time, separately for each condition, or if the reaction time was shorter than 50 ms.
To examine the benefits and costs of having temporal expectations based on periodicity and predictability, hit rates and mean reaction times across trials and within conditions for each subject were subjected to a repeated-measures analysis of variance (ANOVA) with factors
periodicity (high/low), predictability (high/low), and position (on-beat/off-beat). Separate
ANOVAs to highlight simple effects were performed where interaction effects were found (Schwartze et al., 2013).
EEG Recording and Preprocessing
EEG signal acquisition was handled using ActiView 6.05 (BioSemi, the Netherlands). EEG signal preprocessing was handled using the EEGLAB toolbox (Delorme & Makeig, 2004) for
20 Matlab. An EEG system with 64 head channels and 8 external channels was used, and a sample rate of 1024 Hz was used for signal acquisition. EEG data were preprocessed for independent component analysis (ICA). ICA was run on EEG data that were down-sampled to 128 Hz and high-pass filtered at 1 Hz using a FIR filter. ICA was used to remove eye blinks from the data by visual inspection of the components (Jung et al., 2000a; Jung et al., 2000b). Clean EEG data were re-referenced to the average of the left and right mastoid channels and down-sampled to 256 Hz, and bad channels were interpolated.
Delta-Band Phase Analysis and Inter-Trial Phase Clustering
To analyze the relationship between temporal expectations and activity in the delta-band, continuous data were band-pass filtered between 0.5 Hz and 3 Hz using a FIR filter, with cut-off frequencies at 0.25 Hz and 3.25 Hz. For each experimental sequence, an epoch was created from the continuous data, which were then subjected to a Hilbert transform (Luck, 2014; Le van Quyen et al., 2001). Finally, the data was segmented per trial into 60 ms long epochs. Only trials after interval 1 and after interval 3 were used, because these intervals were fixed across conditions. Furthermore, deviant stimuli were ignored, and only standard stimuli were used. Phase angles at the time of each stimulus were extracted from the EEG signal at the FCz electrode, where the effect was expected to be maximal (Breska & Deoull, 2017; Merchant et al., 2015; Nozaradan et al., 2011; Stefanics et al., 2010). Phase data was analyzed using the CircStat toolbox (Berens, 2009) for Matlab. ITPC values for each subject were calculated for each condition. ITPC values correspond to the length of the vector that accompanies the mean phase angle for each subject over all trials of a condition, and ITPC can thus take a value between 0 and 1, where higher numbers reflect greater phase coherence.
To examine the neural oscillatory mechanism of temporal expectations based on periodicity and based on predictability (Cohen, 2011), ITPC values for each subject were subjected to a repeated-measures ANOVA with factors periodicity (high/low), predictability (high/low), and
position (on-beat/off-beat). Separate ANOVAs to highlight simple effects were performed
where interaction effects were found (Schwartze et al., 2013).
Additionally, to examine whether phase coherence was equal across subjects, mean (i.e., preferred) phase angles for each subject were calculated for six factors (high and low predictability, and high and low periodicity with on-beat and off-beat trials separately; see
21
Table 3). Between-subjects mean-phase angles for each factor were used in Rayleigh’s tests, which measures significance of MPC, to examine whether the mean-phase angles were uniformly or unimodally distributed across subjects (Cohen, 2014; Mormann, Lehnertz, David, & Elger, 2000). Furthermore, to examine whether MPC was subject-specific, differences between mean-phase angles for each factor were calculated for each comparison displayed in
Table 3. Mean-phase angles within each comparison were subtracted subject-by-subject. A difference of zero implies that the within-subject mean-phase was identical across factors (i.e, if mean-phases would not differ between factor, phase angle differences would be concentrated around zero). Separate v-tests were then used to test whether the differences in mean-phase angles were significant. The v-test compared the distribution of between-subjects mean-phase angle differences to a phase angle of zero. If a v-test returns significant, mean-phases are equal for the tested factors; while if a v-test returns non-significant, mean-phases are different for the tested factors.
22
Figures & Data
Figure 1. Example of two different patterns of stimuli. Pattern A is low periodic and
corresponds to the pattern in the upper left of Table 2.A, and pattern B is high periodic and corresponds to the pattern in the upper right of Table 2.B. Each depicted digit represents the inter-onset interval (in interval units) between stimuli. Possible positions for a deviant stimulus are presented in red. On-beat positions are indicated with “>”. Note that in the high predictable conditions, the same pattern is looped to create a sequence; while in the low predictable condition, all six patterns from one column in Table 2 are semi-randomly concatenated. unit × ms s min interval - 150 - - beat 4 600 0.6 - pattern 3 1800 1.8 0.03 sequence 128 - 230.4 3.84 experiment 16 - - 61.44
Table 1. Timetable of the experiment, with the duration of each unit in different measures, as
23 A. patterns with 5 stimuli
low periodic high periodic
> > > [5.2 3 1 1.4 1.4 ] [5.2 1.4 1.4 1 3 ] > > > [4 3 1 2 2 ] [4 2 2 1 3 ]
high & low predictable > > > [3 1 1.4 1.4 5.2 ] [1.4 1.4 5.2 3 1 ] [1.4 1.4 1 3 5.2 ] [1 3 5.2 1.4 1.4 ] > > > [3 1 2 2 4 ] [2 2 4 3 1 ] [2 2 1 3 4 ] [1 3 4 2 2 ] only low predictable
B. patterns with 6 stimuli
low periodic high periodic
> > > [4.6 3 1 1.4 1 1 ] [4.6 1 3 1.4 1 1 ] > > > [4 3 1 2 1 1 ] [4 1 3 2 1 1 ]
high & low predictable > > > [3 1 1.4 1 1 4.6 ] [1.4 1 1 4.6 3 1 ] [1 3 1.4 1 1 4.6 ] [1.4 1 1 4.6 1 3 ] > > > [3 1 2 1 1 4 ] [2 1 1 4 3 1 ] [1 3 2 1 1 4 ] [2 1 1 4 1 3 ] only low predictable
Table 2. The patterns with 5 (A) and 6 (B) stimuli that were used in the different conditions.
Of each table, the left column contains patterns with low periodicity, and the right column contains patterns with high periodicity. The top row of each table contains patterns that were only used in the high predictable sequences, while the low predictable sequences used all six patterns within a column of a table. The patterns displayed in the bottom row of each table were derived by selecting different starting points of the top row patterns. Each depicted digit represents the number of interval units that is followed by the stimulus at that position. Digits that are displayed in bold represent positions used for deviants eligible for analysis. “>” indicates the position of an accent (and in the high periodic condition, a beat).
24
Pair Compare Within Independent of
1 Predictability Periodicity & position 2 Position High periodicity Predictability
3 Position Low periodicity Predictability
Table 3. Factor comparisons for mean-phase differences between several experimental
conditions. Mean-phase differences between on-beat and off-beat positions in the low periodicity conditions were tested for baseline reference; if no difference were to be found in the high periodicity conditions, it should be excluded that such a difference is found in the low periodicity conditions.
25
Results
Behavioral Results
Separate repeated-measures ANOVAs were performed on hit rates and mean reaction times (see Table 4). Based on rejection criteria, all hit rates were discarded for four subjects, as well as reaction times of 168 trials across subjects. Hit rates (𝑁 = 224, 𝑀 = .7236, 𝑆𝐷 = .1856, 𝑚𝑖𝑛 = .25, 𝑚𝑎𝑥 = 1.0) were not normally distributed (𝐷224 = .100, 𝑝 < .0005), but
on the condition level six samples were normally distributed. Reaction times (𝑁 = 8959, 𝑀 = 557.4, 𝑆𝐷 = 139.5, 𝑚𝑖𝑛 = 50.2, 𝑚𝑎𝑥 = 1170.8) were not normally distributed (𝐷8959 =
.093, 𝑝 < .0005), but sample size was sufficiently large to ignore these findings. Furthermore, Gold-MSI scores (𝑁 = 32, 𝑀 = 68.25, 𝑆𝐷 = 15.69, 𝑚𝑖𝑛 = 47, 𝑚𝑎𝑥 = 128) were normally distributed, and BAT scores(𝑁 = 32, 𝑀 = 12.75, 𝑆𝐷 = 2.664, 𝑚𝑖𝑛 = 8, 𝑚𝑎𝑥 = 17) were not normally distributed, but these variables were ignored in the current analysis. As each within-subject factor had only two levels, and there were no between-subject factors, homogeneity of variance between conditions was considered instead of sphericity of the data. This assumption was violated for both hit rates (𝐹7,216 = 2.671, 𝑝 = .011) and reaction times (𝐹7,8951 = 18.54, 𝑝 < .0005), but it was possible to ignore this in the current analysis as sample sizes were fairly equal across conditions.
Auditory processing, as expressed by hit rate, was enhanced in high predictable conditions, as well as on on-beat positions, especially in high periodic conditions. Response readiness, as expressed by mean reaction times, was enhanced on on-beat positions (independent of periodicity), as well as for high predictable rhythms, especially if a rhythm was also low periodic. Note that a decrease in mean reaction times reflects an increase in response readiness. Confirming our first hypothesis, we found a significant main effect of predictability on hit rates, with increased hit rates in high predictable conditions (see Figure 2.A). Additionally, we found a significant interaction effect of position and periodicity on hit rates. Resolving this interaction by position showed that the simple effect of periodicity was only significant for off-beat deviants, with decreased hit rates for off-beat deviants in high periodic conditions (see Figure 2.B). Consequently, there was a significantly larger decrease in hit rates for high periodic rhythms when comparing responses for off-beat deviants to responses for on-beat deviants than
26 for low periodic rhythms. Additionally, we found a three-way interaction effect on hit rates (see Table 5), but this was ignored, as this effect had only weak significance.
Contradicting our second hypothesis, we found a significant main effect of position on mean reaction times, with decreased mean reaction times to deviants on on-beat positions (see Figure 3.A). Additionally, we found a significant interaction effect of predictability and periodicity on mean reaction times. Resolving this interaction by predictability showed that the simple effect of periodicity was only significant in high predictable conditions, with decreased mean reaction times in high predictable and low periodic conditions (see Figure 3.B). Consequently, there was a significantly larger decrease in mean reaction times for low periodic rhythms when comparing responses in low predictable conditions to responses in high predictable conditions than for high periodic rhythms.
Additionally, we found significant negative correlations within each condition between hit rates and mean reaction times, which is interpreted as an indicator of inter-individual variability in performance.
Delta-Band Phase Results
A repeated-measures ANOVA was performed on ITPC values (see Table 6). Delta phase data were omitted for one participant with too much noise in the EEG signal. ITPC values (𝑁 = 248, 𝑀 = .0625, 𝑆𝐷 = .0354, 𝑚𝑖𝑛 = 1.678 × 10−3, 𝑚𝑎𝑥 = .2085) were not normally
distributed (𝐷248 = .089, 𝑝 < .0005), but on the condition level five samples were normally distributed, although it was possible to ignore this as sample size was sufficiently large. As each within-subject factor had only two levels, and there were no between-subject factors, homogeneity of variance between conditions was considered instead of sphericity of the data. Variance of ITPC values was equal across conditions (𝐹7,240 = 1.982, 𝑝 = .058).
Delta phase entrainment, as expressed by ITPC, was enhanced in high predictable conditions, as well as in low periodic conditions, and on off-beat positions (see Figure 4). Contradicting our third hypothesis, we found a significant main effect of predictability on ITPC values, with increased ITPC values in high predictable conditions. We found a significant main effect of both periodicity and position on ITPC values, with decreased ITPC values in the high periodic conditions as well as on on-beat positions. Additionally, we found several two-way interaction
27 effects on ITPC values (see Table 5), but this was ignored, as these effects had only weak significance.
To test significance of delta phase entrainment, a post hoc analysis using separate Rayleigh’s tests for each subject and each condition was performed. The results of these tests indicated that only a third of the tested 248 phase angle distributions showed significant delta phase entainment. Moreover, only 8 out of 31 subjects had significant delta phase entrainment at the moment of the beat in periodic rhythms that were either predictable or unpredictable (no subjects showed significant phase entrainment in both those conditions). It should therefore be noted that findings of increased delta phase entrainment do not necessarily involve significant delta phase entrainment.
Several Rayleigh’s tests were performed on the between-subjects mean-phase angles for each factor (see Table 7). The results showed that for each factor, MPC was significant (i.e., mean-phase angles were not distributed uniformly; see Figure 5), indicating that for each factor all subjects had similar mean delta phase at the measured position. Additionally, several v-tests were performed to test for differences in between-subjects mean-phase angles within subjects between factors (see Table 8). The results showed that each distribution of difference angles was significantly unimodal and had a mean direction of zero rad (i.e., we found no difference in the distribution of mean-phase angles between these factors; see Figure 6), indicating that the between-subjects mean delta phase was similar across factors, regardless of manipulations.
28
Figures & Data
Measure Results Simple effects
Hit rates Interaction effect of periodicity and position;
𝐹1,27 = 42.16, 𝑝 < .0005, 𝜂𝑝2 = .610
Simple effect of periodicity, off-beat;
𝐹1,27 = 46.40, 𝑝 < .001, 𝜂𝑝2 = .632
Hit rates Main effect of predictability;
𝐹1,27 = 44.17, 𝑝 < .0005, 𝜂𝑝2 = .621
-
Mean reaction times
Interaction effect of predictability and periodicity;
𝐹1,31 = 7.58, 𝑝 = .010, 𝜂𝑝2 = .196
Simple effect of periodicity, high
predictable;
𝐹1,31 = 5.13, 𝑝 = .031, 𝜂𝑝2 = .142
Mean reaction times
Main effect of position;
𝐹1,31 = 18.28, 𝑝 < .0005, 𝜂𝑝2 = .371
-
Table 4. Significant main and interaction effects from the repeated-measures ANOVA on hit
rates and mean reaction times.
Effect Results Direction
Main effect of predictability on ITPC values
𝐹1,30 = 10.39, 𝑝 = .003, 𝜂𝑝2
= .257
Increase in ITPC values for high predictable conditions Main effect of periodicity
on ITPC values
𝐹1,30 = 11.97, 𝑝 = .002, 𝜂𝑝2
= .285
Increase in ITPC values for low periodic conditions Main effect of position on
ITPC values
𝐹1,30 = 21.95, 𝑝 < .0005, 𝜂𝑝2
= .423
Increase in ITPC values for off-beat positions
Table 6. Significant main and interaction effects from the repeated-measures ANOVA on ITPC
29
Effect Results Direction
Three-way interaction effect of predictability, periodicity and position on hit rates 𝐹1,27= 4.17, 𝑝 = .051, 𝜂𝑝2 = .134
Increase in hit rates for high predictable rhythms, especially in low periodic
conditions, and a larger decrease in hit rates for on-beat deviants than for off-beat deviants when a rhythm is low predictable; the decrease in hit rates is larger for off-beat deviants when a high predictable rhythm is high periodic, and the increase in hit rates for on-beat deviants is larger when a high
periodic rhythm is low predictable Interaction effect of position and predictability on ITPC values 𝐹1,30= 2.99, 𝑝 = .094, 𝜂𝑝2 = .090
Larger increase in ITPC values on off-beat positions for high predictable rhythms, and a larger decrease in ITPC values of low
predictable conditions for off-beat positions Interaction effect of position and periodicity on ITPC values 𝐹1,30= 3.15, 𝑝 = .086, 𝜂𝑝2 = .095
Larger increase in ITPC values on off-beat positions for low periodic rhythms, and a larger increase in ITPC values of low periodic conditions for off-beat positions Interaction effect of predictability and periodicity on ITPC values 𝐹1,30= 3.73, 𝑝 = .063, 𝜂𝑝2 = .111
Larger increase in ITPC values of low periodic conditions for high predictable rhythms, and a larger decrease in ITPC values of low predictable conditions for low periodic rhythms
Table 5. Weakly significant interaction effects from the repeated-measures ANOVAs on hit
30
Factor MPC Mean
angle (rad)
NITPC Rayleigh’s z p-value
Low periodic, off-beat .7034 1.801 62 30.67 <.0005 Low periodic, on-beat .3465 2.099 62 7.44 <.0005 High periodic, off-beat .4602 0.544 62 13.13 <.0005 High periodic, on-beat .3728 1.039 62 8.62 <.0005 Low predictable .4193 1.057 124 21.80 <.0005 High predictable .4148 1.770 124 21.34 <.0005 Table 7. MPC values (with corresponding mean angles) and Rayleigh’s test for each factor.
Pair 1st factor 1st angle 2nd factor 2nd angle Difference angle v statistic p-value 1 Low predictable 1.057 High predictable 1.770 0.385 51.76 <.0005 2 High periodic, off-beat 0.544 High periodic, on-beat 1.039 0.772 26.14 <.0005 3 Low periodic, off-beat 1.801 Low periodic, on-beat 2.099 0.499 26.54 <.0005
31 Figure 2. Hit rates. A. Upper plot shows hit rates across subjects for each condition, with
significance bars indicating the main effect of predictability. B. Lower plot shows hit rates for factors position and periodicity, with significance bars indicating the interaction effect of position and periodicity. Note that significance of simple effects is not displayed. Error bars: ± 2 SE.
32 Figure 3. Mean reaction times. A. Upper plot shows mean reaction times across subjects for
each condition, with significance bars indicating the main effect of position. B. Lower plot shows mean reaction times for factors predictability and periodicity, with significance bars indicating the interaction effect of predictability and periodicity. Note that significance of simple effects is not displayed. Error bars: ± 2 SE.
33 Figure 4. Mean ITPC values across subjects for each condition. Significance bar in yellow
(lower) indicates main effect of periodicity, significance bars in red (middle) indicate main effect of position, and significance bars in green (upper) indicate main effect of predictability. Error bars: ± 2 SE.
Figure 6. Differences in mean-phase angle of each subject per comparison, with the mean
34 Figure 5. A. Upper plot shows a polar histogram of the mean-phase angles of each subject per
factor, with the MPC value in red. B. Lower plot shows the ratios between MPC values from the mean-phase angles of each subject per factor (differences were non-significant). Significance of MPC values are from Rayleigh’s test.
35
Discussion
The current research aimed to definitively and categorically dissociate the effects on temporal expectations in the auditory modality on performance and perceptual processing (i.e., delta phase entrainment) generated by listening to rhythmic patterns while independently manipulating both their predictability and periodicity.
Predictability and Periodicity on-the-Beat Improve Performance
Our results indicated that, in general, both higher predictability and higher periodicity of a rhythm improved both response readiness (i.e., performance speed) and quality of auditory processing (i.e., performance level). Specifically, higher predictability of a rhythm led to an improvement in performance level, and to an improvement in performance speed especially when a rhythm had low periodicity; while higher periodicity of a rhythm led to an improvement in performance level but not for off-beat deviant positions, and to an improvement in performance speed but only when a rhythm had low predictability.
Concerning response readiness, our results showed that high predictability and on-beat deviant position improved mean reaction times. Moreover, the effect of high predictability on mean reaction times diminished when a rhythm also had high periodicity. High periodicity also had a positive effect on mean reaction times, but only for low predictable rhythms. Concerning quality of auditory processing, our results showed increased hit rates for high predictable rhythms, and for on-beat stimuli in high periodic rhythms.
Dynamic Resource Allocation
These results confirm our hypothesis that predictability enhances quality of auditory processing. Unexpectedly however, high predictability also reduced response latencies, while high periodicity only reduced response latencies when there was low predictability. The effect of periodicity indeed occurred only on the beat, but, contradictory to what we hypothesized, only for performance level. Taken together, high predictability had the most positive, condition-free behavioral effect on both hit rates and mean reaction times. Furthermore, our results demonstrate that deviance in high periodic rhythms resulted in a larger behavioral cost, and that the on-beat effect of high periodicity on response readiness provided no additional behavioral benefit over predictability.
36 An explanation for these findings may be provided by Breska & Deouell (2017), who show that the larger behavioral cost when deviances occur in high periodic rhythms is caused by resource withdrawal. Duration-based timing is flexibly reoriented, allowing a shift of resources when deviance occurs. By contrast, beat-based timing requires temporally focused preparation, not necessarily enabling stronger temporal expectations, but rather causing an inability to maintain a high level of preparation in between periodic stimuli. Therefore, Breska & Deouell (2017) suggest a trichotomy in dynamic resource allocation (i.e., continuous, temporally predictive, and rhythmic; where the continuous mode entails that temporal information will always be utilized whenever possible; Herbst & Obleser, 2017). Duration-based timing is described as a more basic, cue-dependent, and flexible allocation of resources reflecting a prolonged state of readiness (Rohenkohl, Cravo, Wyart, & Nobre, 2012); and beat-based timing as more precise and more automatic, but also more costly when deviances occur (Breska & Deouell, 2017; Nozaradan et al., 2011). Thus, our conclusion that high periodicity did not have a larger behavioral benefit is explained by a withdrawal of resources.
Unified Timing Model
When comparing performance measures for the different conditions, we found no distinction, but rather a strong correlation (see Figure 7). Furthermore, we found an interaction effect on response readiness that was found between periodicity and predictability. This indicates that a full dissociation between beat-based timing and duration-based timing does not exist. The direction of the interaction suggests that a beat-based timing system may only be triggered in absence of the possibility to use duration-based timing. Evidence indeed points to the existence of a combined, dual timing mechanism. Contradictory to our findings, however, beat-based timing is suggested to be the default mode in such a mechanism, with duration-based timing functioning as an error-correcting component (Teki et al., 2012). This will be discussed in more detail in the third section of this chapter.
Increased Delta Phase Coherence for High Predictable Rhythms
Our results indicated that phase entrainment of neuronal activity in the delta-band occurred in the high predictability conditions, while we cannot exclude whether delta phase entrainment occurred in the high periodicity conditions. Specifically, higher predictability led to an increase in delta phase entrainment, as measured by ITPC values; while higher periodicity led to a
37 decrease in delta phase entrainment. Interestingly, an increase in delta phase entrainment was also found for off-beat stimuli.
Concerning perceptual processing, our results showed that predictability, but not periodicity or position, promotes delta phase entrainment. ITPC was biggest for off-beat positions in rhythms that were high predictable, but low periodic. Furthermore, between-subjects MPC was significant for each factor, reflecting that the phase of delta oscillations was locked across subjects for these factors.
Individual Differences in Beat Perception
The decrease in delta phase entrainment for high periodic rhythms might be explained if inter-individual value differences in phase values at on-beat sounds in the high periodic rhythms exist. However, no evidence of differences in MPC, which would indicate whether subject-specific phase locking occurred, between conditions was found. Furthermore, the results from a post hoc analysis indicated that only 8 out of 31 subjects showed significant on-beat delta phase entrainment in these conditions, thus not providing a strong basis to draw such a conclusion.
Although we found no direct evidence of phase differences, inter-individual differences in temporal task performance and beat perception do exist. Specifically, it has been found that differences in functional connectivity between motor and auditory areas constitute inter-individual differences in beat perception (Grahn & McAuley, 2009; Grahn & Rowe, 2009; Leow & Grahn, 2014; Vuust et al., 2009). Emphasis on the use of either the beat-based or the duration-based network may rely on individual readiness to engage the involved brain network (Grahn & McAuley, 2009). Furthermore, deficits in beat perception in otherwise healthy individuals may be caused by genetic polymorphisms in the dopamine system that affect motor functioning (Leow & Grahn, 2014; Wiener, Lohoff, & Coslett, 2011).
Evoked Neuronal Activity
Literature suggests that delta phase entrainment, the modulation of phase and frequency of neural oscillations to the beat, would occur for on-beat stimuli in periodic rhythms (Henry & Herrmann, 2014; Schroeder & Lakatos, 2009). However, our results show an increase in delta phase entrainment for off-beat stimuli, which might be explained by remnants from neuronal
38 activity. One possibility is that retrograde influence of evoked neuronal activity from responses to on-beat stimuli caused greater phase coherence right after the on-beat stimulus, i.e. at the off-beat stimulus. An ERP analysis could test this hypothesis. Another possibility is that climbing neuronal activity, a buildup of activity resulting from top-down signals that peaks at the moment when temporal expectation is maximal, affected the phase of delta oscillations (Breska & Deouell, 2017; Herbst & Obleser, 2017). In both cases, stimuli preceding the off-beat stimulus create a strong demand for resources that lead to a phase reset of delta oscillations, which would result in increased phase coherence at the off-beat stimulus. Evoked neural activity, and not neural entrainment, could thus explain increased phase coherence in these conditions.
In any case, these results contradict our hypothesis that delta phase entrainment would occur for on-beat stimuli in periodic rhythms. Instead, our EEG data suggest that phase entrainment of delta-band activity is a neural substrate of duration-based timing, facilitating the processing and perception of rhythms that were high predictable, but not rhythms that were high periodic. Previous studies did not always fully discriminate between predictability and periodicity (e.g., Tal et al., 2017), whereas we did. Therefore, we can conclude that the results presented in the current paper suggest that delta phase entrainment is part of a duration-based timing mechanism, while we cannot exclude delta phase entrainment as part of a beat-based timing mechanism.
Neural Resonance and Predictability
It has previously been suggested that delta phase coherence is not a unique signature of entrainment to periodic rhythm, but is triggered in both duration-based and beat-based timing, and does therefore contradict NRT (Breska & Deouell, 2017). Our findings confirm that the results from the visual domain found by Breska & Deouell (2017) also withstand in the auditory domain, showing that delta phase entrainment non-dissociably reflects temporal predictability as well as periodicity. Indeed, increased delta phase coherence would be expected for high predictable rhythms, as this would reflect optimal neural excitability (Herrmann, Henry, Haegens, & Obleser, 2016). However, Herbst & Obleser (2017) found no enhanced delta phase coherence in their predictive condition, after carefully excluding retrograde influence of evoked neuronal activity. Instead, they suggested that previous findings of increased delta phase coherence for temporal predictability are caused by neural entrainment to periodic
