Seeing is believing: The effect of musicians’ body movements on the valuation of a musical performance

SEEING IS BELIEVING

The effect of musicians’ body movements on the valuation of a

musical performance

Master Thesis Tom Wolfs

05-09-2014

Dr. M. Sadakata

University of Amsterdam

Prof. Dr. H. Honing

2

Index

1.1Evaluating a musical performance ... 6

1.2 Seeing is believing ... 7

1.3Research questions and hypothesis ... 11

2. Experiment 1 (pilot study) ... 13

2.1 Background ... 13

2.2 Methods ... 14

2.2.1 Participants ... 14

2.2.2 Stimuli and tasks ... 14

2.3 Results ... 16

2.3.1 Experiment 1a ... 17

2.3.2 Experiment 1b ... 18

2.4 Discussion ... 20

3. Experiment 2 (main study) ... 22

3.1 Research questions ... 22

3.2 Experimental design ... 22

3.2 Results and discussion ... 24

3.2.1 Expressiveness ... 24 3.2.2 Quality ... 27 3.2.3 Likeability ... 29 3.2.4 Naturalness ... 31 3.3 Discussion ... 33 4. General Discussion ... 34 4.1 Results ... 34

4.2 Strengths and weaknesses ... 35

4.3 Future work ... 37

4.4 Conclusion ... 37

Acknowledgement ... 38

3

Appendix 1 ... 42

Appendix 2 ... 44

Appendix 3 ... 45

4

Preface

In a musical era in which rock artists spend millions of euros on décor, dancers and light for a single show, in which jazz festivals are sold out within one minute and in which the American pop singer Lady Gaga has almost 42 million followers on Twitter, one might wonder what classical musicians do to make their performances more spectacular, since ticket sales keep decreasing in that sector and classical artists more than ever are faced with the label ‘boring’. There is, however, a group of classical musicians that, according to their groupies, isn’t boring and that never need to worry about empty seats. Most of these musicians adopted a very special, intimate and personal style of playing, in which they combine an extremely high level of music making with intense movement.

Examples are violinist Janine Jansen, clarinettist Martin Fröst and pianist Lang Lang. Just like successful political leaders use certain gestures and facial expressions to convey their meanings with a personal touch - Barack Obama wouldn’t have won the 2005 elections without it (Bligh, M. C., & Kohles, J. C., 2009) - these musicians use movements to complete their sonic events and convince the audience of their musical interpretations. The result: standing ovations after every concert.

Being a professional clarinet player myself, I started to wonder how certain body movements of a musician influence the opinion of the listener. Could it be that this physical activity is actually a distraction from the music, but that it is simply very impressive? Playing a musical instrument in front of an audience is, after all, difficult enough and showing that you can add extra movement to the required motion to play that instrument might leave the audience in awe. Or does moving to the music add up to the amount of experienced expressivity? And which amount of movement would then be appropriate for Bach? And for Brahms?

All this became the subject of my master thesis for the master Art Sciences, at the department of Musicology, at the University of Amsterdam (UvA). An experiment was developed in which both professional musicians and non-professionals listened to and watched short music recordings and rated them afterwards on different components. The key question was: what is the effect of musicians’ body movements in the valuation of a musical performance?

5

Abstract

When people are able to select the winners of a music competition based on silent video recordings (Tsay, 2013), one wonders if it is sound that is the most important component of a musical performance. To test the influence of musicians’ body movements on a musical performance, we designed an experiment in which clarinet players recorded musical fragments in different ways, increasing the amount of body movement per fragment. After each fragments, both professional clarinet players and non-professionals were asked to rate the fragments on quality, preference, expressivity and naturalness. Participants were first presented with an audio-only version of the stimuli, followed by a video-only condition and finally an audio-video part.

The findings showed that visual information overruled auditory information in the valuation of a musical performance. Increases in physical activity resulted in higher expressivity-, likeability- and quality- ratings. However, significant differences between clarinettists and non-clarinettists were found, indicating that professional clarinet players were better at judging the exact amount of expressivity and were more sensitive to fluctuations in the quality level as a result of a performer’s body movements.

6

1. Introduction

Everybody knows the image of a piano player who wildly wags on his piano bench, his hair flying up to the roof of the concert hall, while he throws his arms into the air. The same goes for the opera singer: with a dramatic look on her face, she lifts her arms when she reaches the highest notes. Or the violinist, using his bow like a sword. Gestures and facial expression of the performer are an inseparable part of music making (Wanderley et al., 2005 ; Leech-Wilkinson, 2006). Within them, a distinction can be made between

primary movements, reflecting the performers interpretation of the music and which are

introduced intentionally for aesthetic reasons, and secondary movements on the other hand (Thompson & Russo, 2007 ; Thompson, 2009). The latter category is a result of the physical demands of playing a certain instrument. A cellist cannot create sound without moving his bow, a piano player will have to move his hands to press the piano keys. However, body movement is not only an inevitable part of playing a musical instrument, it also adds to the performance. Research shows that listeners that hear a musician play and see him/her move as well, believe the performance to be of a higher quality level than listeners who base their judgment solely on audio input (Rodger, 2012). The added visual component is an important factor in the communication of meaning in music and it increases the authenticity of a concert and the credibility of a performer (Platz & Kopiez, 2012). These findings are support by a recently published and striking research outcome: people are able to select the winners of several music competitions by watching their muted performance, whereas both trained and untrained listeners were not able to do so in an audio-only or an audio-visual condition (Tsay, 2013). It is time to take a closer look at the role of the musicians’ body movements in a musical performance.

1.1 Evaluating a musical performance

Already from early on in childhood, humans are able to interpret incoming sounds and information coming from visual images. Researchers report evidence of 3.5 month old babies being able to detect emotions in familiar – parental - voices (Kahana-Kalman & Walker-Andrews, 2001) and babies 7-months of age that can recognise emotional incongruence between voice and face when presented with congruent or incongruent face-voice stimulus pairs (Grossmann et al., 2006). As we can see, the training of ears

7 and eyes already starts at early age.

This musical training continues. Infants’ constant exposure to sound makes them become familiar with the music of their own culture and country. When reaching school going age “Western children begin to respond to music in ways that reflect their exposure to Western music, much like the responses of adults” (Thompson, 2009, p.89). They learn to detect changes to tonal melodies. By the age of seven a sense for key and harmony is fully developed (Trainor & Trehub, 1994) and children start forming the same

expectations as adults do.

There is a vast amount of literature describing how both young and adult listeners react to a musical performance. A great deal of this literature focuses on music and emotion and on how listeners perceive the emotions and musical meaning that are conveyed by the performer. The most common research topics in this area include the listening state of the listener, the expectations certain music elicits and the anticipation or violation of these expectations and the issue how emotions relate to cognitive processes. The listener and the performed music are the most important subjects of this research. It is not until recently, that the focus is being drawn from the musical performance as a sonic event, towards the performer, the one that interprets the works of a composer and brings music to life for the audience. These recent studies especially focus on the ways in which a musician performs music on stage.

1.2 Seeing is believing

It all starts from the moment that the musician enters the stage. As in real life, the first impressions last in the concert hall as well. What is the performer wearing? How confident does he or she look? Appropriateness of dress style and firmness of step after all influences the valuation of the music performance by the listener (Platz & Kopiez, 2013). From the moment the curtains open until the last note, it is not only sound that matters. Ears and eyes are all involved in experiencing a concert. In the year 2000, researchers from the Division of Biology from the California Institute of Technology joined forces with the Japanese Human and Information Science Laboratory in Kanagawa and published an article in Nature with the title: “What you see is what you hear”

(Shams, Kamitani & Sjimoio, 2000). They found that auditory and visual perception is linked to each other and that a visual illusion can be induced by sound. They set up an

8 experiment in which participants had to look at a black background on which white flashes were projected. These flashes were accompanied by beeps and the task was to rate the amount of flashes that was presented. The outcome of the experiment: when a single visual flash was accompanied by multiple auditory beeps, the single flash was incorrectly perceived as multiple flashes. Translating these findings to music suggests that, if

(ambiguous) visual motion is influenced by auditory stimulation, listeners’ musical experiences will vary if listeners are presented with auditory input only, with visual information only, or with input from both domains. More concrete: attending a live concert results in a totally different experience than listening to a CD recording. Another famous example in the literature that shows the importance of the visual component in a presentation is called the McGurk effect. It entails that we tend to perceive an auditory “baba” combined with a visual “gaga” as “dada” (McGurk & McDonald, 1976). This shows that visual information strongly influences what the ears conceive. It points in the direction that body movement might change the musical meaning that musicians convey through sound. Thus, the audience might be confused if the physical activities of the musician are not congruent with the music. Many musicians perform with movements that are incongruent with the music for a great deal of the time, according to Igor Strawinsky (Strawinsky, 1947). As the famous Russian composer states in one of his six Harvard lectures in 1939:

‘[…] Training in these matters is not available at music schools and conservatories,

since these institutions don’t aim at teaching good manners: a violin teacher will scarcely point out to his students that it is inappropriate to play with one’s legs wide open. It is peculiar that these things are not taught anywhere in the world. Whereas all social activity is dominated by codes and standards for decency and good manners, performing musicians don’t seem to know the most fundamental rules for musical courtesy and stage behaviour; namely, pure and decent savoir-vivre.’

Indeed, physical movement is an important part of a musical performance. It helps to convey structure and it helps to convey emotion (Dahl & Friberg, 2004 ; Dahl & Friberg, 2007) and although music is generally thought of as an art based on sound, rating

musicians’ activities purely on their sonic output would be incomplete and unfair (Bergeron & Lopes, 2009). Chai-Jung Tsay found supporting evidence for above mentioned statements, showing that the visual component in music is even much more

9 decisive than the auditory one when determining overall ratings by the audience (Tsay, 2013). He designed an experiment in which professional and non-professional musicians were presented with 6-second clips of the top three finalists in each of 10 prestigious international classical music competitions and in which these participants were asked to point out the winners. Stimuli were given to the participants in an audio-only condition, presenting sound recordings without video, in a video-only condition, showing the performances of the competitors with muted sound, and a video-plus-sound version. Untrained participants were expected to do no better than chance level (33%), since it is very difficult to distinguish between performers that play at such a high level. On the other hand, the professional musicians were expected to score better at the test.

The following results were found: non-professional musicians performed below chance level with the sound-only recordings (28.8%), which was expected because of the above mentioned reason. They performed approximately at chance with video-plus-sound fragments (35.4%). However, novices did very well in the silent video-only recordings: in 46.6% of the cases they correctly appointed the winner of the competition.

Surprisingly, musical experts performed in the same way: they also identified the winner correctly below chance level in the audio-only condition (25.7%). When provided with both sound and video, they performed almost at chance (29.5%). However, when shown video without sound, professional musicians as well performed better and far above chance level (47.0%). Both groups did not significantly differ from each other, pointing in the direction that a combination of audio and video may be distracting, but for sure that visual information is more important (persuasive?) than commonly acknowledged. Australian researchers Mary Broughton and Catherine Stevens confirmed the importance of the video component in their research on marimba performance. A marimba is a percussion instrument, with limited means for communicating expressiveness. This has, as with a lot of percussion instruments, to do with its lack of possibilities for change in timbre, intonation and duration of notes (Broughton & Stevens, 2008). To counterbalance this lack of sonic expressivity, marimba performers rely heavily on physical movements while performing in order to communicate musical expressive intention. In the

experiment of Broughton and Stevens, the marimba players were asked to perform the musical fragments in two ways: a version in which they moved as little as possible (named the “deadpan version”) and a version in which they were asked to move normally or as they would do in a concert situation (named the “projected version”). An

10 experiment was conducted in which both musically trained and untrained participants had to watch short fragments of marimba performances on a computer and rate them

afterwards on expressiveness and level of interest. These performances were presented in an audio-only and audio-video condition.

Results showed that higher ratings were given for the “projected” fragments compared to the deadpan equivalents. As well, higher ratings were assigned to the audio-video fragments compared to the audio-only versions. The same comparison could be found in the interest ratings, showing that it is much more interesting to both see and hear a musician perform.

Three years later, Jennifer Huang and Carol Lynne Krumhansl from Cornell University in the USA expanded the above mentioned research by Broughton and Stevens (Huang & Krumhansl, 2011). A professional pianist was asked to perform three different pieces from three main eras in the history of classical music: the baroque, the romantic and the modern period. With this experimental set-up the researchers added a focus on the influence of style of the composition. Besides this factor, they added a ‘neutral’

performance version – a version in which the performer tried to play naturally and focus on the music instead of on his body movements – to the ‘deadpan’ and ‘projected’ version. Again an experiment was conducted in which both musically trained and untrained participants had to watch short fragments of the recorded performances on a computer and rate them afterwards. This time participants were presented with questions concerning the structural aspects of music (e.g. musical form), emotions (e.g. the amount of emotion expressed) and summary measures (e.g. overall rating and appropriateness of interpretation). Musical fragments were shown in a purely audio and an audio-plus-video condition.

Forty-eight participants took part in the experiment, divided equally into a professional musician and a novices group. Results showed a significant interaction between musical style and stage behaviour. The average overall ratings for music from the baroque and romantic era increased aligned with the increasing degrees of stage behaviour. On the contrary, music from the modern period was rated highest when performed ‘naturally’. Huang and Krumhansl report that this effect could have been caused by the less clear musical structure of the modern music composition (composed by Aaron Copland), compared to the baroque (composed by Johann Sebastian Bach) and romantic (composed by Frédéric Chopin) compositions. Ratings on the structural aspects

11 of music did not significantly differ between the audio- and audio-video condition,

showing that differences in for example form, melody, harmony and phrasing were perceived equally in both modalities. On the other hand, ratings of emotions and summary measures were lower for the audio-visual modality. The paper by Huang and Krumhansl cites earlier work by Carol Lynne Krumhansl (Vines, Krumhansl et al., 2006), providing supporting evidence for the suggestion that musical structure, but not

emotional meaning, is conveyed well across different modalities. This effect is explained in the article by looking at the ratings of the novices group: non-musicians assigned lower ratings on the emotion scales due to uncomfortable feelings caused by their unfamiliarity with certain stage behaviours of the pianist.

1.3 Research questions and hypothesis

The research by Broughton and Stevens (2008) and Huang and Krumhansl (2011) forms the basis for the current study. As we want to know how body movements of a performer influence the audiences listening experiences (Research question 1), we focus on the comparison between the visual and auditory input of a musical performance. By adding, next to the audio-only and the audio-video condition (Broughton & Stevens, 2008 ; Huang & Krumhansl 2011), a video-only condition as well, ratings for these three modalities (AO = audio-only, AV = audio-video, VO = video-only) can easily be

compared. We expect non-professional musicians’ overall ratings to increase as physical movement increases. Moving a lot on stage will look impressive and helps the performer convey emotion and musical meaning (Thompson et al.,2005 ; Thompson & Russo, 2007), resulting in a higher valuation. On the contrary, we expect professional clarinet players’ quality-, preference- and naturalness ratings to drop as musicians move more than they would do in a ‘natural’ performance. Clarinettists are taught at conservatories that moving too much is a bad habit and they will be annoyed by movements that undermine basic clarinet techniques such as steady air support, control over the mouthpiece and relaxed fingering.

The experimental set-up with three modalities resulted in another research question (Research question 2): “Does the modality in which the musical fragments are presented result in the same experienced expressivity, quality, likeability and naturalness?”. We expect that non-professional listeners’ ratings will not show differences in the audio-only

12 fragments with the four increasing amounts of movement. The differences between these audio-only fragments will be too subtle to hear for non-professional listeners. However, we expect clarinet players to hear these quality and expressivity differences caused by the changing amount of motion of the instrument. Furthermore, we expect expressivity ratings of non-clarinettists to be higher in the audio-video condition compared to the audio-only modality. As comparing skilled musicians purely by sound is a difficult task for non-professional musicians, an important role is attributed to visual input as a defining factor.

The final research question (Research question 3) was: “Do ratings of instrumental specialists drop when the amount of physical movement of a performer increases? I.e. what are the differences in rating of various versions of playing between professional and non-professional listeners?”. This research questions results from the hypothesis that it is not only general musical knowledge, but very specific knowledge of a certain instrument that influences the valuation of a musical performance. Being a specialist on a certain instrument, would let one see the differences between expressive primary movements and the unavoidable secondary movements better than untrained listeners. Exaggerated

movements could be then be thought of as annoying and technique undermining with the result of performance ratings going down.

13

2. Experiment 1 (pilot study)

2.1 Background

The marimba research by Broughton and Stevens (2008) included an experiment in which musically trained and untrained participants were asked to rate 20th century solo marimba excerpts on expressiveness and interest. These excerpts were presented in auditory-only and an auditory-visual presentations. Compared to the research by

Broughton and Stevens a few important changes in the experimental design were made. Instead of choosing the marimba, or the piano (Huang & Krumhansl, 2011), as main instrument again, we assigned that role to the clarinet. The clarinet is a wooden wind instrument that, compared to the marimba and the piano, allows the musician a lot more possibilities for change in timbre, duration, intonation and articulation of tones. Apart from that, clarinet players can freely move their body while playing and are able to face the audience during their performances, since they are not restricted to sitting on a piano bench. Finally, the clarinet is a lot more well-known than the marimba, having a more prominent role in many musical genres and making it therefore very suitable for the current study.

As well as in the previously described studies, in this experiment short fragments were recorded in which (both a male and female) artists performed different versions of the same piece. The “deadpan” and “projected” version were supplemented by a “version in a big concert hall” and a “version for children. The current pilot study purposely adds a visual-only part, based on the research results by Tsay (2013) that underlined the

importance of this visual component. Participants were not only asked to rate the musical fragments on expressiveness and preference (an equivalent of interest), but to rate the musical quality and naturalness of the performance as well. The aim of these questions was to investigate if unnatural or over-exaggerated performances would still be

experiences as expressiveness and of high quality. The professional musician group consisted of clarinet players only, in order to find out if professional knowledge and own experience in playing a certain instrument would influence the quality ratings when body movement of a musician increases. Short musical pieces containing a clear musical structure were chosen, so both participant groups would be able to ‘understand’ the music equally well (see Huang & Krumhansl, 2011).

14 2.2 Methods

2.2.1 Participants

The pilot study consisted of two experiments. For the first experiment (Experiment 1a), 6 participants were divided into two groups: Group A consisted of professional clarinet players (n=3), who completed their studies at a conservatory or were studying at a Dutch conservatory. Group B consisted of participants who neither had ever played the clarinet, nor had ever had any extensive musical training (n=3). Participants in group B did not take instrumental lessons in the last four years and had never attended a conservatory. For the second experiment (Experiment 1b), no distinction between groups was made. Participants were musicology students at the University of Amsterdam (n = 17), who participated in the experiment as a part of the bachelor course ‘Culturele en Cognitieve Musicologie’.

2.2.2 Stimuli and tasks

Participants watched and/or listened to 36 fragments of a musical piece and were asked to rate them on a 7-point Likert scale on quality, expressiveness, naturalness and

appreciation. Participants were also asked to comment on the performance style, for example on how relaxed the performer had occurred to them. Pilot experiment 1 included three conditions: an audio-only, a video-only and an audio-video version. Each of the three conditions contained fragments of three musical pieces, taken from relatively unknown clarinet etude books by Karl Baermann (Baermann, 1864) and Eugene Bozza (Bozza, 1953). There was a slow fragment with a lot of legato slurs (Figure 2.2.2.1), an intermediate one with many repetitive notes (Figure. 2.2.2.2) and a fast fragment with a lot of leaps and staccato notes (Figure 2.2.2.3). They were recorded by a professional, yet as well relatively unknown, clarinet player, who studied at the Conservatory of Utrecht, in four different ways:

1. a ‘deadpan’ version, asking the performer to stand as still as possible

2. a ‘normal’ version, asking the performer to think of nothing and just play the piece 3. a ‘concert hall’ version, asking the performer to pretend playing in a big concert hall 4. a ‘children’ version, asking the performer to pretend playing for little children

15

The four versions corresponded with four different ways of performing. The ‘normal’ version functioned as a control situation, in which the performer played free of thoughts as he or she would do when practising at home. The ‘deadpan’ and the ‘concert hall’ version are two contrasting versions, both on one side of the control situation. In the ‘deadpan’ version the performer tried to play the music fragment as well and beautifully as possible, while avoiding all unnecessary movement. In the ‘concert hall’ version the performer tried to perform in the same way, but while imagining standing in a big concert hall in front of a large audience. Finally, there was the ‘children’ version. The performing clarinettist was asked to forget about clarinet technique and traditional ways of performing and play as expressively as possible to grasp the attention of the ‘imaginary’ children.

The clarinet player played the 15-second fragments by heart and played along a muted metronome that was located next to a video camera, thus guaranteeing a steady tempo of the recordings. Both video and sound were recorded, in such a way that a separate video-only, audio-only and audio-video condition could be created. The clarinettist was standing in front of a neutral coloured background and was wearing white clothes to create a clear contrast with the black clarinet. The recordings were edited in Windows Movie Maker, so that they would start right before the first sound and ended right after the last note.

Fig. 2.2.2.1: fragment from Carl Baermann’s Etude no. 29

Fig. 2.2.2.3: fragment from Eugene Bozza’s Etude no. 10 Fig. 2.2.2.2: fragment from Eugene Bozza’s Etude no. 11

16 This all resulted in a 3 (pieces (Baermann’s Etude No. 29, Bozza’s Etude No. 10 and Bozza’s Etude No. 11)) x 3 (presentation mode (Audio-Only, Audio-Video and Video-Only) x 4 (playing version (“Deadpan”, “Normal”, “Concert hall”, “Children”) design and, consequently, 36 video/audio fragments. Within the three presentation mode conditions, musical pieces and four performance versions were presented in a random order to counterbalance for the learning paradigm. The experiment started with all the fragments from the audio-only condition, to prevent participants from having an association of performance style, since they had not yet seen the performer.

For Pilot experiment 1, all participants signed an informed consent form before taking part in the pilot experiment and were asked to state that they had normal hearing and normal vision. Participants were provided with high quality Sennheiser headphones and sat in a quiet room in front of a laptop. After they had been given time to read the instructions on the opening screen, any remaining questions were answered and the experiment began. Participation in the experiment took approximately 25 minutes per participant.

For Experiment 1b, no informed consent forms were signed. Participants filled in a questionnaire about their personal background and musical experience. They were only presented with the 8 fragments from the audio-video condition that had been shown in Experiment 1a. The video-only and audio-only fragments were not shown, due to a lack of time. The sound-plus-video fragments were presented to all participants at the same time, using a beamer and a large white projection screen. Participants were asked to rate the expressiveness, their preference and the naturalness of the fragments on a 6-point Likert scale.

2.3 Results

Fifty percent of the participants in Experiment 1a (n = 6) was female, as was 82.4 % of the participants in Experiment 1b (n = 17). The average age in the participant group of Experiment 1a was 37.7 years. The average age in the participant group of Experiment 1b was 21.5 years (SD = 2.0 years). Experiment 1a was mainly used to test the software and check if the participants were at ease while doing the task, whereas Experiment 1b effectively aimed on gathering pilot data. Therefore, the boxplots presented below show

17 an overview of the average ratings given in Experiment 1b only. No statistical analysis was made of the observed data.

2.3.1 Experiment 1a

In the AO (audio-only) condition of Experiment 1a, participants assigned a mean

expressivity rating of 4 points (on an 8-point Likert scale) to performance version 1. A

mean expressivity rating of 5 points was assigned to versions 2 and 3. A higher mean rating of 6 points was given for Version 4, indicating that the performance version with the highest amount of body movement sounded the most expressive. In the VO (video-only) condition, participants assigned a mean expressivity rating of 2 points (on an 8-point Likert scale) to performance version 1. Versions 2 and 3 were given a mean rating of 4.7 and 5.5 points respectively and Version 4 received on average 6 points. This as well indicates that performances are experienced as more expressive as the amount of physical movement increases. In the AV (audio-video) condition version 1 was on average rated with 3 points, Version 2 was assigned a mean rating of 4 points, Version 3 was on average rated 5.5 points and Version 4 received a mean expressivity rating of 6 points. Again, expressivity ratings increased aligned with the performance versions. In the AO condition of Experiment 1a, participants assigned a mean likeability rating of 5 points (on an 8-point Likert scale) to performance versions 1 and 3. Versions 2 and 4 were assigned 5.5 and 6 points respectively. In the AV condition, participants again assigned the same mean ratings to versions 1 and 3 (4.6 points) and higher ratings to versions 2 and 4 (Versions 2: 5 points, Version 4: 6 points). This indicates that

participants both like a “normal’ versions and an extreme and exaggerated version for

children.

In Experiment 1a, participants as well rated the quality of the musical performances, while answering the question “What did you think of the overall quality?”. Versions 1 and 2 were rated highest in the AO condition, with a rating of 5.5 points (on an 8-point Likert scale). Versions 3 and 4 were assigned an average rating of 5 points. This indicates that performances in which musicians only use a small amount of body movement sound better. In the VO condition, Version 2 (the “normal” version) was rated highest, with a mean rating of 5.5 points. Versions 3 and 4 were assigned on average 5 points and

18 Version 1 was rated lowest with 3.5 points. This indicates that performances with a ‘regular’ amount of body movement suggest a high performance quality. However, if we look at the results in the AV condition, we see that Versions 2 and 4 are both rated on average with 5.5 points. Compared to the VO condition, Version 1 as well is rated higher, with 5 points on the 8-point Likert scale, but Version 3 receives on average only 4 points. Finally, participants answered questions about the naturalness of the performance. In the VO condition Version 1 was rated lowest, with 2.5 points on an 8-point Likert scale. All other versions received higher mean ratings: versions 2 and 3 were rated 5 points and Version 4 received a mean naturalness rating of 5.4 points. This suggests that standing totally still does not look natural at all and that performing with an extreme amount of body movement is not that unnatural.

2.3.2 Experiment 1b

Figure 2.2.1 shows the mean expressiveness ratings of the AV-condition in Experiment 1b (n = 17). The numbers at the x-axes indicate the four different versions (1: deadpan, 2: normal, 3: concert hall and 4: children), and the y-axis indicates the 6-point Likert scale. The boxplot shows that the mean expressiveness rating of the recordings went up as the clarinet player moved from the ‘deadpan’ version towards the ‘children’ version and

Fig. 2.2.1: The mean expressiveness ratings for the AV (audio-video) condition in Experiment 1b. The numbers on the x-axes represent the versions in which the musical fragments were played: 1 = ‘deadpan’, 2 = ‘normal’, 3 = ‘concert hall’, 4 = ‘children’.

19 body movement increased. This indicates a very important role of visual information, as the physical action of the performer seems to have mainly determined audiences ratings.

Figure 2.2.2 shows the results on likeability of the performances. Higher grades indicate a greater preference for a recording, whereas the lower marks indicate a lower preference for a recording. This figure indicates that participants mostly preferred the ‘normal’ version of the fragments and shows that the average rating of version 1, 3 and 4 was quite low. Combining this result with the expressiveness outcome suggests that participants do not necessarily like the performances with the highest ratings for expressiveness best.

Fig. 2.2.2: The mean likeability ratings for the AV (audio-video) condition in Experiment 1b. The numbers on the x-axes represent the versions in which the musical fragments were played: 1 = ‘deadpan’, 2 = ‘normal’, 3 = ‘concert hall’, 4 = ‘children’.

Fig. 2.2.3: The mean naturalness ratings for the AV (audio-video) condition in Experiment 1b. The numbers on the x-axes represent the versions in which the musical fragments were played: 1 = ‘deadpan’, 2 = ‘normal’, 3 = ‘concert hall’, 4 = ‘children’.

20 Figure 2.2.3 shows the answers to the question “Did the performance occur natural to

you?”. It shows that the ‘deadpan’ version was rated least natural and that the ‘normal’

version was rated most natural. No difference between average rating of naturalness between Version 3 and Version 4 can be seen. The variation of ratings does seem high, with ratings varying between the lowest and highest possible grade. These findings are in line with the set-up of the experiment, in which musicians perform naturally in Version 2 and exaggerate their physical movements in the other versions.

2.4 Discussion

The participants of Experiment 1a reported that the software was working properly and that they were at ease while doing the tasks. However, they reported having troubles commenting about the performance style of the musician after every fragment in the VO and AV conditions. As these questions were implemented only to keep participants’ attention to the screen and as we saw participants were paying attention properly, we decided to remove these questions in the follow-up experiments. Furthermore, a short sound check was added before the start of the main experiment, to make sure all participants listened to the fragments at the same volume level.

The results of Experiment 1a are a contraction of clarinettists’ and non-clarinettists’ ratings and can therefore not provide a preliminary answer to Research question 3 (Do

ratings of instrumental specialists drop when the amount of physical movement of a performer increases?). However, the outcome suggests an important role for both the

amount of physical movement in a musical performance (Research question 1) and for the modality in which a fragment is presented (Research questions 2. We should consider though that only six participants participated in this first pilot study, making the results of a low statistical value.

The musicology students in Experiment 1b provided us with different ratings for each of the four versions and showed that physical movement - and thus visual information - seems to be an important influencer of how we perceive a musical performance. The results are in line with the expectation that performances with the highest amount of body movement are experienced as most expressive. The ‘normal’ versions were rated most natural, which is in line with the hypothesis as well. However, the most expressive versions were not the best liked performances. This is a surprising finding, as we

21 expected the most interesting and touching versions to be rated highest.

These preliminary results encouraged us to adapt and improve the pilot experiments and start testing a larger group of participants.

22

3. Experiment 2 (main study)

3.1 Research questions

As Experiment 1a and Experiment 1b proved and predicted that our experimental design was and would be suitable for testing the three original research questions, we decided not to adapt these questions. Therefore, research questions still were: “How do body movements of a performer influence the audiences listening experiences?” (Research question 1), “Does the modality in which the musical fragments are presented result in the same experienced expressivity, quality, likeability and naturalness?” (Research question 2), and “Do ratings of instrumental specialists drop when the amount of physical movement of a performer increases” (Research question 3).

3.2 Experimental design

Compared to pilot experiments, a few changes in the design of the final experiment were made. The amount of different musical pieces was reduced from three to two. This was due to the quality of the material recordings. The fragment that had been discarded was a difficult one to play correctly with different expressive versions and with a lot of body movement. We thus selected the slow fragment with long legato lines and the fast fragment with many staccato notes, which were easier to play correctly in the different expressive versions. Furthermore, we added another clarinet player, so that both a female and a male musician performed in the recordings. By doing so we also added an extra factor apart from gender: the clarinet players performed at a different level, since the female clarinet player studied two years at a conservatory and the male player completed nearly seven of musical study. This difference was expected to have an effect on the experienced quality level in the professional participant group. However, we were interested if this effect was present in all modalities and if expressiveness and likeability ratings were affected as well. The former clarinettist took care of the fast fragment, while the latter played the slow one.

Participants were asked to rate the fragments on an 8-point Likert scale, instead of on a range of six or seven. By doing so we avoided an uneven entirety (7) and since

23 that particular scale with the middle point we forced them to choose more towards the positive of negative direction on the scale. As for the 6-point Likert scale: changing the scale to 8 points broadened the spectrum and allowed participants to express their feelings of deep disapproval or strong affection. The questions were:

1. How natural did this performance occur to you? (not natural at all – very natural) 2. Did you like this performance? (not at all – very much)

3. What do you think of the overall quality of this performance? (very poor – very good)

4. How expressive was this performance? (not expressive at all – very expressive) After the audio-only condition we did not ask a question about the naturalness of the performance, as this question could occur as too vague to the non-professional participant group. Neither did we ask about the overall quality after the video-only condition, since participants were instructed to answer the quality question based only on their judgement of the sound of the performance.

A total amount of 30 participants took part in the experiment, again divided into professional clarinet players (n = 15) and participants who neither had ever played the clarinet, nor had ever had any extensive musical training (n = 15). All participants received an information brochure, signed an informed consent form and filled in a short questionnaire about their personal details. Participation in the experiment took about 30 minutes.

Fig. 3.1.1: an example of one of the questions shown to the participants after the fragments.

24 3.2 Results and discussion

Sixty percent of the participants in the non-professional musician group was female, as was 53 percent in the professional group. The average age in the non-clarinettist group was 28.5 years (SD = 7.5 years), against 23.9 years (SD = 3.5 years) in the clarinettist group. In the non-professional group, one third of the participants obtained a bachelor’s degree at a university and two thirds finished their master’s. In the clarinettist group, one third of the participants received a high school diploma, one third obtained a bachelor’s degree and one third finished their master’s. Non-professional clarinet players had never played the clarinet and stated that they had not been practising another musical

instrument for more than 1.5 hours a day in the past year. Clarinettists had practised on average 3.4 hours per day in the past year (SD = 0.9h).

3.2.1 Expressiveness

Fig. 3.2.1: The mean ratings of expressiveness with standard error bars. AO (audio-only), AV(audio-video) and VO (video-only) represent the different conditions, the colours display the different versions of the fragments, J and L stand for the fast and the slow music fragments played by different performers each and the 0 and 1 at the bottom represent the professional (1) and nonprofessional (0) participants.

25

Expressiveness – Song = “J”

A three-way mixed-model ANOVA with Versions (1, 2, 3 and 4) and Modality (AO, VO and AV) as within subject factor and Clarinettist (clarinettist vs. non-clarinettist) as between subject factor on the Expressiveness rating for the fast music fragment (Song = J) revealed a strong and significant main effect of Version (F(3,84) = 82.5, p < .001) and of Clarinettist (F(1,28) = 8.5, p < .001), as well as a significant interaction between Modalities and Version (F(6,168) = 18.8, p < .001) and between Clarinettist and Version (F(3,84) = 4.0, p = .007).

Further simple effect analyses on the interaction between Clarinettist and Version indicated that clarinettists rated Version 4 significantly higher than Version 3 (p < .05), whereas non-clarinettists did not do so. This effect can clearly be seen in all three columns in the top row of Figure 3.2.1, and suggests that clarinettists detected significantly more differences in physical movement than non-clarinettist did. This difference can be explained by the clarinettists’ knowledge of how to play the instrument, which allowed them to better differentiate between primary and secondary movements.

Another finding was that non-clarinettists gave higher expressivity ratings than clarinettists for the expressive versions of 2, 3, 4 (p < .01), but not for Version 1. This suggests that clarinettists were more critical about the recordings. The lower ratings indicate that clarinettists had let their expressiveness ratings be influenced by their quality and/or naturalness ratings, especially since this interaction is absent in the slow music fragment “L”, played by a more skilled clarinet player.

The non-clarinettist group assigned significantly higher ratings to Version 3 than to Version 2 (p < .05), while clarinettists did not do so.

Simple effect analyses on the interaction between Modalities and Version indicated that the mean ratings for the expressive versions did not differ in the AO condition for both groups, while they differed significantly in the AV and VO modalities (p values < .01 – .05). This can be seen in the AO part in the top row of Figure 3.2.1, where mean ratings of both clarinettists and non-clarinettists hardly differed between versions in the audio-only condition. The effect could be explained by the high tempo of the music fragment, making it more difficult to perform expressively while moving a lot. It could also be the case that fast music is less easily experienced as expressive with auditory input only or

26

that the quality of a performance influences the expressiveness ratings.

Furthermore, where Version 1 scores about as well as the other versions in the audio-only condition, ratings of both participant groups dropped when video is added (p < .001). This indicates that visual information overrules sonic events, since expressivity ratings drop for the static performances when video is added, but increase rapidly when the performer moves more. There was no difference in the mean expressivity rating found between Versions 3 and 4 in the VO condition and between Versions 2 and 3 in the AV condition.

Expressiveness – Song = “L”

A three-way mixed-model ANOVA with Versions (1, 2, 3 and 4) and Modality (AO, VO and AV) as within subject factor and Clarinettist (clarinettist vs. non-clarinettist) as between subject factor on the Expressiveness rating for the slow music fragment (Song = L) revealed a strong and significant main effect of Modalities (F(2,16) = 16.5, p < 0.001) and of Version (F(3,84) = 85.5, p < 0.001), as well as a significant interaction between Modalities and Version (F(6,168) = 15.6, p < 0.001).

Since there was no significant effect of Clarinettist (p = 0.27, n.s.), nor interactions related with this factor, findings suggest that trained and untrained listeners rated the expressiveness in music fragment L in the same way. The main effect of Version can clearly be seen in the bottom row of Figure 3.2.1, as the mean rating of expressiveness climbs as the amount of movement in the stimuli increases. For example, if we compare versions 1, 2 and 3, we see very significant increases in expressiveness rating (p values all < .001). However, Version 3 and Version 4 do not significantly differ, indicating that both of these recordings with a lot of body movement are experienced as very expressive.

Further simple effect analyses on the interaction between Modalities and Version indicated a more steep increase in the AV and VO condition, compared with the AO condition. This suggests that participants found it more difficult to judge the amount of expression in music when presented with auditory information only. Comparing the AO results with the AV condition results in the bottom row of Figure 3.2.1 shows that adding video resulted in much more variability in the results, in which especially the lower ratings for Version 1 and Version 2 stand out (p < .05).

27 3.2.2 Quality

Quality – Song = “J”

A three-way mixed-model ANOVA with Versions (1, 2, 3 and 4) and Modality (AO, VO and AV) as within subject factor and Clarinettist (clarinettist vs. non-clarinettist) as between subject factor on the Quality rating for the fast music fragment (Song = J) revealed a strong and significant main effect of Version (F(3,84) = 7.9, p < 0.001), of Modalities (F(2,56) = 4.3, p = 0.02) and of Clarinettist (F(1,28) = 9.6, p = 0.003), as well as a significant interaction between Modalities and Version (F(6,168) = 10.4, p < 0.001), as well as an interaction between Clarinettist and Version (F(3,84) = 4.0, p = 0.005) and

Fig. 3.2.2: The mean ratings of quality with standard error bars. AO (audio-only), AV (audio-video) and VO (video-only) display the different conditions, the colours represent different versions of the fragments, J and L stand for the fast and the slow music fragments played by different performers each and the 0 and 1 at the bottom represent the professional (1) and nonprofessional (0) participants.

28

between Clarinettist and Modalities (F(2,56) = 3.8, p = 0.02). As there can be seen in the top row of Figure 3.2.2, in the audio-only condition clarinettists’ ratings dropped as the expressiveness level increased (p values all < .05). This is in line with the hypothesis attached to Research question 3, since playing and moving is from a clarinet-technical point of view inadvisable, especially in fast music fragments. The findings indicate that experienced clarinet players detected tiny errors (e.g. unevenness, small tempo changes) in the most moved recordings, explaining the decreasing mean rating. Non-professional clarinettists did not detect these errors, since Version 4 is rated highest by them in the audio-only condition, whereas that version is rated lowest by clarinet players (p < .05). The hypothesis would account as well for the finding that there was no effect of Clarinettist in the slow recording, since playing and moving to slow music is less difficult than combining fast music with a lot of body movement.

Further simple effect analyses on the interaction between Modalities and Version indicated that the ‘deadpan’ performance (Version 1) was rated lower as video (VO or AV) was added to the audio (respectively p < .05 and p < .01). A large significant

difference between versions 1 and 2 can be seen in the VO modality (p < .05), suggesting that standing as still as possible resulted in a lower experienced quality level than normal or even (over-)exaggerated movement (all p values < .05).

Simple effect analyses on the interaction between Clarinettist and Modalities indicated that clarinettists and non-clarinettist rated the VO condition in the same way. This suggests that it is not musical training, but exposure to music in everyday life that underlies the skills to correctly valuate visual input of a musical performance.

Quality – Song = “L”

A three-way mixed-model ANOVA with Versions (1, 2, 3 and 4) and Modality (AO, VO and AV) as within subject factor and Clarinettist (clarinettist vs. non-clarinettist) as between subject factor on the Quality rating for the slow music fragment (Song = L) revealed a strong and significant main effect of Modalities (F(2,56) = 13.8, p < 0.001) and of Version (F(3,84) = 26.4, p < 0.001), as well as a significant interaction between Modalities and Version (F(6,168) = 6.2, p < 0.001).

29

indicated that overall quality ratings in the audio-only condition dropped when

participants were presented with visual stimuli only and decreased slightly in the audio-visual condition. This can be seen in the comparison between the AO and the VO

condition in the bottom row of Figure 3.2.2. This effect indicates that an unappreciated or inappropriate (see Strawinsky, 1947) visual performance lowers the experienced quality of a musical fragment.

As there can be seen in the video-only column (VO), mean ratings increased for both clarinettists and for non-clarinettists towards the ‘concert hall’ version (p values < .05), appointing Version 3 as the recording of the highest quality. The mean ratings for Version 4, which contained even more physical movement, were lower (p = 0.3, n.s.). Familiarity of non-professional musicians with (classical) music and the standard way of performing for the clarinettist group could account for this finding.

There was no significant effect of Clarinettist. That is an surprising finding, since the former group was expected not to be distracted by the visual component and expected to stay focussed mainly on the audio. Furthermore, in the AO condition clarinettists rated Version 3 and Version 4 higher than Version 1 or Version 2 (p < 0.5). This is incongruent with the hypothesis for the current study (Research question 3), since too much physical movement should have undermined basic clarinet techniques and especially without the distraction of visual information, clarinet players were expected to be more sensitive for the resulting effects in quality.

3.2.3 Likeability

Likeability – Song = “J”

A three-way mixed-model ANOVA with Versions (1, 2, 3 and 4) and Modality (AO and AV) as within subject factor and Clarinettist (clarinettist vs. non-clarinettist) as between subject factor on the Likeability rating for the fast musical fragment (Song = J) revealed a strong and significant main effect of Clarinettist (F(1,28) = 9.6, p < 0.001), as well as a significant interaction between Modalities and Version (F(6,168) = 10.4, p < 0.001), between Clarinettist and Version (F(3,84) = 4.0, p < 0.001) and as well an interaction between Clarinettist and Modalities (F(2,56) = 3.8, p = 0.025). For this part of the

30

A significant main effect of Clarinettist suggests that clarinettists and non-clarinettists differed in their preferences. Further simple effect analyses on the interaction between Clarinettist and Modalities indicated that clarinet players assigned lower ratings to the fragments in the AV condition, but they did so in the same pattern as non-clarinet players.

However, in the audio-only part both participant groups gave contradicting ratings: as the mean likeability rating increased per version for the non-clarinettist group, mean rating dropped for the professional clarinettists (all p values < .05). A similar pattern can be seen in the mean quality ratings (Figure 3.2.2) as well, suggesting that preference ratings of clarinettists are lower due to a dropping quality level.

Further simple effect analyses on the interaction between Modalities and Version

indicated that the modalities in which a performance version is presented have an impact on the likeability rating. For example, the ‘deadpan’ version is in both modalities the

Fig. 3.2.3: The mean ratings of quality with standard error bars. AO (audio-only), AV (audio-video) and VO (video-only) represent the different conditions, the colours display the different versions of the fragments, J and L stand for the fast and the slow music fragments played by different performers each and the 0 and 1 at the bottom represent the professional (1) and nonprofessional (0) participants.

31

version that is liked least by the non-clarinettists (see the top row of Figure 3.2.3). However, the fairly high rating in the AO modality rigorously drops when video is added (p < .01). Visual information thus overrules again auditory information.

Likeability – Song = “L”

A three-way mixed-model ANOVA with Versions (1, 2, 3 and 4) and Modality (AO and AV) as within subject factor and Clarinettist (clarinettist vs. non-clarinettist) as between subject factor on the Likeability rating for the slow musical fragment (Song = L) revealed a strong and significant main effect of Version (F(3,84) = 26.4, p < 0.001), as well as a significant interaction between Modalities and Version (F(6,168) = 6.2, p < 0.001) as well. No data was obtained for the “VO” condition due to a technical error.

Further simple effect analyses on the interaction between Modalities and Version indicated that the mean likeability rating for Version 1 dropped as video was added to the sound of the clarinettist (p < .01), whereas mean ratings approximately stayed the same for other versions (all p values n.s.). This again underlines an important role for visual information.

In general, performances with a lot of body movement (Versions 3 and 4) were both in the AO as in the AV condition preferred over the still (Version 1) or normal (Version 2) presentations by the participants of the experiment. This is an interesting finding, since the non-professional musician group was not expected to hear much difference in the AO recordings. The significant main effect of Version suggests that the clarinettist truly played differently and more appealingly when she was asked to imagine playing in a concert hall or to perform for a group of little children.

3.2.4 Naturalness

Naturalness – Song = “J”

A three-way mixed-model ANOVA with Versions (1, 2, 3 and 4) and Modality (AO and AV) as within subject factor and Clarinettist (clarinettist vs. non-clarinettist) as between subject factor on the Naturalness rating for the fast music fragment (Song = J) revealed a strong and significant main effect of Version (F(3,84) = 17.1, p < 0.001), as well as a significant three-way interaction between Modalities, Version and Clarinettist (F(3,84) = 5.5, p = 0.002).

32

Further simple effect analysis and the interaction between Modalities, Version and Clarinettist indicated a prominent role for the ‘normal’ version (Version 2). Version 2 was both by clarinet players and non-clarinet players rated as most natural (p < .05). After that surprisingly followed the ‘children’ version, which was expected to be rated least natural, since the performer was asked to forget about the traditional ways of making music and focussed on grasping the attention of little children only. This suggests an influence of participants’ likeability ratings on the rating of this fourth performance version.

In the comparison between the two modalities (AV and VO), ratings significantly went up for non-clarinettists (p values < .05) when sound was added to the video fragments, but ratings dropped for clarinettists. Again this effect could be addressed to the critical attitude of professional clarinet players.

Fig. 3.2.4: The mean ratings of quality with standard error bars. AO (audio-only), AV (audio-video) and VO (video-only) represent the different conditions, the colours display the different versions of the fragments, J and L stand for the fast and the slow music fragments played by different performers each and the 0 and 1 at the bottom represent the professional (1) and nonprofessional (0) participants.

33

Naturalness – Song = “L”

A three-way mixed-model ANOVA with Versions (1, 2, 3 and 4) and Modality (VO and AV) as within subject factor and Clarinettist (clarinettist vs. non-clarinettist) as between subject factor on the Naturalness rating for the slow music fragment (Song = L) revealed a strong and significant main effect of Version (F(3,84) = 21.8, p < 0.001) and Modalities

(F(1,28) = 9.1, p = 0.038). There could not be found any significant interaction. The lack of a significant effect of Clarinettist indicates that professionals and

non-professionals rated the naturalness of a performance in the same way. As seems to be the case with expressivity, expertise and constant exposure to classical music do not result in a significant difference for rating the naturalness of a performance. As there can be seen in the bottom row of Figure 3.2.4, high scores were assigned to Version 2, the ‘normal’ version and to Version 3, the ‘concert hall’ version. Version 1 is rated less natural as well in the comparison with Version 2 (p < .01), and appears to be experienced as too stiff. Version 4 is rated less natural than Version 3 (p < .001) and appears to be experienced as over exaggerated. However, the boxplot shows that even the most extreme performance version, meant for little children, looked more natural that the ‘deadpan’ version (p < .05). Summarized, although not of the highest quality, expressivity or likeability, Version 2 is experienced the most natural.

3.3 Discussion

In general, responses for the slow song “L” were similar between the two participants groups. Responses for the fast song “J” tended to be different, indicating that the level of playing of a performer and/or the speed and character of a fragment had an important influence on the listeners’ ratings. The four different questions asked after each music fragment resulted in different outcome patterns, suggesting that we succeeded in

measuring different processes. A more extensive discussion will be provided in Chapter 4. However, four important outcomes will be summarized here: (1) visual information strongly influenced listeners’ ratings and mostly overruled auditory input, (2) clarinettists performed better at detecting the amount of expressivity and (3) were more sensitive to changes in quality of a performance caused by physical movement of the performer. Finally (4), although not rated most natural, performances with a lot of body movement were generally preferred over still versions.

34

4. General Discussion

4.1 Results

The current study investigated the influence of performers’ body movements on the valuation of a musical performance. Visual information plays an important role in the valuation of a musical performance (Van Zijl & Luck, 2013). Seeing a performer play lifts the performance to a higher quality level (Rodger, 2012), helps the process of

communicating musical meaning (Juslin & Laukka, 2003 ; Juslin & Timmers, 2010) and increases the authenticity of a concert (Platz & Kopiez, 2012).

Based on research by Broughton and Stevens (2008) and Huang and Krumhansl (2011), an experiment was conducted in which both professional clarinettists and non-professional musicians had to rate unknown classical music fragments, played by two clarinettists. These short music fragments were presented in three conditions (an audio-only modality (AO), a video-audio-only modality (VO) and an audio-video modality (AV)) and were followed by questions about the expressiveness, the quality, the likeability and the naturalness of the performance. Ratings had to be given on a 8-point Likert scale. The particular effects found in the present study are specific to the particular performers. The research questions were: “How do body movements of a performer influence the audiences listening experiences?” (Research question 1), “Does the modality in which the musical fragments are presented result in the same experienced expressivity, quality, likeability and naturalness?” (Research question 2), and “Do ratings of instrumental specialists drop when the amount of physical movement of a performer increases” (Research question 3).

As for the main research question (Research question 1), the body movements of a musical performer have a huge impact on audiences listening experiences. A lot of physical movement on stage results in a more expressive experienced performance, with generally high likeability and quality ratings. Seeing a musician play his or her

instrument while moving a lot as well covers some of the tiny errors (unevenness’s, changes in timing due to technical difficulties) caused by undermining the basic playing techniques. Clarinettists performed better in detecting these fluctuations in the quality level, as well as in detecting the exact amount of expressivity of a musical performance.

35

instrument, which allowed them to better differentiate between primary and secondary movements.

Concerning Research question 2, results shows that the modality in which a musical fragment is presented has a huge influence on the experience of the performance. Especially visual information has a large impact on the valuation as expressivity and likeability ratings go up as the amount of body movement increases. Furthermore, ratings for “deadpan” performances that were fairly high in the audio-only condition dropped significantly as video was added, indicating that unnatural visual performance manners overrule auditory input. Next to that, non-clarinettists as well did hear differences in the AO condition, underlining that musical experts are not only found at the conservatory.

If we review Research questions 3, we see that the findings of the current study suggest that musical training and expertise of a certain musical instrument result in a greater concern about the quality level of a musical performance. It is not necessarily the case that quality ratings decrease as physical movement of a performer increases – Figure

3.2.2 shows, apart from the AO condition in the fast music fragment, the opposite

tendency. However, clarinettists assigned significantly lower ratings to the clarinettist that studied two years at a conservatory, compared to the clarinettist that completed nearly seven years. Besides that, the findings shows that clarinettists’ expressiveness- and likeability ratings were affected by the quality level of the performances.

Clarinettists and non-clarinettists generally performed in the same way answering questions on naturalness and expressiveness of the performance. The professional clarinettist group differed only in noticing more quality differences in the audio-only condition of the fast music fragment, as well as expressivity differences in versions 3 and 4 of the video-only condition.

4.2 Strengths and weaknesses

I would like to point out certain strengths and weaknesses in this experiment. First of all, participants liked participating in the experiment, as it was short and entertaining. Almost after every participation, a lively discussion arose about beautiful musical experiences and live performances that had made deep and lasting impressions on participants. Secondly, and very important as well, the chosen questions resulted in clear data and covered the participants’ opinions. The questions were clear and the Likert scales were

36 easily understood. The stimuli were nicely transmitted from the computer to the

headphones and everybody was able to handle the keyboard and mouse of the laptop. Then there were as well considerations for a potential follow-up experiment.

First of all, the structure of the stimuli. Our two clarinet players did not play the same music fragments and performed only one fragment per performance version. This made analysing the data more difficult, as both level of playing and differences between the two compositions could account for the findings. As there was only one fragment per performance version per performer, it could be the case that “normal” versions tended a bit too much towards the “concert hall” or towards the “deadpan” versions. For example, was there a clear enough difference between the “concert hall” version and the “children version”? Apart from that, the results suggested that the level of playing of the performer had a big influence on the ratings, especially for clarinettists. As the quality of a musical performance seems to affect the naturalness-, likeability- and expressiveness ratings as well (see Chapter 3.2), the results from the current study cannot be generalized, since they specifically matched the performance levels of the musicians participating in this experiment.

Secondly, the participants. Two enthusiastic groups of participants (n = 30) agreed to participate in the current study. On one hand a professional clarinettist group, on the other hand a group of musical ‘novices’. But this non-clarinettist group did not consist of people who had never encountered music or had never played any musical instrument. Since exposure to music in everyday life for sure has an effect on ratings, ideally a musically ‘blank’ participant group would have been formed.

Thirdly, the tasks. The order of the questions asked in the experiment was fixed. The first questions was about the expressiveness of the performance, followed by questions about the quality, likeability and naturalness. Should this order have been

counterbalanced for? Could it be that a very happy or angry feeling after a fragment influenced participants’ rating of the question about expressivity, but not the ones following that? Did extreme performances provoke extreme ratings?

37 4.3 Future work

In a potential follow-up experiment, the experimental design could be expanded. It would be interesting to include a large amount of clarinet players, playing a large amount of musical fragments in different performance version. All players should then be playing the same fragments, to control for differences caused by gender or level of playing. Next to that, it would be interesting to test a much bigger group of participants and to add a control group consisting of professional musicians that did never play the clarinet. Thus, it would be possible to find out if our findings are widespread and valid for other

instruments as well.

The results presented in this experiment are of interest for clarinet players in

particular, but for musicians in general and for teachers at the conservatories as well. The influence of body movement of a performer and visual input during a musical

performance should not be underestimated. Music lovers as well could benefit from our experiences: listening to music at home will totally change if a video of the performing artist is added and played simultaneously with the CD recording.

4.4 Conclusion

Exactly one year ago, a good friend of mine said “Listening to a song on Spotify is like watching a soccer game on a television with a broken screen”. The point was: something is missing. That something is visual input and it adds to the sound of the commentary and makes watching a sports game much more exciting. I feel the same in the field of

classical music. As listening to classical music on a CD, on YouTube or on Spotify is very often labelled as ‘boring’ (see Mitchell & MadDonald, 2014 and Thompson & Russo, 2007), I believe that going to a live performance of a classical musician really can be a mind-blowing experience. Musicians are not musical statues, they use body

movement to make the structure of the music clear and convey emotional meaning (Thompson, Graham & Russo, 2005) and they try to take you with them in their

imagination of the music (Juslin & Timmers, 2010). Adding visual input to the auditory information of a musical performance results in a totally different listening experience. Seeing is believing.

38 Acknowledgement

I would like to thank Makiko Sadakata very much for her very kind, helpful and

professional guidance throughout the whole process of writing this thesis. Furthermore, I would also like to thank Janne Weijkamp, for all her advice and help in programming the experiment. Finally, I would like to thank Henkjan Honing for his time and effort for being the second reader of my thesis.