o
o o o
o
o o o o
o o o o o o
o
o
o
o
o
o
o
o
o
o
o
o
o
o o o
o o o o
o o
o o o o o o
o o o
effect("Posterize Time")(1)/(effect("Echo")("Echo Time (seconds)")*-1)
value-thisComp.layer("control").effect("Pitch")("Slider")
o o o
o o o
The support of music education: a visual feedback system overview
By Jasmijn Kruijshaar
Introduction
In the Dutch educational system, the demand for quality music education in primary schools is growing. One of the reasons for this is that introducing children to music at a young age helps them to acquire intellectual, emotional, physical and social skills [1]. Furthermore, music education contributes to the growth of a child’s personality [1]. As a consequence, the importance of music education for the development of children is more and more recognized.
However, many primary school teachers lack confidence while giving music education to students. These teachers feel like they lack the skills and accompanying knowledge on the subject of music. In fact, there is not only a link found between confidence to teach music and musical qualifications, but research also found that teachers felt that their training did not equip them to teach music effectively [2]. This all contributes to the unconfidence of primary school teachers and as a result, the quality of music education is poor if there is any music education at all.
Consequently, teachers should be supported during their training to increase their confidence. One way to achieve this with the help of a supportive visual feedback system, which is why a research needs to be conducted on this topic. Therefore, the main objective of this literature review is to provide an overview of feedback systems designed in context of music education. Hereby, the question “What kind of feedback systems are already designed in music education and what factors make these systems strong and weak?” will form the base of this research. Accordingly, this paper will discuss three visual feedback systems that provide support in music education. From these systems, their strong and weak aspects will be identified and
possibilities for implementations will be discussed. In this way, a base is formed which can be used in further design of a new system to support pre- service teachers during their training.
Feedback systems in music education Music paint machine
The first system of interest is the music paint machine which is used in music instrument teaching. This interactive system allows students to create
paintings by combining musical aspects and body movement. In this way, it provides real-time visual feedback on their music performance [3]. In order to translate movement and sound into visualisations, the system makes use of drawing commands that control the paintbrush on the canvas. In particular, the x-position of the paintbrush is determined by the movement of the torso, the y- position is determined by pitch and the thickness of the brush corresponds with loudness. Apart from this, the user has the option to access a third
dimension on a Z-axis that displays time for analysing and replaying the music performance [3]. This method is supported by Kawahara et al. [4], who points
out that the use of real-time visualization has great potential for musical training.
The music paint machine has multiple strong aspects. One of these concerns the translation from musical improvisation into painting or drawing.
[3] In other words, the system lowers the threshold to play music by changing the task to something students are more familiar with which makes the interaction with the system more fun. As a result, students become more confident and creative when playing music and dare to challenge themselves when improvising. Improvising, or composing music in general, is also already a benefit in itself, since it teaches children creativity and problem solving skills, as stated by Nikolaidou et al. [5] In addition, a second strong aspect of the system entails the possibility for reflective discussion by using the students artistic output as documentation. More precisely, the paintings and music from a student can be compared which provides insights in the student’s music playing and learning process [3]
Despite the benefits that the system has to offer, Nijs et al. [3] mention scepticism from instrumental music teachers towards the use of visual
feedback, because they fear that these systems interfere with listening and understanding music. In contrast, Nijs et al. refute this and state that visual and auditory feedback can reinforce one another which increases the musical experience and learning process. Saltzman and Lahav [8] agree that it is better to combine audio and visual stimuli. They found that visual feedback facilitates pitch recognition and auditory memory. Therefore, objections against the use of visual feedback systems emanate from misconceptions of musical understanding and the nature of music [3].
Andantino
The second system of interest is Andantino which is used to teach children to play the piano by using light silhouettes in various ways. This interactive music system is designed as a more effective and enjoyable learning environment beyond the focus on reading musical scores. More specifically, instead of traditional musical lines, Andantino projects miniature light silhouettes that seem to walk on the piano or keyboard [6]. This new way of visualisation can help children in various steps of the piano learning process. A first example is listening, since the system can enact music on the piano accompanied with the moving figures which can help children with focused listening to the qualities of the acoustic sound. Secondly, the figures can provide meaning to musical aspects, in which the head corresponds to pitch, the feet to rhythm and the body to musical lines. In addition, different walks of different
characters can display different moods. Thirdly, children can gradually learn to play along with the piano by shadowing the movements of the keys. Lastly, the system teaches children about harmonies by projecting columns on the piano which represent the root, dominant and subdominant harmony [6].
The use of Andantino over traditional sheet music notation highlights multiple strong aspects of the system. The first entails emotional engagement of the children with the system. The playful nature of the figures light the moods of students and appeal to their imagination and encourages to practise along. [6] This leads to the second strong aspect of the system, namely, the use of figures makes memorization easier than the use of blocks and score.
[6] The benefits of this alternative visualisation are confirmed by Rogers et al.
[7] who point out that traditional sheet music notation does not support the mapping of notes to piano keys, which is especially challenging for novices.
This corresponds with the third aspect which entails that the figures promote
deeper listening. In other words, the figures encourage children to play by ear rather than solely relying on the music score, which results in a more effective and enjoyable learning process. [6]
P.I.A.N.O.
The third and last system of interest is P.I.A.N.O. which is used as a support to learn to play the piano by means of an interactive projection. To be more precise, the system projects upcoming notes of a song on an extended
surface behind a keyboard and the piano keys themselves are lighted in order to help the student with mapping the notes to the piano keys. Furthermore, the notes in this visualization are represented as vertical bars, similar to an
enhanced piano roll. Hereby, the horizontal position indicates pitch, the vertical position indicates when to play the notes and the vertical length corresponds to note duration. [7] In addition, recommended fingering is indicated by contrasting colours and a preview of upcoming notes are
highlighted as thin extension lines from the scroll onto the instrument itself. In this way, the visualisation supports the same musical aspects as traditional sheet music notation. Finally, the system provides three sequential learning modes: listen mode to train attention, practise mode to support retention and play mode to reproduce songs and provide motivation. [7]
The P.I.A.N.O. system has many strong aspects of which the three most important ones are discussed here. The first aspect entails that the system supports faster learning without making use of sight-reading skills, because the notes are mapped directly onto the instrument [7]. As a matter of fact, the play mode of the system supports this faster learning by providing the learner with live feedback by colour coding correct, incorrect and missed notes which helps to identify their progress and provides motivation [7]. The second strong aspect of the system concerns an increased user experience because the interactive projection is easier to understand and more fun than sheet music notation. Surely, the user experience is affected by the encouraging nature of the practice mode of P.I.A.N.O. in which the system awaits key presses before continuing the song and highlights brightly the wrong and correct key when the learner plays the wrong note. [7] The last strong aspect regards the projected notation of the system which reduces the complexity of note notation and therefore requires significantly less cognitive load than other systems. [7] Rogers et al suggest that the system reduces split attention.
Kalyuga [9] supports this finding and points out that by directing the attention of the learner by colour coding techniques, split-attention can be avoided.
On the other hand, the system also shows two major concerns. The first concern comes from the learners who believe themselves to be too dependent on the system. They have a hard time reproducing learned songs without the system and can’t learn new songs, because the system does not cover sight-reading skills. In contrast, the second concern comes from piano teachers who fear that the system focuses too much on correctness. This would result in stiff playing and obstruct the development of musical expression. [7] The importance of expressivity in music performance is described by Woody [10] who explains that understanding differences in expression is the first step of expressivity and the implementation in musical performance the second step which sets the performer apart from the rest.
This process corresponds with the aim of P.I.A.N.O to support faster learning and playing focusing on basic playing skills. Subsequently, teachers could provide further guidance for next steps such as fingering techniques and musical expression. [7]
Conclusion
In this literature review, an overview of three visualisation feedback systems is provided and their strong and weak aspects are identified. These systems all provide support in music education in which they all give meaning to musical aspects in a different way. The music paint machine translates musical input directly into a painting by using pitch and loudness for the position and
thickness of the brush. Whereas, P.I.A.N.O. builds on traditional note notation and tries to use interactive projection to reduce its complexity. In comparison, Andantino combines aspects of the other two systems, because it makes use of both projection and translates musical aspects in fun figures that display pitch, rhythm and musical lines. As a result, learners are emotionally engaged with the visualisation of Andantino which motivates practise. In the same way, P.I.A.N.O. shows an increased user experience due to its patient and fun practise mode which also encourages learners. Likewise, the music paint machine lowers the threshold to play music, because the artistic output is more familiar to the user. This results in increased confidence and creativity.
In contrast, there are also some aspects that are promising but that were not met by all three systems. The first aspect entails that the complexity of musical aspects is reduced by both the Andantino and P.I.A.N.O. systems.
Namely, the figures from Andantino promote easier memorization than the traditional musical score. Similarly, P.I.A.N.O. maps the notes directly onto the instrument and uses colour coding to guide the learner. However, the music paint machine focuses more on improvisation and fun interaction than to simplify the output. The second and last aspect to consider entails the promotion of deeper listening which the music paint machine and Andantino both evoke. For example, the combination of visual and auditory feedback of the music paint machine increases the learning process of pitch recognition. In addition, the figures of Andantino lets the learner play by ear instead of
reading musical scores. On the contrary, the system of P.I.A.N.O. focuses more on the visual projection and colour codes that guides the user. As a result, learners fear they are too dependent on the system. In conclusion, the findings discussed in this section form a foundation which can be implemented in the design of a new visual feedback system. This system aims to increase the confidence of pre-service teachers and that supports them during their training.
References
[1] De Ioris, L., Moggio, F., & Giovannella, C. (2012). Beyond the "orff schulwerk", toward a new approach to the technology enhanced musical education. Paper presented at the Proceedings of the 12th IEEE International Conference on Advanced Learning Technologies, ICALT 2012, 662-663.
doi:10.1109/ICALT.2012.189 Retrieved from www.scopus.com
[2] Holden, H., & Button, S. (2006). The teaching of music in the primary school by the non-music specialist. British Journal of Music Education, 23(1), 23-38. doi:10.1017/S0265051705006728
[3] Nijs, L., Coussement, P., Muller, C., Lesaffre, M., & Leman, M. (2010). The music paint machine: A multimodal interactive platform to stimulate musical creativity in instrumental practice. Paper presented at the CSEDU 2010 - 2nd
International Conference on Computer Supported Education, Proceedings, , 1 331-336. Retrieved from www.scopus.com
[4] H. Kawahara, E. Haneishi and K. Hagiwara, "Realtime feedback of singing voice information for assisting students learning music therapy," 2017
International Conference on Orange Technologies (ICOT), Singapore, 2017, pp. 99-102.
[5] G. N. Nikolaidou, V. T. Iliadou, S. G. Kaprinis, L. J. Hadjileontiadis and G.
S. Kaprinis, "Primary School Music Education and the Effect of Auditory Processing Disorders: Pedagogical/ICT-Based Implications," 2008 Eighth IEEE International Conference on Advanced Learning Technologies, Santander, Cantabria, 2008, pp. 1030-1031.
[6] Xiao, X., Puentes, P., Ackermann, E., & Ishii, H. (2016). Andantino:
Teaching children piano with projected animated characters. Paper presented at the Proceedings of IDC 2016 - the 15th International Conference on
Interaction Design and Children, 37-45. doi:10.1145/2930674.2930689 Retrieved from www.scopus.com
[7] Rogers, K., Röhlig, A., Weing, M., Gugenheimer, J., Könings, B., Klepsch, M., . . . Weber, M. (2014). P.I.A.N.O.: Faster piano learning with interactive projection. Paper presented at the ITS 2014 - Proceedings of the 2014 ACM International Conference on Interactive Tabletops and Surfaces, 149-158.
doi:10.1145/2669485.2669514 Retrieved from www.scopus.com
[8] Eldridge, Marcus & Saltzman, Elliot & Lahav, Amir. (2010). Seeing what you hear: Visual feedback improves pitch recognition. EUROPEAN JOURNAL OF COGNITIVE PSYCHOLOGY. 22. 1078-1091.
10.1080/09541440903316136.
[9] Kalyuga, S., Chandler, P., & Sweller, J. (2011). Managing split-attention and redundancy in multimedia instruction. Applied Cognitive Psychology, 25(SUPPL. 1), S123-S144. doi:10.1002/acp.1773
[10] Woody, R. H. (2000). Learning Expressivity in Music Performance: An Exploratory Study. Research Studies in Music Education, 14(1), 14–23.
https://doi.org/10.1177/1321103X0001400102
→
→
→
→
→
o
o
o
__________________ __________________
