Tilburg University
Alone or together
Shahid, S.; Krahmer, E.J.; Swerts, M.G.J.
Published in:Fun and Games
Publication date: 2008
Document Version
Publisher's PDF, also known as Version of record
Link to publication in Tilburg University Research Portal
Citation for published version (APA):
Shahid, S., Krahmer, E. J., & Swerts, M. G. J. (2008). Alone or together: Exploring the effect of physical co-presence on the emotional expressions of game playing children across cultures. In P. Markopoulus (Ed.), Fun and Games (pp. 94-105). (Lecture Notes in Computer Science LNCS; No. 5294). Springer.
General rights
Copyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights. • Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain
• You may freely distribute the URL identifying the publication in the public portal
Take down policy
If you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediately and investigate your claim.
P. Markopoulos et al. (Eds.): Fun and Games 2008, LNCS 5294, pp. 94–105, 2008. © Springer-Verlag Berlin Heidelberg 2008
Co-presence on the Emotional Expressions of Game
Playing Children Across Cultures
Suleman Shahid, Emiel Krahmer, and Marc Swerts Department of Information and Communication Sciences,
Faculty of Humanities, University of Tilburg, Tilburg, The Netherlands
{S.Shahid,E.J.Krahmer,M.G.J.Swerts}@uvt.nl
Abstract. In this paper, we investigate the influence of physical co-presence on the emotional expressions of game playing children. We show that the emotional response of children belonging to different age groups and different cultural backgrounds varies when they play a game alone or together with their friends. A simple but quite effective number guessing game was developed to use as a tool for inducing emotions in an ethical way, which was played by Pakistani and Dutch individuals and pairs. The audiovisual emotional data thus collected was used in two perception tests in which Dutch viewers observed and classified the emotional response of Pakistani and Dutch children. Results show that the correct classification in both cultures is higher for children playing games in pairs, thus children in pairs are more expressive than individuals. Furthermore, both Pakistani individuals and pairs are more expressive than Dutch ones.
Keywords: Social gaming, Physical co-presence, Positive and Negative
emo-tions, Cross-cultural comparison.
1 Introduction
Emotions are often seen as deeply personal experiences and many researchers believe that an important function of emotional facial expressions is to display the internal state of the individual [1]. However, there are also researchers who emphasize that these emotional expressions primarily serve as social signals [2]. In general, researchers subscribing to both perspectives will accept that non-verbal expressions of emotions, either via voice or via facial expressions, do play a role in social communication, but the general trend in research has been to study the expression of emotions by looking at individuals, mostly without taking the social context into account [3]. With a few notable exceptions, the focus of emotion research has either been on posed expressions or on expressions elicited in non-social situations [4].
they tended to be designed and used for non-social situations, so that results gained from these techniques are not easy to generalize because of their artificially controlled settings [6]. Moreover, such methods may be effective in one particular culture or age group, but may not generalize to people with other backgrounds. So, there is a clear need for developing innovative techniques that meet different criteria: ideally, they should (1) lead to emotions in a natural and ethical way, and (2) be applicable across cultures and groups. In this paper, we explore the possibility of using games as a tool to induce emotions, which meets the criteria above.
Generally games are regarded as interactive, result oriented, competitive, dynamic and engaging in nature [7] and precisely these features can be exploited for using them as emotion inducers. Kaiser et al [6] reported that games give more control (to both players and experimenters) to create a natural ambience and go beyond traditional experimental settings of emotion induction. While playing games, people become engaged (both negatively and positively) and their level of engagement in games can lead to an intense emotional state (e.g., a state of flow) [8]. Moreover, it has been shown that there is a relationship between fun/enjoyment and emotions (expression of these emotions) [9] where fun in an activity can lead to an overall positive experience [10]. So the relation between emotions and games is interesting from a double perspective: on the one hand, games appear to present themselves as handy tools to induce natural emotions; on the other hand, the level of emotional involvement while playing games can be used as a metric to measure the fun and engagement in the games.
In our study, we look at the spontaneous expressions of two different emotions, one positive (“joy”) and one negative (“disappointment”), elicited using the GamE (Games as a method for eliciting Emotions) paradigm, with a specific focus on the role of the social context on the expression of these emotions. Under the GamE paradigm, our objective is to design a number of games to be used as a tool for inducing emotions in a natural and ethical way [11] and the card game developed during this particular research is one example of this. One specific innovation is that we focus on facial expressions as potentially relevant correlates of the internal emotional state of game players. While there is a whole body of research into facial expressions as “windows to the human soul” [12], these have so far not been included in tests to systematically evaluate games.
In the following, we elaborate on two potentially important factors that may affect the way games are played and appreciated, namely social context and age. Regarding the former, it is of course known that many different factors may contribute to the social context, but here we limit ourselves to two important ones, namely physical
co-presence and culture. If emotional expressions primarily serve to reflect the
individual state of an individual, then we can hypothesize that it does not really matter whether emotional expressions are collected from children that play a game alone or from pairs of children that play the game together. However, if the expressions also have a considerable social aspect, we would expect to find differences in expressivity between individual players and pairs of players.
which compare the expression of emotions across cultures in social context [14] but overall most of the work has focused on intra-cultural aspects of emotions. As a result it is still largely unknown how a culture influences the overall experience and expressiveness of emotions in specific social contexts.
Besides these two aspects of the broader context, we also include one individual factor in our studies, namely age. An interesting albeit largely unexplored question is how children learn to “use” their non-verbal expressions in a social context. Generally speaking, adults are more stable in their expression of emotions than children, who are overall more expressive than adults. In particular, young children use emotions as a communication aid because of their limited verbal abilities. This raises the question how children “grow” towards more adult-like expressiveness, and whether this development process is the same for different cultures and contextual settings.
In this paper, we focus on 8 and 12 years old children from Pakistan (a south-Asian culture) and from the Netherlands (a western-European culture), who either play a card guessing game alone or in pairs (section 2). The data collected with the GamE paradigm was then used for two perception studies (described in section 3) where adults look at both Pakistani and Dutch individual children (who had either played the game alone, or with another child) to find out whether differences in expressiveness could be found.
2 Experiment 1: Data Collection
2.1 Setup (Game Design)
For inducing positive and negative emotions in children in a natural manner, we used the GamE paradigm. We developed a simple card game to use as a tool for inducing emotions in children. In the card game, children have to guess whether an upcoming number would be higher or lower than the previous (reference) number.
When the game starts, players see a row of six cards on the screen where the number of the first card is visible (‘7’ in the case of the example in Fig 1) and the other five cards are placed upside down so the numbers are hidden. All the numbers
Fig. 2. Winning variant of the game
on the cards are between 1 to 10 and a number displayed once is not repeated in a particular game. Players have to guess whether the number on the next card will be higher or lower than the previous number. Once players have guessed the number, the relevant card is turned around and the correct answer is visible on the screen. Players are also informed about the correctness or incorrectness of their answer via a characteristic non-speech sound. If players make a wrong guess, the game is finished and they move to the next game. If players guess the number correctly then they are asked to guess the next number and players only win a game if they guess all numbers in the row correctly. In the case of winning a game, they receive a coin. At the end of the experiment, consisting of a number of games, players can exchange the coins they won with a prize.
2.2 Participants
In total, 144 children played the game of which 96 children played the game in pairs, 48 Dutch children (24 pairs) and 48 Pakistani ones (24 pairs). The remaining 48 children (24 Dutch and 24 Pakistani ones) played the game individually. In both conditions (individual and pairs), half of the Dutch and Pakistani children were around 8 years old (group 4 in the Dutch elementary school system) and the other half were around 12 years old (Dutch group 8). In the case of pairs, always children of the same age were paired. Parents and teachers gave prior written permission for their child to participate, and signed a consent form stating that the recordings could be used for research purposes.
2.3 Procedure
The experiment was conducted in four elementary schools, two schools in Tilburg (the Netherlands) with Dutch participants and two schools in Lahore (Pakistan) with Pakistani participants. The procedure for both conditions (pairs and individuals) in all four schools was essentially the same. A separate room was chosen in all four schools where depending on the condition, children in pairs (self-selected pairs of the same age group) or individual children were invited and asked to sit on the chair(s) placed in front of a desk on which a laptop computer was placed. On the top of the laptop, a camcorder was placed in such a way that it could record the children’s faces and the upper part of their body. Before the start of each experiment, the camera was adjusted to the children’s height. Another computer was attached to the laptop to facilitate the experimenter in controlling the game. The experimenter was outside of the visual field of the game-playing children.
Once the children were in the room and had chosen the appropriate seat(s) for sitting, the experimenter welcomed the children and started a small talk discussion by asking a few questions to break the ice (“How old are you? Do you like to play games?” Etc.). After this introductory phase, the experimenter gave spoken instructions, telling the children about the game and the coins they could win. All the game rules outlined in the previous section were explained to the children (in Urdu and Dutch for Pakistani and Dutch children, respectively), and when they seemed to understand the rules, the experimenter started a practice game (“So you only have to say whether the next card is higher or lower. This is just an exercise and it doesn’t really count”). After this exercise, the experimenter asked the children whether they had any questions, and if not the experimenter left the children’s field of vision and started the first experimental game.
Fig. 3. Representative stills for Pakistani players, with from left to right: 8 years old winning
and 12 year old loosing individuals (top), 8 years old loosing and 12 years old winning individuals taken from pairs (bottom)
Fig. 4. Representative stills for Dutch players, with from left to right: 8 years old loosing and
2.4 Results and Discussion
Figures 2 and 3 show representative stills of winning and losing individual children, and individual children taken from couples. Note that in the latter case, the children were actually sitting next to another child with whom they were playing the game together. Overall, the game worked quite well, in that all individual and pairs of participants indeed made the logical choices that were expected in most of the cases, so that each individual child and pair of children lost at least two games and won at least two games. Furthermore, not even a single child noticed and reported that the game was in fact a deterministic simulation. The data gathered are rich and constitute a valuable collection of audiovisual emotional child speech, consisting of both decision-making dialogues (in the case of pairs), and audiovisual responses to winning or losing a game. Informal observations reveal clear emotional differences between individual players and players in pairs. There are also clear differences among different age groups and cultures and we attempt to quantify all these differences in a series of cross-cultural perception experiments described below.
3 Experiment 2: Cross-Cultural Perception Studies
3.1 Stimuli
From all of the individuals and pairs that participated in Experiment I, we selected the first response of their winning game (in which they made a correct prediction for the last card) and the first response of their losing games (in which the final guess turned out to be incorrect). In addition, from the clips of child-pairs, we selected randomly one child from each pair by zooming in on his/her face. In this selection, half of the children sitting on the right chair and half of the children sitting on the left chair were selected. We also tried to select equal numbers of boys and girls. The stimuli were cut from the moment the final card was turned until their primary response was finished. This resulted in 96 Dutch stimuli: 2[alone/together] x 2[win/lost] x 2[boy/girl] x 2[8/12 years old] x 6[instances]. In this similar fashion, the 96 Pakistani stimuli were developed. Stimuli were presented to participants in a random order, in a vision-only format to avoid participants from relying on auditory cues.
3.2 Participants
72 Dutch adults, with a roughly equal number of men and women, participated in the perception experiments.
3.3 Procedure
Two group experiments were conducted: Dutch viewers judging Pakistani children [36 participants] and Dutch viewers judging Dutch children [36 participants] with essentially the same procedure for both experiments.
In addition, they were instructed that the children had just won or lost a game and they, as viewers, had to guess from the children’s facial expression whether they had won or lost the game. Each stimulus was preceded by a number displayed on the screen indicating the upcoming stimulus, and followed by a six second pause during which participants could fill in their score on the answer form. Before the actual experiment, there was a short training session in which 4 clips were shown (different from the ones shown in the actual experiment) to make participants familiar with the stimuli and the experimental task. If there were no further questions, the actual experiment started which lasted for approximately 18 minutes. During the experiment there was no interaction between participants and experimenter.
3.4 Statistical Analysis
All tests for significance were performed using a repeated measurement analysis of variance (ANOVA) with three within-subjects factors, namely Co-presence (levels: individual, individual from pairs), Age (levels: 8 years old, 12 years old) and Sex (levels: boy, girl), and with one between-subjects factor: Culture (levels: Dutch, Pakistani) and with percentage of correct classification as the dependent variable.
3.5 Results
Table 1 summarizes the results. The repeated measurement analysis of variance shows a significant main effect of co-presence (individuals vs. pairs) (F (1,70) = 716.753, p < .001, ηp2 = .911). The average of correct classifications is higher for
individuals taken from pairs (M = .812) than it is for individuals playing alone (M= .605). Another significant main effect was found of culture (F(1,70) = 168.9, p < .001,
ηp2 = .707). The average of correct classifications for Pakistani children (M = .782) is
higher than for Dutch children (M= .636). Finally, no significant main effect of sex nor of age was found.
Besides these main effects, we discuss a number of significant two-way interactions. We found a significant interaction between culture and co-presence (F (1,70) = 24.177, p < .001, ηp2 = .257). This interaction can be explained as follows:
even though Pakistani children are overall more expressive than Dutch children (higher percentage correct), the difference between Pakistani individuals (M = .66) and Pakistani individuals from pairs (M = .90) is larger than the difference between Dutch individual (M = .55) and Dutch individuals from pairs (M = .72). We also found a significant interaction between Culture and Sex (F (1,70) = 7.774, p < .01, ηp2
= .11). This interaction can be explained by the fact that both Pakistani boys and girls are more expressive than Dutch boys and girl, but this difference is overall higher between Pakistani and Dutch boys. Another interesting interaction was found between Culture and Age (F (1,70) = 46.782, p < .001, ηp2 = .40). This interaction can be
Table 1. Mean of correct classification results for Dutch observers judging stimuli from Dutch
and Pakistani kids (individual and individual taken from pairs) in 2 age groups
Culture Co-presence Sex Age Mean (Std. Error)
Dutch Individuals Boy 8 .625 (.019)
12 .461 (.018) Girl 8 .565 (.019) 12 .554 (.017) Individuals Boy 8 .760 (.014) taken from 12 .679 (.015) pairs Girl 8 .719 (.019) 12 .724 (.014)
Pakistani Individuals Boy 8 .617 (.019)
12 .784 (.018) Girl 8 .595 (.019) 12 .639 (.017) Individuals Boy 8 .920 (.014) taken from 12 .870 (.015) pairs Girl 8 .887 (.019) 12 .941 (.014)
4 Conclusion and Discussion
In this paper we investigated how the emotional response of children belonging to different age groups and different cultural background varies when they play a game individually or in pairs. For collecting emotional data in a natural way, we developed a simple but effective game in which individual participants or pairs of participants have to guess whether a card will contain a higher or lower number than a reference card. The emotional data collected in this way was used in two perception experiments, in which Dutch viewers in Tilburg saw fragments (without sound) of Pakistani and Dutch children; they were instructed to guess from the facial expressions of children whether children had won or lost the game.
taken from pairs were more expressive than their Dutch counterparts. This is in line with our previous findings [17], which show that Pakistani children (in both age groups) are more expressive than their Dutch counterparts. The pattern that Pakistanis are more emotional/expressive than Dutch was also found in totally different settings with adults [18].
Another interesting result related to the fact that the Pakistani 12 years old girls (especially the individuals taken from pairs) are not only more expressive than their Dutch counterparts but also more expressive than the 8 years old Pakistani girls and boys. This is an interesting finding, which, at first sight, seems to be incompatible with expectations based on the age theory [19] that younger children are more expressive than older ones, though it is consistent with the outcome of the work by [20] that girls are more expressive in showing their emotions than boys. This result is also in line with another study where it was shown that females were more expressive in the presence of a friend than in the presence of stranger [4]. Future analyses into this issue may reveal why it is that Pakistani 12 years old girls are the most expressive of the children we analysed, which could possibly be due to culture-specific display rules which dictate that Pakistani girls at a certain age are no longer supposed to be expressive in normal circumstances, which they may compensate for in game situations like the ones we analysed.
Other than the emotional differences after winning or losing a game, inspection of the recordings revealed that there were also differences in how Dutch and Pakistani children behaved during the time before the card was turned around, so when they were deciding whether the upcoming card would contain a higher or lower number. One salient difference was that Pakistani children often made praying gestures (forming a cup with the hands to pray, and finishing the prayer by wiping the hands across the face) while Dutch children never prayed. Gesturing was more common when children were playing game in pairs. Another interesting difference is that the interpersonal distance between Pakistani children appeared to be smaller than that between Dutch children; there was more touching, holding hands, putting head on shoulder with the Pakistani than with the Dutch children. This strong social bonding and interaction could be observed both during the decision making phase, and after the game result was known.
Overall, the card game developed under the GamE paradigm was quite engaging for children. Almost all children reported that they found the game full of fun because it was challenging, as it contained both winning and loosing situations with a fair amount of uncertainty. The game is also useful because it appears to generalize to multiple settings: it worked quite well in both cultures, across different age groups and in different social contexts and proved to be a good basic tool for inducing emotions without any artificial posing.
5 Future Work
In the future, we plan to run the perception test in Pakistan with Pakistani viewers to explore whether there are any cross-cultural differences in perceiving the emotional response of Pakistani and Dutch children. Furthermore, the audio-visual data gained from these experiments offers a rich collection of not only facial expressions but also body movements, like hand gestures. We would like to explore to what extent such gestures contain additional information that is useful for judging the emotional responses of children.
Our results shows that there is a strong influence of physical co-presence on the emotional expression of children but in future we would like to see that how different combinations of pairs (e.g., a pair of two strangers, a pair consisting of one active and one passive partner, a pair of a child and a robot) would effect these emotional expressions of children. Last but not the least we would like to investigate the short term and long term effect of game playing sessions on children’s mood.
Acknowledgments. We thank Sjoukje Hoebers and Remco Stultiens for their help in
collecting some of the production and perception data. We thank the children who participated in this study and specially their parents who gave their consent for this research. We thank the principal and teachers of schools in Tilburg and Lahore for their cooperation, and we thank all our perception test participants. Finally, thanks to the four anonymous reviewers for their constructive comments on a previous version of this paper.
References
1. Ekman, P.: Strong evidence for universals in facial expressions: a reply to Russell’s mistaken critique. Psychological Bulletin 115, 268–287 (1994)
2. Fridlund, A.: Human facial expression: an evolutionary view. Academic Press, New York (1994)
3. Manstead, A.S.R.: The social dimension of emotions. Psychologist (18), Part (8) (2005) 4. Wagner, H.L., Smith, J.: Facial expressions in the presence of friends and stranger. Journal
of nonverbal Behaviour 15(4) (1991)
5. Cornelius, R.R.: The science of emotion. Research and tradition in the psychology of emotion. Prentice-Hall, Upper Saddle River (1996)
6. Kaiser, S., Wehrle, T.: Situated emotional problem solving in interactive computer games. In: Frijda, N.H. (ed.) Proceedings of the IXth Conference of the International Society for Research on Emotions, pp. 276–280. ISRE Publications, Toronto (1996)
7. Salen, K., Zimmerman, E.: Rules of Play, Game Design Fundamentals. The MIT Press, Massachusetts (2003)
8. Lazzaro, N.: Why We Play Games: Four Keys to More Emotion Without Story. In: Game Developer Conerence (GDC), San Fransisco, USA (2004)
9. Podilchak, W.: Distinctions of fun, enjoyment and leisure. Leisure Studies 10(2), 133–148 (1991)
11. Shahid, S., Krahmer, E., Swerts, M.: GamE: Game as a method for eliciting Emotions. In: Proceedings of Measuring Behavior 2008, Maastricht, the Netherlands (to appear, 2008) 12. Kappas, A.: The Fascination With Faces: Are They Windows to Our Soul? Journal of
Nonverbal Behavior 21(3), 157–161 (1997)
13. Elfenbein, H., Ambady, N.: On the universality and cultural specificity of emotion recognition. Psychological Bulletin 128, 203–235 (2002)
14. Masuda, T., Ellsworth, P.C., Mesquita, B., et al.: Placing the face in context: Cultural differences in the perception of facial emotion. Journal of Personality and Social Psychology 94(3), 365–381 (2008)
15. Thompson, J., Berbank-Green, B., Cusworth, N.: Game Design: Principles, Practice, and Techniques - The Ultimate Guide for the Aspiring Game Designer. John Wiley & Sons, Hoboken (2007)
16. Wagner, H., Lee, V.: Facial behavior alone and in the presence of others. In: Philippott, P., et al. (eds.) The social context of nonverbal behavior, pp. 262–286. Cambridge University Press, New York (1999)
17. Shahid, S., Krahmer, E., Swerts, M.: Audiovisual emotional speech of game playing children: Effect of age and culture. In: Proceedings of Interspeech 2007 (IS 2007), Antwerpen, Belgium (2007)
18. Shahid, S., Krahmer, E., Swerts, M.: Real vs. acted emotional speech: Comparing Caucasian and South-Asian speakers and observers. In: Proceedings of Prosody 2008, Brazil (2008)
19. Thompson.: Emotion Regulation: A Theme in Search of Definition. In: The Development of Emotion Regulation, pp. 25–52 (1994)
