A Tabletop Board Game Interface for
Multi-User Interaction with a Storytelling System
Thijs Alofs1, Mari¨et Theune1, and Ivo Swartjes?2
1
University of Twente, PO Box 217, 7500 AE Enschede, The Netherlands
2
Ranj Serious Games, Lloydstraat 21m, 3024 EA Rotterdam, The Netherlands t.alofs@gmail.com; m.theune@utwente.nl; ivo@ranj.nl
Abstract. The Interactive Storyteller is an interactive storytelling sys-tem with a multi-user tabletop interface. Our goal was to design a generic framework combining emergent narrative, where stories emerge from the actions of autonomous intelligent agents, with the social aspects of tradi-tional board games. As a visual representation of the story world, a map is displayed on a multi-touch table. Users can interact with the story by touching an interface on the table surface with their fingers and by mov-ing tangible objects that represent the characters. This type of interface, where multiple users are gathered around a table with equal access to the characters and the story world, offers a more social setting for interaction than most existing interfaces for AI-based interactive storytelling.
1 Introduction
In this paper we present a generic tabletop board game interface for interactive storytelling. The setting of a tabletop board game stimulates face-to-face con-tact and social behaviour, which we think is important in multi-user interactive storytelling. The idea of using tabletop interfaces for digital storytelling is not entirely new. Several tabletop interfaces exist for storytelling systems [1, 2, 3], but they only focus on facilitating storytelling, trying to stimulate collaboration and creativity. Unlike our system, they do not use AI to contribute to the story. The interfaces of current AI-based storytelling systems are mostly like those of computer games, where a single user interacts from a first person perspective with 2D or 3D virtual characters on a computer screen [4, 5, 6]. A few systems fo-cus on collaborative storytelling by multiple users, but these also have interfaces similar to computer games [7, 8]. To our knowledge, our Interactive Storyteller is the first AI-based storytelling system with a table-top interface.
The idea behind The Interactive Storyteller is to let users control the ac-tions of one or more of the characters in a simulated story world. Each user can play one character, but collaborative control is also possible, with users co-operating to make decisions for the characters. From the ongoing interaction between the characters (which can be either player or computer controlled) and
?The work was carried out while the third author was working as a postdoctoral
through their choices of actions a story emerges; this approach to interactive sto-rytelling is called ‘emergent narrative’ [4]. The technical framework underlying The Interactive Storyteller is a multi-agent system for story generation, in which intelligent agents act out the role of characters in the story. These agents can plan and execute sequences of actions to satisfy their character’s goals, taking into account the current state of the story world, and the mental state of the character. For more information on the storytelling framework, see [9].
By using a multi-touch table we aim to achieve interactive storytelling that resembles the social setting of a tabletop board game. Like existing digital table-top board games [10, 11, 12], we try to combine the dynamics and intelligence of computer games with the social advantages of traditional board games. To reinforce the resemblance with board games, we investigate the use of tangible playing pieces that represent characters for physical interaction.
Next, we discuss the interface design and design choices based on related work and preliminary user tests. We end with a discussion on future work.
2 Interface Design
The interface design of The Interactive Storyteller is meant to be generic and easily adaptable to different story domains. We use a multi-touch table based on infrared reflection that is capable of identifying tangible objects through fiducial markers. The MT4j framework1is used for multi-touch support.
Story World and View. The visual representation of the story world is pre-sented to users and possible spectators on a shared visual surface. Just as with traditional board games, it is important that people on all sides of the table have a similar view on the story world, therefore we chose a top-down map view.
Two story domains are currently available in our storytelling framework: a domain about pirates, and a domain based on the “Little Red Riding Hood” (LRRH) story [13]. We used the latter domain in the prototype, because we consider it to be more coherent, easier to understand for new users, and easier to visualise (see the screenshot in Fig. 1). Given that LRRH is a children’s fairy-tale, we decided to focus our research predominantly on children aged 6 to 11. However, a goal kept in mind was that the interface should also be appropriate for adults (with more adult domains and corresponding images).
The story world of LRRH contains three characters (Red, grandma, and the wolf) and five locations (Red’s house, grandma’s house, the clearing in the forest, the lake, and the beach). These locations are marked by blue circles on the map. Typical actions for a character currently available in the LRRH domain are amongst others: walking, greeting someone, stealing things, crying, eating something, baking a cake, and poisoning food. Characters can plan a series of such actions to try to achieve a goal they have. For instance, for the goal of ‘Red’ wanting to poison the wolf, a possible series of events might be: Grandma
Fig. 1. Screenshot of interface design. On the actual tabletop the character images are covered by the tangible objects representing the characters.
bakes a cake, Red poisons it with cyanide because she expects the wolf to steal it, which he does, the wolf eats the cake and dies.
For aesthetic reasons, we decided to draw the characters and houses on the LRRH map not strictly from their top-side view, but from a more recognisable angle. However, to prevent one side of the table from being optimal for perceiv-ing the story world, characters are displayed in one direction and houses are projected the other way (see Fig. 1). Another solution for this orientation issue is autorotation, as was used in KnightMage [10]. Autorotation is only possible when there is always just one user that controls each character and the position of this user at the table is known, both of which do not apply to our system.
To enable an existing story world to be used in The Interactive Storyteller, the only required additions are a map and pictures that represent the characters in the world. The coordinates of the locations on the map, which link the story world to its visual representation, have to be provided in a properties file. Physical Interaction. Like in the systems KnightMage [10] and False Prophets [11], users can change the locations of characters by moving physical toys that represent the story characters across the surface of the multi-touch table. These tangibles provide tactile interaction that is expected to be intuitive because it very much resembles the interaction offered by many familiar board games.
In our storytelling system locations are always discrete: characters are at one location, or the next, but never half-way in between. When a user moves a tangible to an adjacent location, the blue circle of the destination location turns green to indicate that this is an allowed action. When the user moves the tangible
to a location that is not in direct reach of the character’s current location, the circle of the destination turns red to indicate that this is not an allowed action. Users might put tangibles outside the circles that mark locations on the map. The system is unable to physically move tangibles away from such non-locations. Interventions, like the system asking to move a tangible to a particular location, are not used because they distract the user from the story. This means the system has to be able to deal with tangibles being anywhere on the map. When a tangible is not at its character’s actual location in the story world, a thin dotted blue line is shown connecting the tangible to its character’s location. Interface Elements. For the selection of non-move actions by users, there is an Action Selection Interface (ASI). An important requirement we had for this interface was that it should be quickly usable for multiple storytelling domains. A very specific and intuitive ASI can be developed by focussing on one particular domain to fit the users’ needs in that particular virtual world. This is usually done in computer games, but we consider it more important that the interface stays generic. Therefore, we decided to refrain from icon-based or other graphics-based ways for action selection, and use a flexible text-graphics-based approach.
Every time the turn goes to a new character, the knowledge base retrieves the set of all possible actions for that character, at that location, at that time, that are available in the story world. The ASI can be seen in the centre of Fig. 1. Because of the young target user group, all text is displayed in our native language Dutch. The user first selects a category in the centre bar of the ASI and then an action within that category. After that, the round confirmation button in the centre bar is enabled and can be used to confirm the selected action and pass the turn to the next character.
Users and spectators can read the results of actions that characters perform in the story areas, which are the two scrolls that can be seen on the screenshot in Fig. 1. If users want more or fewer lines of text to be visible in a story area, this can be achieved by touching the end of the scroll and rolling it up or down. The ASI and the story areas occlude the view of the part of the map behind them. A balance has to be found between good visibility of the map and its con-tents, and the readability of the ASI and story areas. To achieve this subjective balance, we decided to keep the user in control of the size and placement of the ASI and the story areas. The user can move these elements around by dragging them with a finger. By dragging with two fingers at the same time, it is possible to rotate and resize them. The user can choose to find a static arrangement that generally works well in a particular story world, or keep changing sizes and arrangements depending on the current state of the story and places of interest. Because we consider it to be important that users or spectators from all sides of the table have an equal view, all text in the ASI is presented in two directions instead of one. When users or spectators are standing on all four sides of the multi-touch table, the optimal layout is to position the ASI under an angle of 45 degrees with the sides of the table. We expect this angle to be acceptable for most readers. Having a shared ASI saves much space compared to having separate control areas on all four sides of the tabletop for different users, as in
the SIDES system [12]. Moreover, if everybody is standing on one side of the tabletop, the text at the opposite side of the ASI can be hidden by touching the minus symbol on that side, conserving even more valuable screen estate. Preliminary User Tests. We performed some informal user tests with five test subjects (three boys aged 8, 8 and 10; and two girls aged 10 and 11) inter-acting with an early prototype of the system. We found that despite the limited graphics, the children were very engaged by the system and enjoyed playing with it. With only a very limited explanation they understood how to interact with the system. Several children discussed possible actions together and some even planned a sequence of actions to pursue a particular storyline.
Based on observations, several improvements to the system were made. For example, we discovered that users often lost track of turn-taking. To make clear to which character the actions in the ASI belong, the ASI is now connected by a solid blue line to the character that has the turn.
Another observation was that fingertips were often badly recognised because they were very small. At the same time the rest of the hand did get recognised while hovering above the surface. By fine-tuning some recognition parameters we managed to reduce these issues, but the used hardware is a limiting factor.
Although told to do so, not every child looked at one of the story areas to read the results of actions performed by other characters. This often resulted in these users ending up confused and less immersed in the story. To address this issue, we decided to offer the same information in another modality by vocalising it with Loquendo text-to-speech. The story areas are kept as a time-independent source of the same information about the story. After introducing the new modality, we decided to also allow the addition of action specific sounds. Although domain specific, associating actions with sounds is is a quick and easy way to present audible feedback of an action to enrich the user experience.
3 Discussion and Future Work
Because our interface combines tabletop interaction with the advantages of computer- and board games, we expect the system to be a suitable interface for interactive storytelling. The system facilitates group play as opposed to soli-tary game play. Our next step will be a formal user test to evaluate whether The Interactive Storyteller provides a suitable interface for interactive storytelling for children. Because questionnaires are not very suitable for young children, we intend to use an observation scheme like the Play Observation Scale [14].
We also intend to investigate whether the use of tangibles in our system setup has advantages over a touch-only approach. To answer this question, we made a touch-only version of our prototype which only uses images to represent and move characters. Both versions will be compared in the user tests. These tests should also show whether users prefer to play one character each, like actors playing a role, or to choose the actions for all characters together in deliberation.
The multi-touch table used in this research makes use of an ordinary video projector, non-diffuse IR-beams, and an average webcam. The lack of precision of the used setup irritated users in the preliminary user test. In the future we would like to test the system on a high-end multi-touch table or to port The Interactive Storyteller to the new generation tablet PC’s. These devices live up to the high expectations and increasing demands of present-day users and provide more ac-curacy for new directions in multi-touch research. One of the research challenges to be addressed is allowing the possibility to add engaging visual elements (e.g., object inventories, animations) while keeping the framework generic.
References
1. Alves, A., Lopes, R., Matos, P., Velho, L., Silva, D.: Reactoon: Storytelling in a Tangible Environment. In: 3rd IEEE International Conference on Digital Game and Intelligent Toy Enhanced Learning. (2010) 161–165
2. Cappelletti, A., Gelmini, G., Pianesi, F., Rossi, F., Zancanaro, M.: Enforcing Cooperative Storytelling: First Studies. In: 4th IEEE International Conference on Advanced Learning Technologies. (2004) 281–285
3. Helmes, J., Cao, X., Lindley, S., Sellen, A.: Developing the Story: Designing an Interactive Storytelling Application. In: ACM International Conference on Inter-active Tabletops and Surfaces. (2009) 49–52
4. Aylett, R., Louchart, S., Dias, J., Paiva, A., Vala, M.: FearNot! - An Experiment in Emergent Narrative. In: Intelligent Virtual Agents. Volume 3661 of Springer LNCS. (2005) 305–316
5. Mateas, M., Stern, A.: Fa¸cade: An Experiment in Building a Fully-Realized Inter-active Drama. In: Game Developers Conference: Game Design Track. (2003) 6. Pizzi, D., Cavazza, M.: Affective Storytelling based on Characters’ Feelings. In:
Intelligent Narrative Technologies: Papers from the AAAI Fall Symposium. (2007) 111–118
7. Kriegel, M., Lim, M., Aylett, R., Leichtenstern, K., Hall, L., Rizzo, P.: A Case Study In Multimodal Interaction Design For Autonomous Game Characters. In: 3rd Workshop on Multimodal Output Generation. (2010) 15–25
8. Prada, R., Paiva, A., Machado, I., Gouveia, C.: “You cannot use my broom! I’m the witch, you’re the prince”: Collaboration in a Virtual Dramatic Game. In: Intelligent Tutoring Systems. Volume 2363 of Springer LNCS. (2002) 913–922 9. Swartjes, I., Theune, M.: The Virtual Storyteller: Story Generation by Simulation.
In: 20th Belgian-Netherlands Conference on Artificial Intelligence. (2008) 257–265 10. Magerkurth, C., Memisoglu, M., Engelke, T., Streitz, N.: Towards the Next Gener-ation of Tabletop Gaming Experiences. In: Graphics Interface 2004. (2004) 73–80 11. Mandryk, R., Maranan, D.: False Prophets: Exploring Hybrid Board/Video Games. In: Conference on Human Factors in Computing Systems. (2002) 640– 641
12. Piper, A., O’Brien, E., Morris, M., Winograd, T.: SIDES: A Cooperative Tabletop Computer Game for Social Skills Development. In: 20th Anniversary Conference on Computer Supported Cooperative Work. (2006) 1–10
13. Swartjes, I., Theune, M.: Iterative Authoring Using Story Generation Feedback: Debugging or Co-creation? In: 2nd International Conference on Interactive Digital Storytelling. Volume 5915 of Springer LNCS. (2009) 62–73
