Adaptation step-by-step : challenges for real-time spatial
personalization
Citation for published version (APA):
Hage, van, W. R., Stash, N., Wang, Y., & Aroyo, L. M. (2010). Adaptation step-by-step : challenges for real-time spatial personalization. In S. Berkovsky, F. Bohnert, F. Carmagnola, D. Cheng, D. Heckmann, T. Kuflik, P. Nurmi, & K. Partridge (Eds.), Workshop on Pervasive User Modeling and Personalization (PUMP 2010) at User Modeling, Adaptation and Personalization (UMAP 2010, Big Island HI, USA, June 20-24, 2010) (pp. 40-47)
Document status and date: Published: 01/01/2010
Document Version:
Publisher’s PDF, also known as Version of Record (includes final page, issue and volume numbers)
Please check the document version of this publication:
• A submitted manuscript is the version of the article upon submission and before peer-review. There can be important differences between the submitted version and the official published version of record. People interested in the research are advised to contact the author for the final version of the publication, or visit the DOI to the publisher's website.
• The final author version and the galley proof are versions of the publication after peer review.
• The final published version features the final layout of the paper including the volume, issue and page numbers.
Link to publication
General rights
Copyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights. • Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain
• You may freely distribute the URL identifying the publication in the public portal.
If the publication is distributed under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license above, please follow below link for the End User Agreement:
Real-time Spatial Personalization
Willem Robert van Hage1
, Natalia Stash2
, Yiwen Wang2
, and Lora Aroyo1
1
VU University Amsterdam wrvhage@few.vu.nl, l.m.aroyo@cs.vu.nl
2
Eindhoven University of Technology n.v.stash@tue.nl, y.wang@tue.nl
Abstract. In this paper we outline challenges for user modeling and personalization with spatial information. To illustrate those challenges we use a use case with a real-time routing system that implements a mobile museum guide for providing personalized tours tailored to the user position inside the museum and her interests. In this scenario we combine on the one hand (1) interactive discovery of user’s interests applied for semantic recommendations of artworks and art-related topics, and on the other hand (2) dynamic step-by-step adaptation of a user’s route through a museum based on her current position and changing interests. For this, the existing CHIP mobile museum guide was extended with a routing mechanism based on the SWI-Prolog Space package.
1
Challenges in Dynamic Spatial Adaptation
When a visitor moves around in a museum, exploring the collection of artworks based on her interest in these artworks, many spatial aspects can be considered in the process of recommending the visitor what to see. The same holds for a mobile museum guide, where we exploit techniques of dynamic personalization for such recommendations. Adding the spatial aspects allows to improve the dynamic adaptation further. For example, the assessment of the user’s constantly evolving interests can be improved by step-by-step spatial information.
If we include spatial considerations into the many different elements of dy-namic adaptation, we have many opportunities, but we also have as many re-search challenges. The opportunities we illustrate in this paper are based on our experience with the concrete museum demonstrators from our work in the CHIP project and we have elicited research challenges from that same experience.
– Dealing with real-time information: When we deal with real-time in-formation in the process of dynamic adaptation, we can consider the user’s position, her context, and her social interaction:
• User location: Detecting a user’s location inside the physical space is a first challenging step. In a museum this requires a positioning system that considers the boundaries and constraints (i.e. the walls, doors, stairs) of the space. In our case, we have an indoor space, and therefore methods
2 Willem Robert van Hage et al.
using different hardware solutions have been proposed to increase the accuracy of the indoor user positioning.
• User context: For the adaptation in the collection-based semantic rec-ommendation process, identifying the relevant context is important. This context typically includes the identification of artworks in her neighbor-hood, the artworks that have been already seen, the time she has already spent in the museum and additional temporal constraints (e.g. how much time is available), her general interests in art, and potentially also her physiological and emotional state [2]. The main challenge that we met in our case is to know the relevant context from a spatial perspective. • Social interactions: Social interactions can play a role in the
adapta-tion process, e.g. the interacadapta-tions between people that visit the museum together, and the spatial aspect can impact the process even more. Think of routes that the people take and potential meetings that they might have inside the museum. The social interactions are relevant both as input and output of the adaptation process and including the spatial aspect in these social interactions offers a challenging improvement. – Coping with the limited resources of a mobile device: The dynamic
adaptation and context-awareness ask a lot from the infrastructure and the limited resources that mobile devices offer. We have experimented in CHIP with different re-routing algorithms for the purpose of adaptation. The cur-rent algorithm can provide re-routing of a tour of artworks based on the user’s position. It would be interesting however to consider more complex algorithms that would also take user preferences into account and possibly decide to add additional artworks to the tour that might be interesting for the user based on the user’s close proximity.
– Overcoming mobile platform dependance: The personalization and recommendation process is based on knowledge about the content and the user. To reduce complexity and to ensure reusability of the knowledge rep-resentation and inference mechanisms, a flexible web-based approach is re-quired that allows different types of systems to exchange and augment in-formation on users and particular situations [2]. Also, the web-based archi-tecture allows the use of multiple types of mobile devices.
– Maintaining distributed user model: The personalization in an appli-cation like the ones we consider here, is not a single standalone one. The user model that is relevant for this kind of dynamic adaptation in CHIP asks for the capability to exchange and integrate user model knowledge in a distributed fashion. For this reason, we have chosen a distributed and open web-based solution for user model knowledge representation.
– Integration with third-party applications: It would be interesting to consider technologies like Google Goggles3
to show information about an artwork when the user points with her device to it. This is an example of an integration with third-party applications, and in our case we have chosen an approach that facilitates this space-oriented integration, that as we will show later is mainly based on offering standard interfaces and interoperability.
3
– Evaluating in real-life settings: In order to improve this real-time adap-tation process, patterns of user’s navigation and evolution of interests would be very helpful. However, collecting large volumes of such data over long periods of time is very difficult.
In the remainder of this paper, we describe how we brought the CHIP demon-strators to a next version that includes the spatial dimension.
2
Spatial Personalization in Museums: Related Work
Museum curators typically would offer tours on different topics based on the highlights of the collections. Thus, the resulting tours are characterized by a pre-defined selection and fixed sequence of artworks. An audio tour would still offer a predefined selection of artworks, however it allows for determining your own sequence of artworks. A number of museums, e.g. Tate Modern, Science Museum Boston, are exploring the potential of personalized museum guides. Personalized virtual tours, on the other hand, help visitors construct their own narratives4
, however they are only limited to online collections. Multimedia guides provide a promising alternative to bridge the gap between the visitor’s interests and the static museum tours by using personalization techniques [5]. An adaptive mobile museum guide acts as a museum expert and provides the user with information adapted to the current situation [2]. For example, the MIT Media Lab5
audio and visual narration adapts to the user’s interest acquired from the physical path in the museum and length of the user stops. The mobile museum guides developed within Hippie [3] and PEACH [4] projects provide content adaptation based on technical restrictions of specific presentation devices as well as visitor’s preferences and knowledge. The mobile museum guide built within Sotto Voce [1] project takes into account the special needs of groups visiting a museum and facilitates social interaction between group members. Another example for fostering social interaction between visitors is given by the AgentSalon [6] sys-tem, and ARCHIE [10] provides a socially-aware handheld guide that stimu-lates interaction between group members. The Kubadji mobile tour guide6
uses a collaborative filtering approach for predicting visitor’s viewing times of unseen exhibits from his viewing times at visited exhibits. The context-aware museum guide in [11] is adapting by dropping artworks if the visitor falls behind the tour or is suggesting additional artworks or taking a break at a nearby restaurant if the visitor has extra time. The environment also supports p2p interactions between visitors, to find each other, share ratings and comments about exhibits. Important here is the fact that spatial information is used in relatively lim-ited aspects for adaptation. Usually, the real implementation of such approaches depends on the availability of an indoor localization of people and objects.
4
Virtual Museum (of Canada), http://www.museevirtuel-virtualmuseum.ca/ 5
http://www.media.mit.edu/ 6
4 Willem Robert van Hage et al.
3
Use Case: Space-CHIP Step-by-Step Route Adaptation
The CHIP project is a cross-disciplinary project, combining aspects from cul-tural heritage and information technologies to provide a personalized access to the museum collection both online and inside the museum7
, e.g., generating per-sonalized museum tours, getting recommendations about interesting artworks to see, and quickly finding ways in the museum. One important aspect of the project is the use of a common distributed user model, which collects user interaction data and interprets it in terms of user’s interests used further for generating rec-ommendations and personalized tours. Additionally, we also use the e-Culture Semantic Search8
open API to allow to find semantically related topics and art-works to include in the personalized tours. The Mobile Museum Guide allows users to access their tours created with the online Tour Wizard on their mo-bile devices in the museum. Details about the design and implementation of the CHIP Art Recommender, Tour Wizard and Mobile Museum Guide (ver 0.1 and ver 1.0) can be found in [9].
The CHIP Mobile Museum Guide (ver 0.1 and ver 1.0) can adapt to the user in many different ways, but mainly based on the known user preferences and availability in the museum. To relate the step-by-step adaptation also to the real physical space, spatial constraints have to be taken into account in the generation of both recommendations and museum tours. Suppose the user follows a tour of recommended artworks. If she provides a rating to an artwork that she sees, the CHIP demonstrator updates the user model and as consequence the list of recommended artworks. However, those versions of the Mobile Museum Guide do not take spatial constraints as well as information about already seen artworks into account for the adaptation. Further we show how we implement the Space-CHIP Mobile Museum Guide that includes adaptation with spatial constrains, http://www.chip-project.org/spacechip. The implementation is based on the SWI-Prolog Space package.
The basic approach in the Art Recommender is to recommend based on the estimated likelihood that the user will like the artworks. Even with a theme-based layout of the rooms in the Rijksmuseum, e.g., rooms for the Dutch republic or works by Rembrandt and his pupils, a set of recommended artworks can in re-ality be distributed over the entire museum. To improve the user experience, we therefore reorder the results of the Art Recommender to allow for an efficient walk from one artwork to the other. Such a route minimizes the walking effort, while maximizing the number of top recommendations. Also, it takes into ac-count optional caps to the walking distance and the number of artworks. This helps the user to decide where to go in limited time.
Computing an efficient route through a museum is very similar to the trav-eling salesman problem. However, it is a significantly easier problem than the general traveling salesman problem. First, if you consider the artworks, rooms, doors, hallways, and stairs to be nodes in a connectivity graph (e.g. Fig. 1),
7
See http://www.chip-project.org/demo 8
then this graph is not fully connected, as there are walls and floors in the way. Second, from the way in which exhibits are created, it makes sense to view all works from a single room together. Third, floor transitions take a lot of effort. For these reasons there are only a few sensible paths through the museum.
room
display display display
door hallway stairs door door room display display door room display display
Fig. 1.Example connectivity graph
If you set the transition weight of the edges in the connectivity graph to the experienced distance instead of the actual distance, then nearest neighbor search sends the visitor to works in the same room first before making the transition to another room (or floor), which is good in the Rijksmuseum, but bad in the general case. The SWI-Prolog Space package [8] provides nearest neighbor search. As this search is unaware of the restrictions posed by the walls and floors, we base our routing on a connectivity graph search algorithm that uses intersection queries as opposed to nearest neighbor queries. First, we compute a connectivity graph between all the artworks, rooms, stairwells, etc. that considers where the doors are. Then, we compute the weighted shortest path between all artworks. The weight is based on graph distance, the type of transition (e.g. moving to another floor is more expensive), and on the distance between locations inside a room. This shortest distance matrix is used to compute an efficient path along all the recommended artworks. The exact method for route calculation is as follows:
– Pre-compute artwork distance matrix once 1. define that stairs, hallways, toilets, are rooms 2. define works that are on display in the museum
(a) give the artwork a hx, y, zi coordinate 3. define what it means to be connected
(a) places (displays, doors) space intersect with same room (b) places are stated to be connected by A chip:connectsTo B 4. assert A chip:connectsTo B for each connected pair hA, Bi 5. make connectivity graph of chip:connectsTo
6. compute weights for each transition
(a) graph distance plus distance within room
(b) door transitions get a higher graph distance than artwork-artwork transitions
(c) stairs transitions get an even higher graph distance
7. compute and cache upper triangle matrix of weighted graph shortest path distances between all places
6 Willem Robert van Hage et al.
Fig. 2.Mapping CHIP user model (UM) to the simple event model (SEM)
– Apply routing algorithm for each request
1. fetch set of recommended artworks (given by Art Recommender) 2. fetch current position (given by user interface)
3. fetch remaining time in museum (given by user interface)
4. fetch maximum number of artworks to route (given by user interface) 5. greedy nearest neighbor search in weighted distance graph until list of
recommended artworks is empty:
(a) look up the nearest recommended artwork (b) remove artwork from list of candidates
(c) add path from current position to the artwork to recommended route (d) set current position to the location of the artwork
(e) add length of path to total length of recommended route
6. while total path length of recommended route takes longer than remain-ing time in museum
(a) remove furthest artwork from current position (b) apply greedy nearest neighbor search again (step 5)
In order to provide data exchange between CHIP and the SWI-Prolog Space package we mapped (see Fig. 2) the original CHIP user model (UM) [9] to the Simple Event Model (SEM)9
which is proposed by van Hage et al. [7].
For example, when the user rates with four stars both the painting “Woman Reading a Letter” and its creator Johannes Vermeer, this results in a list of rec-ommended artworks, which further used to generate the Tour of Recrec-ommended Artworks (see Fig. 3). We use icons in a different color to indicate artworks that are in the tour and connect them with the tour line. The user location is indi-cated with an icon at the entrance door on the ground floor. During the visit the user views artworks that are in the tour but is also attracted by other artworks outside her tour. In that case, the tour may be re-routed taking into account the user’s interest in these additional artworks. Similarly, the user can also rate any
9
For this work we use this version: http://semanticweb.cs.vu.nl/2009/04/event/. A newer version is available at http://semanticweb.cs.vu.nl/2009/11/sem/.
artwork she sees on her way. These actions result in the tour being dynamically adapted taking into account the history of seen artworks, changing interests and current location. Thus, if the user likes the works by Frans Hals and Ferdinand Bol that she comes across on her way to the recommended Johannes Vermeer works, she can submit new rating and this automatically updates the tour. The updated tour is shown in Fig. 4. For the sake of clarity we have highlighted the works from the original tour with red, the new Frans Hals recommendations with yellow and the new Ferdinand Bol work with blue.
Fig. 3.Initial route of the tour of recommended recommended artworks
In [12] we evaluated (1) the usefulness of recommendations to the users and (2) the efficiency of the route calculation.
Acknowledgments. The work on the CHIP project is funded by the Dutch Sci-ence Foundation CATCH program and is in collaboration with the Rijksmuseum Amsterdam. Part of this work has been carried out as a part of the Poseidon project in collaboration with Thales Nederland, under the responsibilities of the Embedded Systems Institute (ESI). This project is partially supported by the Dutch Ministry of Economic Affairs under the BSIK program.
References
1. Aoki, P.M., Grinter, R.E., Hurst, A., Szymanski, M.H., Thornton, J.D., Woodru, A.: Sottovoce: exploring the interplay of conversation and mobile audio spaces. In: SIGCHI Conference on Human factors in computing systems (2002)
8 Willem Robert van Hage et al.
Fig. 4.Re-routed tour of recommended artworks
2. Kruger, A., Jorg, B., Heckmann, D., Kruppa, M., Wasinger, R.: Adaptive Mobile Guides. In: The Adaptive Web, pp. 521–549. Springer (2007)
3. Oppermann, R., Specht, M.: A nomadic information system for adaptive exhibi-tion guidance. In: ICHIM99 Conference (1999)
4. Rocchi, C., Stock, O., Zancanaro, M., Kruppa, M., Kruger, A.: The museum visit: generating seamless personalized presentations on multiple devices. In: IUI (2004) 5. Roes, I., Stash, N., Wang, Y., Aroyo, L. : A personalized walk through the museum:
The CHIP interactive tour guide. In: ACM CHI-Student Competition (2009) 6. Sumi, Y., Mase, K.: Interface Agents that facilitate knowledge interactions between
community members. In: Cognitive Technologies Series, pp. 405–427 (2004) 7. van Hage, W.R., Malais´e, V., de Vries, G., Schreiber, G., van Someren, M.:
Com-bining ship trajectories and semantics with the simple event model (sem). In: 1st ACM International Workshop on Events in Multimedia. Sheridan Publishers (2009) 8. van Hage, W.R., Wielemaker, J., Schreiber, G.: The space package: Tight integration
of space and semantics. In: 8th ISWC Workshop: TerraCognita (2009)
9. Wang, Y., Stash, N., Aroyo, L., Gorgels, P., Rutledge, L., Schreiber, G.: Recom-mendations based on semantically enriched museum collections. Journal of Web Semantics. 6(4), 43–58 (2008)
10. Luyten, K., Van Loon, H., Teunkens, D., Gabrils, K., Coninx, K., Manshoven, E.: ARCHIE: Disclosing a Museum by a Socially-aware Mobile Guide. In: 7th Interna-tional Symposium on Virtual Reality, Archaeology and Cultural Heritage (2006) 11. Chou, S.-C., Hsieh, W.-T., Gandon, F., L., Sadeh, N.M.: Semantic Web
Technolo-gies for Context-Aware Museum Tour Guide Applications. In: 19th Int. Conference on Advanced Information Networking and Applications, 709–714 (2005)
12. van Hage, W.R., Stash, N., Wang, Y., Aroyo, L.: Finding Your Way through the Rijksmuseum with an Adaptive Mobile Museum Guide. In: ESWC (2010)
