Presenting Knowledge on the Semantic Web

(1)

Presenting Knowledge on the Semantic Web

Lynda Hardman

Multimedia and Human-Computer Interaction

Frank Nack Jacco van Ossenbruggen

Lloyd Rutledge http://media.cwi.nl

(2)

Long-term goal

• Develop knowledge-intensive models and document processing technologies that are able to:

generate coherent multimedia presentations tailored to an individual user

taking into account their preferences, abilities, device capabilities and environment.

(3)

Goals of research

• Understand how to

– incorporate design and discourse knowledge into multimedia presentations

– represent this knowledge in Web and Semantic Web technologies

• Semantic Web provides

– source content for inclusion in presentations – a means of expressing knowledge needed

Amsterdam Hypermedia

Model

Cuypers

Topia, DISC

Noadster

Vox Populi

From syntax to semantics

SyntaxSemantics Time

(4)

Model

Cuypers

Topia, DISC

Noadster

Vox Populi

From syntax to semantics

Space/time trade-offs

(5)

Space/time trade-offs

• Media repository from Rijksmuseum

• Quantitative constraints insufficient using pixel-based positioning

• Qualitative constraints also used

specification of constraints at higher level A not-overlap B, B after C

• If insoluble then backtrack to other solutions using Prolog

• Joost Geurts

MMM 2001, WWW 2001

Model

Cuypers

Topia, DISC

Noadster

Vox Populi

From syntax to semantics

(6)

Inferring document structure

• Topia

• Rijksmuseum ARIA database -> RDF

• Clustering on results of query

• Presentation showing “table of contents”

and current focus

• Lloyd Rutledge ACM Hypertext 2003

(7)

Semantic graph to presentation

• DISC

• Rijksmuseum repository of media items

• Semantic graph is not enough

Rembrandt married-to Saskia also need discourse structures

for deriving grouping, ordering and priorities

• Biography template created painter is-a profession

• Stefano Bocconi, Joost Geurts ISWC 2003

(8)

Model

Cuypers

Topia, DISC

Noadster

Vox Populi

From syntax to semantics

(9)

Semantic Web browsing

• Noadster

• Generalised semantic web browsing

• Integrating global and local browsing

• Lloyd Rutledge, WWW 2005

Model

Cuypers

Topia, DISC

Noadster

Vox Populi

From syntax to semantics

(10)

Argument generation in video

• Vox Populi

• Database of video clips

• Annotated with topic and agree/disagree

• Argumentation model (Toulmin)

• User specifies query and video sequence generated

• Stefano Bocconi, ICME 2005

(11)

Scientific challenges

• Making (multimedia) discourse and design knowledge explicit

• Expressing re-usable semantics of media assets

• Designing architectures for multimedia presentation generation

Design dependencies

Content Presentation

structure

Aesthetics abcdefghij

abcdefghijk ABCDEFGHIJK abcdefghijk abcdefghijk

(12)

genre

cityscapes

Presentation structure depends on content

Vermeer

1665 1658

1665

1623

1628

MMDB Metadata

DB

Design Model Discourse

Model Domain

Model User Model

Device Profile Media Model

Discourse organization Presentation

design Constraint

layer Domain selection

System architecture

(13)

Conclusions

• Long term goal since a long way

• From projects described we have learned much:

– distinguish stages in process – separate discourse knowledge

– mappings between domain and discourse knowledge

• User is still very much missing – new projects:

CHIP user interaction, partner Rijksmuseum Passepartout media installation, partner V2_

• Starting work on investigating properties of time – in order to display to the user: MultimediaN

This research is supported by

• NWO I²RP

Intelligent Information Retrieval and Presentation

• NWO CHIME

Cultural Heritage in an Interactive Multimedia Environment

• NWO NASH

Networked Adaptive Structured Hypermedia

• Telematica Instituut Topia

• Images courtesy of Rijksmuseum, Amsterdam