1 / 23

CS 586 – Distributed Multimedia Information Management

CS 586 – Distributed Multimedia Information Management. Prof. Dennis McLeod. About the paper. Towards Ontology-Driven Discourse: From Semantic Graphs to Multimedia Presentations In Proceedings of the 2nd International Semantic Web Conference, 2003

abel-abbott
Download Presentation

CS 586 – Distributed Multimedia Information Management

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CS 586 – Distributed Multimedia Information Management Prof. Dennis McLeod

  2. About the paper . . . Towards Ontology-Driven Discourse: From Semantic Graphs to Multimedia Presentations In Proceedings of the 2nd International Semantic Web Conference, 2003 By: Joost Geurts, Stefano Bocconi, Jacco van Ossenbruggen, and Lynda Hardman Presented by: Gabriela Gutierrez, February 11, 2004

  3. Overview • Introduction • Example Scenario: Rembrandt • Process 1: From Semantic Graph to Structured Progression • Process 2: From Structured Progression to Multimedia Presentation • Conclusion

  4. Introduction • Traditionally • Applying Semantic Web technology to multimedia information systems focuses on using annotations and ontologies to improve retrieval process • Presentation of data is “detail” best left to CSS or XSLT style sheets • In this paper • Claim that importance and complexity of effective presentation design is grossly underestimated • Concentration on improving the presentation of the retrieval results

  5. Introduction • Human professional designers must understand: • Underlying semantics of the client’s information • Most effective order, grouping and priorities for structuring this information • Most effective means of using the chosen medium to convey the information

  6. Introduction • Information presentation design is a knowledge-driven process. It requires: • Sufficient knowledge about domain • Knowledge on ordering, grouping and prioritizing information • Knowledge about media design • Selection of most appropriate medium • Understanding of medium characteristics in order to choose an effective means to achieve the communication goal

  7. Introduction • Problem: • Professional designers can only design data-driven web sites if the underlying data, its semantics and target audience are relatively homogeneous. • Variety of data sources, semantic relations, output devices, and user profiles forces content providers to adopt one-size-fits-all approach. • Automation is needed in order to make the presentation of information knowledge-driven.

  8. Introduction • Assumptions: • Multimedia items are properly annotated • Annotations represent domain relations in a semantic graph (e.g. RDF) • Graph has associated Domain ontology • There is a Discourse ontology containing information about different document genres and building blocks for creating documents for each genre • There is a Design ontology containing media design knowledge

  9. Introduction

  10. Example Scenario: Rembrandt • Web query: “life and work of Rembrandt” • User-selected type of structured progression: disc:Biography • User-selected output medium: non-interactive multimedia presentation • Semantic graph = retrieval component’s results + domain ontology semantics relations • Structured progression = typical facts (name, DOB,…) + career facts + personal life info

  11. Process 1: From Semantic Graph to Structured Progression • CSS and XSLT operate purely on the XML level of RDF’s serialization syntax w/o any understanding or support for semantics of RDF data model • Transformation process needs access to knowledge on RDF Schema level • For querying underlying domain ontology • For access to its own operating knowledge

  12. Process 1: From Semantic Graph to Structured Progression • Several transformations prototyped in Java and Prolog environments • Direct access to a Sesame RDF Schema-based repository • Can use any query language supported by Sesame (RQL, RDQL, SeRQL) to gain direct access on the RDF instance level and the RDF Schema Level • Transformation process uses (declarative) domain and discourse-specific knowledge, while (procedural) transformation code remains generic

  13. Process 1: From Semantic Graph to Structured Progression • Transformation code uses RQL query to retrieve classes that Rembrandt instance belongs to . . . dom:Artist • Discourse ontology defines instance of disc:ArtistBiography that has disc:Subject property with value dom:Artist • Structured progressions have a disc:narrativeUnits property that specifies the disc:NarrativeUnits that can be used to construct it (e.g. disc:PersonalData, disc:PrivateLife and disc:Career)

  14. Process 1: From Semantic Graph to Structured Progression • Narrative Units have associated rules used to select matching content • Example: disc:PrivateLife • Rules to select information about family relations from semantic graph • Graph includes relation dom:isMarried between Rembrandt and Saskia_Uylenburgh • Rule #3 in following table can use domain relation to select Saskia in the disc:Role of disc:Spouse • Rules can be applied recursively • Rule #3 specifies that PrivateLife is the narrative unit that can be used for a subsequent nested story line • Process continues until no more rules can be applied or a rule specifies that no further expansion should happen

  15. Process 1: From Semantic Graph to Structured Progression • After all rules have been applied: • Biography w/ 3 narrative units • disc:PersonalData (Rembrandt in role of disc:MainCharacter) • disc:Career (Chiaroscuro in role of disc:Technique) • Disc:PrivateLife (Saskia_Uylenburgh in role of disc:Spouse)

  16. Process 2: From Structured Progression to Multimedia Presentation • Two-step process: • Structured progression transformed into Document Structure • Decisions on output medium (e.g. text, interactive hypermedia, passive multimedia) • Document Structure transformed into a tree of formatting objects • Detailed layout and formatting decisions (e.g. timing of presentation, transition effects)

  17. Process 2: From Structured Progression to Multimedia Presentation • Advantage: • Mapping discourse-specific narrative units to more general document elements allows for more commonly applicable formatting rules (e.g. disc:PrivateLife can be mapped to document section element, relying on common formatting rules for section-level elements) • Disadvantage: • There is always a level that can no longer be specified in terms of document structure (e.g. a figure w/ too much detail) • Solution: detailed structures copied directly into document structure in step 1 in order to define specific rules in step 2 to deal w/ formatting directly

  18. From Structured Progression to Document Structure

  19. Conveying Document Structure • Transforming a document structure into presentation constructs uses Cuypers library • Uses constraint solving techniques to verify that a presentation construct conforms to delivery-context constraints (e.g. screen size) • Allows alternative formatting specification if constraints are violated • A rule that transforms a document structure into presentation construct has 2 discourse parameters: • disc:NarrativeType • disc:Role • Parameters allow system to adapt formatting of presentation to convey message more effectively

  20. Conveying Discourse Semantics Directly • Depending on their function, we need to define formatting for different media types • Rembrandt self-portrait (disc:Portrait in disc:PersonalData vs. disc:Painting illustrating Chiarocuro) • Awareness of impact of different media modalities • Fall-back rules • Image not identified as either disc:Portrait or disc:Painting should be applied generic formatting for images since mm:Painting and mm:Portrait are subclasses of mm:Image

  21. Conclusion • Only short presentations have been generated to date, based on restricted domain ontology • Focus has been on single discourse structure (biography) and single document structure (multimedia presentation) • Additional research required to scale the system to more realistic scenarios • Under investigation: how knowledge about the user interacts w/ discourse and design knowledge used in current prototype

More Related