semantic multimedia web n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Semantic Multimedia Web PowerPoint Presentation
Download Presentation
Semantic Multimedia Web

Loading in 2 Seconds...

play fullscreen
1 / 81

Semantic Multimedia Web - PowerPoint PPT Presentation


  • 110 Views
  • Uploaded on

Semantic Multimedia Web. Ansgar Scherp Basierend auf Folien von Carsten Saathoff, Raphael Troncy und Lynda Hardman. Was bisher geschah. MMDB als Erweiterung von ORDBMS Information Retrieval als Basis für Queries Feature Extraktion um Inhalt zu beschreiben

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Semantic Multimedia Web' - jarvis


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
semantic multimedia web
Semantic Multimedia Web

Ansgar Scherp

Basierend auf Folien von Carsten Saathoff,

Raphael Troncy und Lynda Hardman

was bisher geschah
Was bisher geschah...

MMDB als Erweiterung von ORDBMS

Information Retrieval als Basis für Queries

Feature Extraktion um Inhalt zu beschreiben

Feature Transformation um kompaktere Darstellung zu bekommen

Fokus auf low-level features

Distanzen und Ähnlichkeiten

Indizierung von Features für schnellen Zugriff

probleme traditioneller mmdb
Probleme traditioneller MMDB

Datenstrukturen und Schemata meist proprietär

MMDB typischerweise für eine Applikation aufgesetzt.

Spätere Integration mit anderen Applikationen schwer

Ad-Hoc Integration eher unmöglich

Starker Fokus auf Low-Level Features

Semantische Lücke

Kein direktes Mapping zwischen Low-Level Features und Semantik des Bildes

Retrieval primär über Ähnlichkeit

Fast alle Studien zeigen, dass Nutzer dadurch nicht zufriedengestellt werden

Nutzer wollen semantisch Anfragen

metadaten 2
Metadaten (2)‏

Stichworte

GPS Information

Kamera Daten

Datum

metadaten1
Metadaten

Daten über Daten

Autor, Creation-Date, Keywords, ...

Wie repräsentieren?

Relationales Schema

XML

Semantic Web Technologien

Metadaten sollten interoperabel sein

Web

Desktop

Intranets

Viele Applikation müssen kommunizieren

berblick
Überblick

Semantic Web + Multimedia

Semantische Lücke

CanonicalProcessfor Multimedia Production

MPEG7 und COMM

Probleme mit MPEG7

Core Ontology on Multimedia (COMM)‏

Linked Open Data

KAT – K-Space Annotation Tool

Semi-Automatische Effiziente Annotation

Szenarien

semantic web auf einer folie
Semantic Web auf einer Folie

cooperatesWith

Ontology

rdfs:Domain

rdfs:Range

Person

rdfs:subClass

Employee

rdfs:subClass

rdfs:subClass

PostDoc

Professor

rdf:type

rdf:type

<swrc:PostDoc rdf:ID="person_sha"> <swrc:name>Siegfried Handschuh</swrc:name>

...

</swrc:PostDoc>

<swrc:Professor rdf:ID="person_sst">

<swrc:name>Steffen Staab </swrc:name>

...

</swrc:Professor>

Meta-data

<swrc:cooperatesWith rdf:resource = "http://www.uni-koblenz.de/~staab/#person_sst"/>

swrc:cooperatesWith

Webpage

URL

http://www.deri.ie/~sha

http://www.uni-koblenz.de/~staab

semantic web for multimedia
Semantic Web for Multimedia

Multimedia Ontology

Domain Ontology

IsWeb @ Bad Kreuznach 2007

http://isweb.uni-koblenz.de/

ResearchMeeting:=

>=1 depicts.Researcher

ResearchMeeting

depicts

Researcher

rdf:type

depicts

http://kodemaniak.de/foaf.rdf

http://wiamis2008.org

hasName

WIAMIS 2008 in Klagenfurt

„Carsten Saathoff“

Zeig mir alle Bilder von Research Meetings!

berblick1
Überblick

Semantic Web + Multimedia

Semantische Lücke

CanonicalProcessfor Multimedia Production

MPEG7 und COMM

Probleme mit MPEG7

Core Ontology on Multimedia (COMM)‏

Linked Open Data

KAT – K-Space Annotation Tool

Semi-Automatische Effiziente Annotation

Szenarien

semantische l cke
Semantische Lücke

0010EE -> bläulich

Visuell ähnlich! Aber semantisch

unterschiedlich!

0033FE -> bläulich

semantische l cke1
Semantische Lücke

Visuell ähnlich, semantisch ähnlich, aber...

USA

Italien

ebenen der semantik
Ebenen der Semantik

Generische Objekte

Eine Person

Generische Szene

Personen unterhalten

sich

Wissen

Spezifische Objekte

Churchill

Spezifische Szene

Churchill, Roosevelt, Stalin

sitzen zusammen

Abstrakte Objekte

Churchill, Premierminister,

GB, ...

Abstrakte Szene

Big Three, Yalta Konferenz

WWII, ...

berblick2
Überblick

Semantic Web + Multimedia

Semantische Lücke

CanonicalProcessfor Multimedia Production

MPEG7 und COMM

Probleme mit MPEG7

Core Ontology on Multimedia (COMM)‏

Linked Open Data

KAT – K-Space Annotation Tool

Semi-Automatische Effiziente Annotation

Szenarien

example 2 vox populi video sequences generation
Example 2: Vox Populi Video Sequences Generation
  • Stefano Bocconi, Frank Nack
  • Interview with Americavideo footage with interviews and background material about the opinion of American people after 9-11http://www.interviewwithamerica.com
  • Example question:What do you think of the war in Afghanistan?

“I am never a fan of military action, in the big picture I don’t think it is ever a good thing, but I think there are circumstances in which I certainly can’t think of a more effective way to counter this sort of thing…”

vox populi premeditate process
Vox Populi Premeditate Process
  • Analogous to the pre-production process in the film industry
    • Static versus dynamic video artifact
  • Output
    • Script, planning of the videos to be captured
    • Questions to the interviewee prepared
    • Profiles of the people interviewed: education, age, gender, race
    • Locations where the interviews take place
vox populi annotations
Vox Populi Annotations
  • Contextual
    • Interviewee (social), locations
  • Descriptive
    • Question asked and transcription of the answers
    • Filmic continuity, examples:
      • gaze direction of speaker (left, centre, right)
      • framing (close-up, medium shot, long shot)
  • Rhetorical
    • Rhetorical Statement
    • Argumentation model: Toulmin model
vox populi statement annotations
Vox Populi Statement Annotations
  • Statement formally annotated:
    • <subject> <modifier> <predicate>
    • E.g. “warbestsolution”
  • A thesaurus containing:
    • Terms on the topics discussed (155)
    • Relations between terms: similar(72), opposite(108), generalization(10), specialization(10)
    • E.g. waroppositediplomacy
toulmin model
Toulmin Model

Qualifier

Data

Claim

Warrant

Condition

Backing

Concession

57 Claims, 16 Data, 4 Concessions, 3 Warrants, 1 Condition

vox populi organize process
Vox Populi Organize Process

diplomacy best solution

contradict

support

war best solution

war not solution

  • Using the thesaurus, create a graph of related statements
    • nodes are the statements (corresponding to video segments)“war best solution”,“diplomacy best solution”,“war not solution”
    • edges are either support or contradict
result of vox populi query
Result of Vox Populi Query

I am not a fan of military

actions

I cannot think of a more effective solution

War has never

solved

anything

Two billions

dollar bombs

on tents

canonical processes 101
Canonical Processes 101
  • Canonical: reduced to the simplest and most significant form possible without loss of generality
  • Formalization of each process in UML diagrams
    • Process
    • Process artifacts
    • Process actors
    • External world artifacts
create media asset
Create Media Asset
  • Process where media assets are captured, generated or transformed
semantic annotate
Semantic Annotate
  • The annotation uses some controlled vocabularies
    • Subject matter annotations of your photos
    • Rhetorical annotations in Vox Populi
package
Package
  • Process where process artifacts are logically and physically packed
sum up
Sum Up
  • Community agreement, not “yet another model”
  • Large proportion of the functionality provided by multimedia applications can be described in terms of this model
  • Initial step towards the definition of open web-based data structures for describing and sharing semantically annotated media assets
berblick3
Überblick

Semantic Web + Multimedia

Semantische Lücke

CanonicalProcessfor Multimedia Production

MPEG7 und COMM

Probleme mit MPEG7

Core Ontology on Multimedia (COMM)‏

Linked Open Data

KAT – K-Space Annotation Tool

Semi-Automatische Effiziente Annotation

Szenarien

mpeg7
MPEG7

ISO Standard der MPEG Community

Einheitliches Format zu Speicherung von Multimedia Metadaten

Struktur

Features

Semantik

Extrem (!) umfangreich

Darauf basierend MPEG21 mit Fokus auf DRM etc.

Hat im Gegensatz zu MPEG1-4 nichts mit Kodierung zu tun

mpeg7 2
MPEG7 (2)‏

Basiert auf XML

MPEG7 Beschreibung ist eine Hierarchie

Deskriptoren beschreiben Eigenschaften von Multimedia Daten

Struktur

Video -> Shots -> Frames

Bilder -> Segmente

Semantik

Low-Level Features

Um Flexibilität zu gewahren, können Deskriptoren sehr vielseitig kombiniert werden.

issues
Issues

Winston ChurchillRecognizer

Franklin RooseveltRecognizer

Josef StalinRecognizer

<Mpeg7>

<Description xsi:type="ContentEntityType">

<MultimediaContent xsi:type=„ImageType">

<Image>

<SpatialDecomposition>

How do you formulate a query to get imagesshowing Churchill et al.?

<StillRegion id=„SR1“>

<TextAnnotation>

<KeywordAnnotation xml:lang="en">

<Keyword>Churchill</Keyword>

</KeywordAnnotation>

</TextAnnotation>

</StillRegion>

First Shot (Xpath)://StillRegion[.//Keyword=“Churchill” or .//Keyword=”Roosevelt” or .//Keyword=”Stalin”]

<StillRegion id=„SR2“>

<Semantic>

<Label>

<Name>Roosevelt</Name>

<Label>

</Semantic>

</StillRegion>

<StillRegion id=„SR3“>

<Semantic>

<Definition> <!-- Also TextAnnotation!! -->

<StructuredAnnotation>

<WhatObject>

<Name xml:lang="en">Stalin</Name>

</WhatObject>

</StructuredAnnotation>

</Definition>

</Semantic>

</StillRegion>

...

probleme mit mpeg 7
Probleme mit MPEG-7?

Annotationen sind nicht interoperabel!

Mehrdeutigkeiten

Mehrere Möglichkeiten um semantisch identische Annotationen zu beschreiben

Deskriptoren können auf viele Arten kombiniert werden

Komplexe Anfragen müssen alle Alternativen beachten

capabilities and maturity levels
Capabilities and Maturity Levels
  • no standard, no vocabulary
  • manual 1:1 agreement on format and semantics
  • tight coupling of data and applications
  • standard vocabulary
  • manual 1:1 agreement onmpeg-7 vocabulary
  • tight coupling of data and applications
  • standard vocabulary
  • pre-defined meaning
  • ad-hoc coupling of data and applications
  • CORE ONTOLOGY

Integration Automation

COMM

Nächster Teil der VL

MPEG-7

Formerly

Former Situation

Current Situation

Future / Desired Situation

ontology stack
Ontology Stack
  • Foundational Ontologies
    • Span across multiple fields, each covering multiple domains
    • Modelling of the most abstract concepts like event, object, ...
  • Core Ontologies
    • Situated in one field, but spans multiple domains
    • Can base on foundational ontologies
    • Examples fields: events, annotation, communication, ...
  • Domain Ontology
    • Fora specific domain, e.g., fishery, human body, etc.

Core Ontologies

Foundational Ontologies

Domain

Ontologies

slide39

Legend

Challenge

BuildingBlock

COMM

Requirements on a high quality MM Ontology

MPEG-7

requirements for comm
Requirements for COMM

ReusabilityDesign a core ontology for any multimediarelated application

MPEG-7-ComplianceSupport most important description tools

ExtensibilityEnable inclusion of further

description tools(even those that are not part of MPEG-7!)‏

media types

Separation of ConcernsClear separation of domain knowledge andknowledge about structure

ModularityEnable customization of multimedia ontology

High degree of axiomatization Ensure interoperability throughmachine accessible semantics

<Mpeg7>

...

</Mpeg7>

<Mpeg7>

...

</Mpeg7>

Josef StalinRecognizer

ChurchillRecognizer

FaceDetector

AuthoringTool

PhotoManager

MusicManager

  • audio descriptors

...

decomposition

  • visual descriptors

SemanticAnnotation

CompoundDocument

TextDescriptor

is mpeg 7 a good basis for a high quality ontology
Is MPEG-7 a good Basis for a high Quality Ontology?

Shortcomings of badly modelled ontologies[Oberle et al., 2006]:

Conceptual ambiguity

Difficulties in understanding themeaning of concepts and theirrelations

Poor axiomatization

Axiomatization of well definedconcepts is missing

Loose Design

Presence of modelling artefacts(concepts without ontological meaning)‏

Shortcomings mainly hinder

Extensibility

Interoperability

Especially 1) and 2) are major shortcomings of MPEG-7

1-to-1 translations from MPEG-7 to OWL/RDFS (e.g. [Hunter, 2003a]) will not result in high quality ontologies!

<StillRegion id=„SR1“>

<TextAnnotation>

<KeywordAnnotation xml:lang="en">

<Keyword>Churchill</Keyword>

</KeywordAnnotation>

</TextAnnotation>

</StillRegion>

<StillRegion id=„SR2“>

<Semantic>

<Label>

<Name>Roosevelt</Name>

<Label>

</Semantic>

</StillRegion>

<StillRegion id=„SR3“>

<Semantic>

<Definition> <!-- Also TextAnnotation!! -->

<StructuredAnnotation>

<WhatObject>

<Name xml:lang="en">Stalin</Name>

</WhatObject>

</StructuredAnnotation>

</Definition>

</Semantic>

</StillRegion>

slide42

Legend

Challenge

BuildingBlock

COMM

Quality of Ontologies

Requirements on a high quality MM Ontology

MPEG-7

how to design a high quality multimedia ontology
How to Design a High Quality Multimedia Ontology?

Approach from [Oberle, 2005], [Oberle et al., 2006]:Use a well designed foundational ontology as a modelling basis to avoid shortcomings

Foundational ontologies provide

Formal precision

Domain independence

Broad scope

Building upon foundational ontologies

prevents easy inclusion of modeling artefacts

reduces conceptual ambiguity

inherit rich axiomatization

methodology
Methodology

COMM

Reference Ontologie

MPEG-7Compliance

Quality Measures for Ontologies

Quality of Ontologies

Legend

Challenge

Requirements: High Quality MM Ontology

BuildingBlock

MPEG-7

methodology for design pattern definition
Methodology for Design Pattern Definition

Identification of most important MPEG-7 functionalities[Arndt et al., 2007]:

Decomposition of multimedia content into segments

Annotation of segments with meta data (e.g. visual descriptor, media information, creation & production, …)‏

General:

Identify repetitive structures and describe them at an abstract level

Describe digital data by digital data at an arbitrary level of granularity

Additional patterns are needed for:

Complex data types of MPEG-7

Semantic annotation by using domain ontologies

Interface between reusable multimedia core and domain specific knowledge

dolce design patterns oio
DOLCE Design Patterns: OIO

Foundational ontology DOLCE+DnSUltralight

Aims at capturing the most essential aspects in the world

Defines disjunctive upper classesEvent, Object, Quality and Abstract

Follows a pattern-oriented approach for ontology design

2 design patterns (extensions) that are especially important for MPEG-7:

Ontology of Information objects (OIO): Formalization of information exchange

Information object represents pure abstract information (message)‏

Relevance for multimedia ontology:

MPEG-7 describes digital data (multimedia information objects) with digital data (annotation)‏

Digital data entities are information objects

dolce design patterns d s
DOLCE Design Patterns: D&S
  • Descriptions & Situations (D&S): Formalization of Context
  • Relevance for multimedia ontology:
    • Meaning of digital data depends on context
    • Digital data entities are connected through computational situations (e.g. input and output data of an algorithm)
    • Algorithms are descriptions
    • Annotations and decompositions are situations that satisfy the rules of an algorithm / method
methodology1
Methodology

COMM

Pattern definition through Specialization

Repr. of Context

Identification of repetitive structures

Reference Ontologie

Repr. of Information

MPEG-7Compliance

Quality Measures for Ontologies

Quality of Ontologies

Legend

Challenge

Requirements: High Quality MM Ontology

BuildingBlock

MPEG-7

example
Example

Information Object „Graz Tourist Guide“

Information Realization

http://cms.graztourismus.at/cms/ziel/42425/EN/

Booklet

Information Encoding:

English

German

About: Places, Buildings (e.g. Clock Tower)‏

Agent: 1. iMedia Visitor / 2. Tourist Officer / 3. Graphics Designer

Expresses:

Walking Path through Graz

Small-Size Tourist Guide

Arrangement of Illustrations

descriptions situations d s
Descriptions & Situations (D&S)‏
  • Distinction between:
    • DOLCE ground entities (regions, endurants, perdurants)‏
    • Descriptive entities (parameters, roles, courses)‏
  • Descriptions
    • Formalize context
    • Define descriptive concepts
  • Situations
    • Are explained by descriptions
    • Are settings for ground entities
example1
Example

Image1 playsRole SegmentationInput

Segment1 playsRole SegmOutp

Segment1 playsRole SegmInput

Segment3 playsRole SegmOutp

Segment4 playsRole SegmOutp

Segment2 playsRole SegmOutp

Via its role in a computational task the different parts may be arbitrarily nested and related to different computing algorithms

Querying for all subparts takes place along a well-defined pattern

modular architecture
Modular Architecture
  • Multimedia ontology consists of
    • Core module that contains the design patterns
    • Modules that specialize the core module for different media types
    • Modules that contain media independent MPEG-7 description tools such as media information or creation & production
    • Data type module that formalizes MPEG-7 data types e.g. matrices, vectors, unsigned-int-5, float-vector, probability-vector, …
does the multimedia ontology fulfil the requirements
Does the Multimedia Ontology fulfil the Requirements?

Reusability

MPEG-7-Compliance

Design patterns enable therepresentation of description tools

Extensibility

Design patterns are media independent  possibility to include

further media types

arbitrary descriptors

Extensions of multimedia ontologywill not affect legacy annotations dueto DOLCE+D&S+OIO

Separation of Concerns

Clear separation between domainspecific and multimedia relatedknowledge

Modularity

Modular architecture allows customization

High degree of axiomatization

Design patterns come with genericaxiomatization that is refined in derivedontology modules

Josef StalinRecognizer

ChurchillRecognizer

COMM

PhotoManager

One such extension has already beendone for Text Annotation.Another one for compund documentannotation is currently developed!

  • Content & Media Annotation Pattern
  • Semantic Annotation Pattern
  • See slide before this slide!
  • OWL-DL version available for download.
benefits of a dolce aligned multimedia ontology
Benefits of a DOLCE-aligned Multimedia Ontology

Usage of DOLCE enforces clean design

Constraints prohibit arbitrary placement of MPEG-7 concepts into DOLCE

Similar concepts will be placed on similar locations of the taxonomy

Things that are different, have to be separated (e.g. data and the perceivable content that is carried)‏

Extensibility due to underlying general taxonomy of DOLCE

Possibility to describe multimedia domain at anarbitrary level of detail (e.g. segments have pixels as atomic parts)‏

Rigorous application of the D&S and OIO patterns allows description of digital data in different contexts(e.g. data acting as input or output for an algorithm)

benefits compared to mpeg 7
Benefits compared to MPEG-7

Linkage with domain ontologies allows meaningful semantic annotation of multimedia content

Semantic part can be entirely replaced with a domain ontology

Clear separation between domain ontologies and multimedia core ontology through semantic annotation pattern

Easier queries

Annotation pattern guarantees equal representation of all annotations

Complex data type pattern guarantees uniform access to nested data

No complex XML-structures to parse

Multimedia ontology only uses restricted inventory of DOLCE predicates

Higher interoperability through machine accessible semantics and underlying DOLCE axiomatization

creating a multimedia presentation revisited
“Creating a Multimedia Presentation” Revisited

Josef StalinRecognizer

Winston ChurchillRecognizer

Franklin D. RooseveltRecognizer

History Ontology

...

World War II

Yalta

...

PhotoManager

AuthoringTool

Churchill

Roosevelt

Stalin

SR3

SR2

SR1

http://en.wikipedia.org/wiki/Yalta_Conference

Sparql: select ?image where {

?image plays AnnotatedDataRole.

?x plays SemanticLabelRole.

?x rdf:type pol:President }

berblick4
Überblick

Semantic Web + Multimedia

Semantische Lücke

CanonicalProcessfor Multimedia Production

MPEG7 und COMM

Probleme mit MPEG7

Core Ontology on Multimedia (COMM)‏

Linked Open Data

KAT – K-Space Annotation Tool

Semi-Automatische Effiziente Annotation

Szenarien

a giant graph open to the world
A Giant Graph Open to the World

<rdf:Descriptionrdf:about="Ganesh.jpg"> <dc:title>An image of the Elephant Ganesh</dc:title> <dc:creator>RaphaëlTroncy</dc:creator>

</rdf:Description>

Annotate the content (interpretation)Elephant, Ganesh, Thailande, Holidays, Chiang Mai

Link to knowledge on the Web:img foaf:depicts dbpedia:Ganeshdbpedia:Ganesh rdfs:label "Vinayaka"dbpedia:Ganesh skos:altlabel "Ganapati" dbpedia:Ganesh rdf:type wn:synset-Deities-noun-1dbpedia:Ganesh owl:sameas wn:synset-Ganesh-noun-1

linking open data project
Linking Open Data Project
  • Expose open datasets in RDF
  • Set RDF links among the data items for different datasets
  • Over 2 billion triples, 3 millions links (March 2008)

http://richard.cyganiak.de/2007/10/lod/

warum ist lod wichtig
Warumist LOD wichtig?
  • RDFawird von der Google Suchmaschineverarbeitet
    • Content provider bieten nun RDFa an
    • Erhöht Click-trough-rate auf Webseiten (Werbeanzeigen)
    • Erhöht Ranking derWebseiten in Google
  • SIOC Ontology zurVerlinkung von Online Communities wirdgenutzt von Yahoo!
    • WirddurchSearchMonkeyeingesammelt
    • Tools um RDFazupublishen
  • Usw.
dbpedia
DBpedia
  • DBpedia is a community effort to:
    • extract structured "infobox" information from Wikipedia
    • interlink DBpedia with other datasets on the Web
automatic links among open datasets
Automatic Links Among Open Datasets

Processors can switch automatically from one to the other …

take home message
Take Home Message
  • Reuse what is there
    • Of course, one could create RDF data manually …… but that is unrealistic on a large scale
    • Goal is to generate RDF data automatically when possible and "fill in" by hand only when necessary
      • service to get RDF from flickr imageshttp://www.kanzaki.com/works/2005/imgdsc/flickr2rdf
      • service to get RDF from XMPhttp://www.ivan-herman.net/cgi-bin/blosxom.cgi/WorkRelated/SemanticWeb/xmpextract.html
  • Expose what you make
berblick5
Überblick

Semantic Web + Multimedia

Semantische Lücke

CanonicalProcessfor Multimedia Production

MPEG7 und COMM

Probleme mit MPEG7

Core Ontology on Multimedia (COMM)‏

Linked Open Data

KAT – K-Space Annotation Tool

Semi-Automatische Effiziente Annotation

Szenarien

kat k space annotation tool
KAT: K-Space Annotation Tool
  • Goal
    • Efficient annotation of multimedia content
    • Means to create semantically rich annotations
  • KAT provides framework for
    • Executing analysis plugins
    • Providing visualisation plugins
      • Displaying/annotating content
      • Browsing
    • Interfaces with Core Ontology for Multimedia (COMM)‏
      • Provides the common model
    • Role based messaging to leverage reuse of components
efficient annotation
Efficient Annotation
  • Reduce time required by user for annotating content
  • Integration of
    • Automatic analysis methods
      • Region labeling, object detection
      • Key Frame Extraction, Shot Boundary Detection
    • Automatic Organisation
      • Clustering
    • Inferencing
      • Based on formal domain ontologies

Semi-automation

semantically rich annotations
Semantically Rich Annotations
  • Relational Annotation
    • Express how depicted entities are related
    • Example: Soccer Game
      • Who is tackling whom?
      • Why was the penalty given?
    • Ontologies provide means to express relations
    • KAT aims at providing the means to efficiently create them
  • Event-Based annotation
    • Events are prominent in multimedia
    • Create and manage events
    • Relate events and media
    • Allow for event-based retrieval and exploration
architecture
Architecture

GUI

KAT-Core

Plugin View

register

Plugin

Plugin

...

store

display

COMM

Repository

Repository

...

szenarien
Szenarien

Effiziente Annotation von persönlichen Bildern

Semi-Automatische, Semantische Annotation von Sport-Ereignissen

Browsing von Bildsammlungen im Web

pers nliche bildsammlungen
Persönliche Bildsammlungen

Heutzutage typischerweise in Ordnern auf Festplatte

Wenig Annotationen weil zu aufwendig

Annotationen enthalten

Viel Hintergrund Wissen

Gefühle und persönliche Momente

Frage: Wie kann ein Nutzer hier unterstützt werden?

Automatische Annotation liefert nur einfache Semantik

Daher: Clustering um Ereignisse zu finden

User kann dann ganze Ereignisse annotieren

Verwendung von NLP (Textanalyse) um semantische Annotation zu erzeugen

sportereignisse
Sportereignisse

Kombination verschiedener Algorithmen

Highlight Detection: Goals, Corner-Shot, ...

Features: Motion, Geräusche, sichtbare Konzepte

Analyse von Minute-by-Minute Reports

Liefert andere Ereignisse

Ergebnisse oft nur global

Manuelles Refinement

Zuweisen von Namen zu Spielern im Video/Bild

Zuweisen von Rollen:

Wer hat das Faul begangen, wer war Opfer

Verknüpfen von Ereignissen

Ziel: möglichst vollständige Annotationen effizient erstellen

browsen von bildkollektionen
Browsen von Bildkollektionen

Flickr als COMM

Maping von Tags in Wordnet

Wordnet: linguistische Ontologie

Mapping von Geoinformationen nach Geonames

Ontologie von geographischen Informationen (Länder, Orte, ...)‏

Mapping nach dbpedia

Wikipedia als maschienenlesbare Version

Tags bekommen Kontext

Anzeige werwandter Bilder, komplexe Queries

Ergänzen der Annotationen

slide77
KAT as basis for der User Interface
  • Objectives
    • Explore and visualize semantic Web 2.0 data in real-time
    • Acquainting oneself about an area of interest
  • Semantic data
    • 1 billion triples from DBpedia, GeoNames, WordNet, FOAF files and Flickr
    • Very large, mixed-quality, semantically heterogeneous
  • Winner of Billion Triples Track, Semantic Web Conference 2008, Karlsruhe [ISWC2008]
semaplorer web 2 0 content brower
SemaPlorer – Web 2.0 Content Brower

Activefacetslike tags, location

Information on locations, persons, tags

Searchforlocations, personsand tags

  • SemaPlorer Live!
    • http://btc.isweb.uni-koblenz.de/

Geo-referencedimagefromFlickr

Mapshowinglocations, sights, pictures

werbeblock
Werbeblock
  • HiWi-Jobs für SemaPlorer++
    • Interesse an Arbeit in einerGruppe
    • Tätigkeit die KenntnisseausdemStudium (und darüberhinaus) praktischenanwendenlässt
    • Spaß am “Tütfteln”
    • Entwicklungmit Java
  • Mail mitBeschreibung an Erfahrungen / Lebenslauf an scherp@uni-koblenz.de
werbeblock1
Werbeblock
  • ImageAtlas II Projektpraktikum
    • EntwicklungeinerPlattformzumTaggen und zurDiskussion von BildernderKunstgeschichte und Bildwissenschaft
    • ZusammenmitdemInstitutfürKunstwissenschaft
  • Anmeldung: Jetztüber KLIPS
  • http://isweb.uni-koblenz.de/ -> Lehre -> WS09/10
  • Fragen? Mail an scherp@uni-koblenz.de
werbeblock2
Werbeblock
  • Diplomarbeiten
  • BeispielefürThemen
    • KontextsensitiveVisualisierung von Events und Objekten auf derKarte
    • Repräsentation von dynamischenorganisationalenProzessen am Beispiel des Notfallmanagements
    • TripleRanked Faceted Browsing Interface of Linked Open Data
  • Auch online unter
  • http://isweb.uni-koblenz.de/interactiveweb
  • Und weitereThemen …