Logic based semantically enriched integration of multi feature mir
Download
1 / 21

Logic-based Semantically Enriched Integration of Multi-Feature MIR - PowerPoint PPT Presentation


  • 75 Views
  • Uploaded on

Logic-based Semantically Enriched Integration of Multi-Feature MIR. Dominik Lübbers Computer Science Department V (Information Systems) Prof. Matthias Jarke RWTH Aachen, Germany. Some Personal Information. Name: Dominik Lübbers

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Logic-based Semantically Enriched Integration of Multi-Feature MIR' - iolani


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Logic based semantically enriched integration of multi feature mir

Logic-based Semantically Enriched Integration of Multi-Feature MIR

Dominik Lübbers

Computer Science Department V (Information Systems) Prof. Matthias Jarke

RWTH Aachen, Germany


Some personal information
Some Personal Information Multi-Feature MIR

  • Name: Dominik Lübbers

  • Ph.D. student at CS Department V(Information Systems, Prof. Jarke), RWTH Aachen, Germany

  • main CS research interests:

    • Multimedia Databases

    • Information Retrieval

    • Formal Logics, esp. Description Logics and its applications

    • Data Mining

    • Data Quality

  • Musical interests

    • Playing piano & organ

    • part-time church musician

    • Singing in (mainly chamber) choirs


Motivation
Motivation Multi-Feature MIR

  • One reason for difficulty of MIR:

    • User cannot express his information demand easily

    • Multiple features as „hints“

      • Standard metadata (author, title, …)

      • Melody Similarity (e.g. Query by Humming)

      • Sound Similarity (e.g. Query By Example)

      • Lyric fragments („classical“ information retrieval)

    • Queries cannot be understood without respecting (user‘s) background knowledge

  • Central goal: Formalism and Retrieval Mechanisms to

    • Integrate similarity-based queries with

    • Ontology-based Information Retrieval

      => Intelligent Music Information Retrieval Systems


Overall approach
Overall Approach Multi-Feature MIR

  • Based on previous work done by Umberto Straccia

    • Relevance description logics

    • Fuzzy description logics

    • Ph.D thesis „Foundations of a logic based approach to Multimedia Document Retrieval“ (1999)

  • Information retrieval as logical inference (van Rijsbergen):

    • Relevance of for


Overall approach1
Overall Approach Multi-Feature MIR

Form

Semantics

Piece

Instrument

playedBy

hasRecording

class hierarchy

Recording

Organ

Piano

Orchestra

Int_As

Track 1

bwv565

organKreuzkircheBonn

Track 2

hasRecording

playedBy

MIDI file

recSimonPreston

hasRecording

Track 10

Measure 10-15

recHistoricTelecasts

chicagoSymphonyOrchestra

playedBy

media dependant information

OO model

media independant information

Non-standard DL model


Agenda
Agenda Multi-Feature MIR

  • Object oriented modelling of media dependant information (form part)

  • Stepwise development of suitable logic

    • Standard Description Logics

    • Inconsistencies and Relevance Logics

    • Imprecision and Fuzzy Logics

  • Reasoning about form in

  • Reasoning about form and semantics

  • (Some of the many) open questions


Classic description logics i
Classic Description Logics I Multi-Feature MIR

  • Main purpose: Ontology formalization

    • Represent relationships between terms in a domain of interest

  • Many application areas:

    • Conceptual Data Modelling

    • Semantic Query Optimization

    • Software Engineering

    • Configuration Management

    • Representation of the meaning of Web resources: Semantic Web (OWL-DL is a Description Logic), …

  • Good compromise between expressivity and computational complexity

  • Thoroughly investigated family of logics with many variantsWe concentrate on basic DL


Classic description logics ii
Classic Description Logics II Multi-Feature MIR

: TBox (terminological knowledge)

s : ABox (Assertions, statements about objects in the KB)


Relevance and inconsistency
Relevance and Inconsistency Multi-Feature MIR

  • Material implication allows for valid sentences, although is not related to

    • (Inconsistent knowledge base)

      • Since this piece is by Hensel and by Mendelssohn, Beethoven wrote 11 symphonies

    • (tautologies independent of premise)

      • Since Beethoven wrote 11 symphonies,this piece is by Mendelssohn or not.

  • Relevance Logic: Reject logical inferences where the premise is not relevant to the conclusion


Relevance and inconsistency ii
Relevance and Inconsistency II Multi-Feature MIR

  • Avoid „fallacies of relevance“ by four-valued logic

  • Denotational semantics:

    • Explicitly and independently interpret falsehood

Be aware:


Imprecision and fuzzy logic
Imprecision and Fuzzy Logic Multi-Feature MIR

  • Semantic model of application domain is imperfect and can contain vague concepts (think of genre…)

  • Approach:

    • Replace crisp interpretation of conceptsby fuzzy interpretations:

    • Fuzzy assertions:

    • Terminological constraints:

  • Allows for integration of form-based similarity measures as fuzzy predicates


  • Horn rules allow basic reasoning with Multi-Feature MIRfuzzy n-ary predicates

  • Specify combination of evidence by nondecreasing function of membership degrees

  • So far: no recursive rules possible

  • Queries:


Reasoning about form
Reasoning about form Multi-Feature MIR

  • OO model of media-dependant data as concrete domain in with fixed interpretation

  • Represent similarity as fuzzy binary predicate, e.g.

  • Combine similarity queries, e.g.


Reasoning about form semantics
Reasoning about form + semantics Multi-Feature MIR

  • Link form and semantics by (fuzzy) predicate Int_As


Some example queries
Some example queries Multi-Feature MIR

Find all MIDI files that have melody A, contain lyrics B and are transcribed organ pieces

Find all MIDI files that additionally sound similar to C (somewhere)


Some open questions
(Some) Open Questions Multi-Feature MIR

  • What parts of the model language do we need for music information? What must be extended?

  • What are suitable models for the form perspective of music data? What are meaningful ontologies?

  • How to integrate similarity measures? What interfaces are meaningful?

  • How to combine membership degrees?

  • Complexity issues, „query plan evaluation“

  • User interfaces for query formulation, …



Modelling the form dimension
Modelling the form dimension Multi-Feature MIR

  • Modelling of media-dependent information according to OO-principles

  • Classify documents by assigning a class with defined attributes (+ types)

  • Basic class: MDO (media data object) represents linear stream of bytes

  • Organize classes in a specialization hierarchy

  • Describe parts of MDOs as CSMO (complex single media objects) by Region-functions

  • Aggregate CSMO to more complex CSMO

  • Model similarity measurement as methods in classes


Modelling the form dimension example
Modelling the form dimension - Example Multi-Feature MIR

MDO

MidiFile

CSMO

MidiTrack

MidiTrack provides method


Classic description logics
Classic Description Logics Multi-Feature MIR

  • Syntax

    • Primitive concepts: (~ unary predicates)

    • Roles: (~ binary predicates)

    • Concept term operators

  • Semantics

    • Interpretation


Projects and competences at i5
Projects and Competences at i5 Multi-Feature MIR

ConceptBase 

SEWASIE 

ProLearn 

CRC 427: Media&Cultural Communication 

SFB 476 Improve 

PRIME 

DWQ 

MEMO 

  • Deductive Databases

  • Conceptual Modelling

  • Formal Logics, esp. Description Logics

  • Multimedia Data Management, esp. MPEG-7

  • Service-oriented Information Systems

  • Ontology Engineering

  • Semantic Web

  • Data Quality

  • Data Mining

  • E-Commerce, esp. Electronic Marketplaces


ad