semantic video classification based on subtitles and domain terminologies l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Semantic Video Classification Based on Subtitles and Domain Terminologies PowerPoint Presentation
Download Presentation
Semantic Video Classification Based on Subtitles and Domain Terminologies

Loading in 2 Seconds...

play fullscreen
1 / 15

Semantic Video Classification Based on Subtitles and Domain Terminologies - PowerPoint PPT Presentation


  • 182 Views
  • Uploaded on

Semantic Video Classification Based on Subtitles and Domain Terminologies. Polyxeni Katsiouli, Vassileios Tsetsos, Stathes Hadjiefthymiades P ervasive C omputing R esearch G roup C ommunication N etworks L aboratory Department of Informatics and Telecommunications

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Semantic Video Classification Based on Subtitles and Domain Terminologies' - eben


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
semantic video classification based on subtitles and domain terminologies

Semantic Video Classification Based on Subtitles and Domain Terminologies

Polyxeni Katsiouli, Vassileios Tsetsos, Stathes Hadjiefthymiades

Pervasive Computing Research Group

Communication Networks Laboratory

Department of Informatics and Telecommunications

University of Athens – Greece

KAMC ‘07 @ Genoa, Italy

polina@di.uoa.gr

polina@di.uoa.gr

polina@di.uoa.gr

b.tsetsos@di.uoa.gr

b.tsetsos@di.uoa.gr

b.tsetsos@di.uoa.gr

shadj@di.uoa.gr

shadj@di.uoa.gr

outline
Outline
  • The Polysema Platform
  • Introduction-Motivation
  • VideoCategorizationMethod
  • ExperimentalEvaluation
  • Conclusions - Future Work
polysema platform
Polysema platform
  • Develops an end-to-end platform for iTV services
  • Semantics-related research focuses on the development of:
    • semantics extraction techniques for automatic annotation of audiovisual content,
    • a personalization framework for iTV services with SW technologies,
    • a tool with GUI for video annotation and MPEG-7 metadata creation

http://polysema.di.uoa.gr

introduction motivation
Introduction - Motivation
  • Multimedia databases are becoming popular
  • Most video classification methods are based on visual/audio signal processing
  • Text processing is more lightweight than visual/audio processing
  • High-level semantics are more closely related to human language than to visual features
  • Subtitles capture the semantics of the corresponding video
step 1 text preprocessing
Step 1: Text Preprocessing
  • Subtitles are segmented into sentences
  • A Part of Speech Tagger is applied to each sentence
  • Stop words (e.g., “to”, “him”) are removed based on a stop words list
step 2 keyword extraction
Step 2: Keyword extraction
  • We used the TextRank algorithm to extract keywords
  • TextRank
    • represents the text as a graph,
    • applies to the vertices a ranking algorithm based on Google’s PageRank,
    • sorts vertices in decreasing rank order,
    • extracts the top highly ranked vertices for further processing

TextRank:

Mihalcea, R., Tarau, P.: TextRank: Bringing Order into Texts, in Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP 2004), Barcelona, Spain, July 2004

step 3 word sense disambiguation
Step 3: Word Sense Disambiguation
  • Words have many possible meanings, called senses
  • A Word Sense Disambiguation (WSD) algorithm is applied to determine the correct sense of each word
  • WSD
    • is based on the lexical database WordNet,
    • is a variation of Lesk’s WSD algorithm

WSD:

Banerjee, S., Pedersen, T.: An Adapted Lesk Algorithm for Word Sense Disambiguation Using WordNet. In the Proceedings of the 3rd International Conference on Intelligent Text Processing and Computational Linguistics (CICLING-02) Mexico City, Mexico (2002)

step 4 wordnet domains extraction 1 2
Step 4: WordNet Domains Extraction (1/2)
  • augment WordNet with domain labels
  • a taxonomy of ~200 domain labels
  • synsets have been annotated with at least one domain label

WordNet domains

WN domains:

http://wndomains.itc.it/wordnetdomains.html

step 4 wordnet domains extraction 2 2
Step 4: WordNet Domains Extraction (2/2)
  • For each video:
    • Extract the WordNet domains for each keyword’s sense
    • Calculate the frequency occurrence of each domain label
    • Sort domain labels in decreasing order according to their occurrence frequency
step 5 correspondences between categories wn domains
Step 5: Correspondences between categories & WN domains
  • For each category label:
    • Look up in WordNet the senses related to it (include senses related through hypernym & hyponym relations)
    • Obtain the corresponding WordNet domains
    • Calculate the occurrence score for each domain
    • Sort domains in decreasing occurrence order

Example:

step 6 category label assignment
Step6: Category label assignment
  • Compare the ordered list with the WN domains of each video with the ordered list of the WN domains of each category

Example:

WN domains of a video

science

animals

experimental evaluation 1 2
Experimental Evaluation (1/2)
  • 36 subtitle files of documentaries
  • 36 subtitle files of documentaries

Statistical information of files (average values):

  • Classify under the categories: geography, animals, history, war, technology, science, accidents, music, transportation, people, religious, politics, arts
  • Classify under the categories: geography, animals, history, war, technology, science, accidents, music, transportation, people, religious, politics, arts
experimental evaluation 2 2
Experimental Evaluation (2/2)
  • Classifiers:
    • Proposed method
    • Proposed method in which Step 6 has been replaced with Spearman’s footrule distance
    • J4.8
      • decision tree classifier
      • supervised approach
conclusions future work
Conclusions – Future Work
  • Conclusions
    • A novel approach that is based only on text and uses natural language processing techniques
    • No training phase is required (unsupervised approach)
  • Future Work
    • The application of a method on a per video segment basis
    • Definition of domain knowledge more close to movie classification
    • Performance comparison with other unsupervised approaches
thank you
Thank you!

Questions???

http://p-comp.di.uoa.gr