Slide image retrieval a preliminary study
1 / 16

Slide Image Retrieval: A Preliminary Study - PowerPoint PPT Presentation

  • Uploaded on

Slide Image Retrieval: A Preliminary Study . Guo Min Liew and Min-Yen Kan National University of Singapore Web IR / NLP Group (WING). Papers don’t store all of the information about a discovery Dataset Tools Implementation details / conditions

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Slide Image Retrieval: A Preliminary Study' - druce

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Slide image retrieval a preliminary study l.jpg

Slide Image Retrieval: A Preliminary Study

Guo Min Liew and Min-Yen Kan

National University of Singapore

Web IR / NLP Group (WING)

Beyond text in digital libraries l.jpg

Papers don’t store all of the information about a discovery



Implementation details / conditions

They also don’t help a person learn the research



Beyond Text in Digital Libraries

We’ll focus on this

JCDL: Beyond Text


What is slide image retrieval l.jpg
What is Slide Image Retrieval? discovery

  • Hypothesis: image and presentation features can enhance text-based features

JCDL: Beyond Text


Slidir architecture l.jpg

Training discovery

3. Machine Learning

Relevance Judgments

Presentation Slides


1. Pre-


2. Feature


Text + Image + Presentation Features


4. Image



Presentation Slides

Slidir Architecture

JCDL: Beyond Text


1 preprocessing l.jpg
1. Preprocessing discovery

Simple preprocessing, not meant to be state-of-the-art

Generally consistent background  Perform collective separation

Top down approach using projection profile cuts with thresholds from PPT template

JCDL: Beyond Text


Sample extraction results l.jpg
Sample Extraction Results discovery

JCDL: Beyond Text


2 feature extraction l.jpg
2. Feature Extraction discovery

Utilizes textual, image and presentation features

  • Textual Features

    • Main component of feature set

    • Most images identified by surrounding and embedded text

  • Image Features

    • Supplement to textual features

    • Useful in ranking images with similar textual feature scores

  • Presentation Features

    • Features unique to presentation slides

JCDL: Beyond Text


Sample features l.jpg
Sample Features discovery


  • Image Size

  • Image Colors

  • NPIC image type (CIVR, 06)


  • Slide Order

  • Image position on Slide

  • Number of Images on Slide


All text

2. Slide text

3. Next slide’s text

4. Previous slide’s text

5. Slide Title

6. Presentation Title

7. Slide Image Text

JCDL: Beyond Text


Machine learning l.jpg
Machine Learning discovery

Regression Model

  • Relevance judgments and feature vectors input into machine learner

  • Linear regression used to produce continuous output value for image ranking

    Human Relevance Judgments

  • 9 participants

  • Four presentation sets of 10 queries each

  • Participants rank up to five most relevant images to the query

JCDL: Beyond Text


Regression analysis l.jpg
Regression Analysis discovery

  • 10-fold cross validation

  • System with fielded text features able to replicate human upper bound

  • Image and presentation features further improve correlation

Still a gap toimprove

Looks pretty good

Comparison of mean absolute error and correlation

JCDL: Beyond Text


Binary classification l.jpg
Binary Classification discovery

  • Human participants judgments converted to binary

  • Evaluated using SVM and J48 decision tree classifier

  • General accuracy fairly good

  • Decision tree algorithm slightly more precise  sequential testing adequate

10-fold performance over 8062 instances

JCDL: Beyond Text


Feature analysis l.jpg
Feature Analysis discovery

  • Human subject study with 20 volunteers

  • Subjects select most and least relevant images

Sample Human Subject Survey

JCDL: Beyond Text


Slide13 l.jpg

Textual discovery features most important, image and presentation features still play significant role

JCDL: Beyond Text


Image ranking evaluation l.jpg
Image Ranking Evaluation discovery

  • Online survey with 15 volunteers

  • 8 given queries and sampled dataset of 100 images

  • Volunteers indicate relevant images to query

  • 53% of top 10 images are relevant (max of 15 relevant)

  • Average precision of 0.74

JCDL: Beyond Text


Future work l.jpg

Limitations discovery

Very small data set – 100 images from an AI course


Improve segmentation and collective extraction

Field Implementation – Ongoing work

Incorporate with previous work on SlideSeer (JCDL, 2007) that aligns presentation and document pairs

More information:

Liew Guo Min(2008) SLIDIR: Slide Image Retrieval, Undergraduate Thesis, School of Computing, National University of Singapore.

Future Work

JCDL: Beyond Text


Slide16 l.jpg

See you in Singapore! discovery

Next month or next year!


JCDL: Beyond Text