A lightweight image retrieval system for paintings
This presentation is the property of its rightful owner.
Sponsored Links
1 / 23

A Lightweight Image Retrieval System for Paintings PowerPoint PPT Presentation


  • 52 Views
  • Uploaded on
  • Presentation posted in: General

A Lightweight Image Retrieval System for Paintings. T. Lombardi, S. Cha, and C. Tappert January 19th, 2005. Introduction. Students of art history learn three primary skills: Formal analysis Comparison Classification How can computer science contribute to the development

Download Presentation

A Lightweight Image Retrieval System for Paintings

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


A lightweight image retrieval system for paintings

A Lightweight Image Retrieval System for Paintings

T. Lombardi, S. Cha, and C. Tappert

January 19th, 2005


Introduction

Introduction

Students of art history learn

three primary skills:

  • Formal analysis

  • Comparison

  • Classification

    How can computer science

    contribute to the development

    of these skills?

Figure 1: Girl with a Pearl Earring,

Jan Vermeer, 1665

Electronic Imaging 2005


Working hypothesis

Working Hypothesis

  • An Interactive Indexing and Image Retrieval System (IIR) for fine-art paintings can aid students in these endeavors by providing:

    • a mathematical summarization of an image

    • a measurable basis for comparing two images

    • an elementary way to classify an image relative to those in a database

Electronic Imaging 2005


Previous work

Previous Work

We synthesize the goals of two research areas:

  • Classification of paintings:

    • R. Sablatnig, P. Kammerer, and E. Zolda, “Hierarchical Classification of Paintings Using Face- and Brush Stroke Models”, in Proc. of the 14th International Conference on Pattern Recognition (1998).

    • D. Keren, “Painter Identification Using Local Features and Naïve Bayes”, in Proc. of the 16th International Conference on Pattern Recognition (2002).

  • Image retrieval which aims to bridge the semantic gap:

    • J. Corridoni, A. Del Bimbo, and P. Pala, “Retrieval of Paintings using Effects Induced by Color Features”, in Proc. of the International Workshop on Content-Based Access of Image and Video Databases (1998).

  • Can we construct a feature set that satisfies the objectives of both areas while providing analytically relevant data to students?

Electronic Imaging 2005


System overview

System Overview

The system consists of two major components:

  • Image Database

    • stores images, thumbnail images, and extracted features for later retrieval and analysis.

  • Graphical User Interface

    • provides interactive query capabilities to the end user

Electronic Imaging 2005


Database construction

Database Construction

  • An XML index file stores extracted features and control information.

  • A file system stores images and thumbnail images.

  • The open design of the database contributes to the goals of ease of use and exchange of information.

Electronic Imaging 2005


Database construction cont

Database Construction – Cont.

Figure 3: File System

Figure 2: XML Index File

Electronic Imaging 2005


Global feature extraction

Global Feature Extraction

Two different kinds of features are extracted:

  • Palette features

    • concern the set of colors in an image (color map)

    • examples: palette scope

  • Canvas features

    • concern the spatial and frequency distribution of colors in an image (image index)

    • examples: max, min, median, mean (for each color channel)

Electronic Imaging 2005


Sample feature set

Sample Feature Set

Table 1: Sample Features used for Web Museum Interactive Test

Electronic Imaging 2005


Example palette scope

Example: Palette Scope

Figure 4: Hallucinogenic Toreador

Salvador Dali, 1970

Figure 5: Composition with Large Blue Plane,

Red, Black, Yellow, and Gray

Piet Mondrian, 1921

Palette Scope -- the total number of unique colors used in an image.

We expect Dali’s piece to have a higher palette depth than Mondrian’s work.

Electronic Imaging 2005


Example palette scope cont

Example: Palette Scope – Cont.

Formal definition of Palette Scope (U):

U = C/P

Where

C=Total # of unique colors measured in RGB or HSV triples.

P= Total # of pixels in an image.

Electronic Imaging 2005


Example palette scope cont1

Example: Palette Scope – Cont.

Table 2: Palette Scope statistics.

We see that Dali uses more of the color spectrum than Mondrian.

Palette depth is an important feature for artist and period style identification because many styles are defined by color, i.e. Picasso’s Blue Period and fauvism.

Electronic Imaging 2005


Graphical user interface

Graphical User Interface

  • The GUI consists of three primary windows for:

    • Analysis

    • Comparison

    • Classification

Electronic Imaging 2005


Analysis window

Analysis Window

Figure 6: The Analysis Window

Electronic Imaging 2005


Comparison window

Comparison Window

Figure 7: The Comparison Window

Electronic Imaging 2005


Classification window

Classification Window

Figure 8: The Classification Window

Electronic Imaging 2005


Test results

Test Results

Two types of tests were conducted:

  • Feature tests

    • Feature tests focus on the accuracy of specific collections of features.

  • Interactive tests

    • Interactive tests assess the accuracy of the system as a whole.

Electronic Imaging 2005


Feature test

Feature Test

Figure 9: Les Demoiselles d’Avignon,

Pablo Picasso, 1907.

Figure 10: Road with Cypress and Star,

Vincent Van Gogh, 1890.

Table 3: Feature test to distinguish the work of Picasso and Van Gogh.

Electronic Imaging 2005


Initial interactive test

Initial Interactive Test

Database of 10 works of each of the following ten artists:

Braque, Cezanne, De Chirico, El Greco, Gauguin,

Modigliani, Mondrian, Picasso, Rembrandt, and Van

Gogh.

Table 4: Initial Interactive Test

Electronic Imaging 2005


Interactive test web museum

Interactive Test: Web Museum

Table 5: Results from Web Museum Interactive Test

Electronic Imaging 2005


Evaluation of web museum test results

Evaluation ofWeb Museum Test Results

  • Overall result: 56.3% accuracy

    • 36.3% better than blind guessing (10 guesses/50 artists = 20%)

  • Dissecting the classification mistakes reveals some intelligent mistakes

    • Rembrandt is most often confused with Caravaggio, Ast, and Vermeer

Electronic Imaging 2005


Conclusions

Conclusions

  • Simple palette and canvas features are sufficient for an interactive classification system

  • A single feature set can serve for classification and image retrieval applications

  • A general feature set can adequately serve for educational applications

  • Although showing promise, we currently have a low confidence system

Electronic Imaging 2005


Future work

Future Work

  • Add texture features

  • Improved color features: hue histograms

  • Improved distance metrics: modulo comparison of hue histograms

  • Test against larger datasets

Electronic Imaging 2005


  • Login