Con text text detection using background connectivity for fine grained object classification
1 / 15

Con-Text: Text Detection Using Background Connectivity for Fine-Grained Object Classification - PowerPoint PPT Presentation

  • Uploaded on

Con-Text: Text Detection Using Background Connectivity for Fine-Grained Object Classification. Sezer Karaoglu, Jan van Gemert, Theo Gevers. Can we achieve a better object recognition with the help of scene-text ?. Goal.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about ' Con-Text: Text Detection Using Background Connectivity for Fine-Grained Object Classification' - jania

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Con text text detection using background connectivity for fine grained object classification

Con-Text: Text Detection Using Background Connectivity for Fine-Grained Object Classification

Sezer Karaoglu, Jan van Gemert, Theo Gevers

Can we achieve a better Fine-Grained Object Classificationobject recognition with the help of scene-text?

Goal Fine-Grained Object Classification

  • Exploit hidden details by text in the scene to improve visual classification of very similar instances.

DJ SUBS Breakfast

Starbucks Coffee







Application : Linking images from Google street view to textual business inforation as e.g. the Yellow pages, Geo-referencing, Information retrieval

Challenges of text detection in natural scene images
Challenges of Text Detection in Natural Scene Images Fine-Grained Object Classification

  • Lighting

  • Surface Reflections

  • Unknown background

  • Non-Planar objects

  • Unknown Text Font

  • Unknown Text Size

  • Blur

Literature review text detection
Literature Review Fine-Grained Object Classification Text Detection

  • Texture Based:

    Wang et al. “End-to-End Scene Text Recognition”

    ICCV ‘11

  • Computational Complexity

  • Dataset specific

  • Do not rely on heuristic rules

  • Region Based:

    Epshtein et al. “Detecting Text in Natural Scenes

    with Stroke Width Transform ” CVPR ‘10

  • Hard to define connectivity

  • Segmentation helps to improve ocr performance

Motivation to remove background for text detection
Motivation to remove background Fine-Grained Object Classification for Text Detection

  • To reduce majority of image regions for further processes.

  • To reduce false positives caused by text like image regions (fences, bricks, windows, and vegetation).

  • To reduce dependency on text style.

Propose d text detection method
Propose Fine-Grained Object Classificationd Text Detection Method

Text detection by BG


Automatic BG seed selection

BG reconstruction

Background seed selection
Background Seed Selection Fine-Grained Object Classification

  • Color, contrast and objectness responses are used as feature.

  • Random Forest classifier with 100 trees based on out-of-bag error are used to create forest.

  • Each tree is constructed with three random features.

  • The splitting of the nodes is made based on GINI criterion.

Original Image

Color Boosting



Conditional dilation for bg connectivity
Conditional Fine-Grained Object ClassificationDilationfor BG connectivity

where B is the structring element (3 by-3 square), M is the binary image where bg seeds are ones and X is the gray level input image



Text recognition experiment s
Text Recognition Fine-Grained Object ClassificationExperiments

  • ICDAR’03 Dataset with 251 test images, 5370 characters, 1106 words.

Icdar 2003 dataset char recognition results
ICDAR 2003 Dataset Fine-Grained Object ClassificationChar. Recognition Results

The proposed system removes 87% of the non-text regions where on average

91% of the test set contains non-text regions. It retains approximately %98 of text regions.

Imagenet dataset
ImageNet Dataset Fine-Grained Object Classification


Country House

Discount House




  • ImageNet building and place of business dataset ( 24255 images 28 classes, largest dataset ever used for scene tekst recognition)

  • The images do not necessarily contain scene text.

  • Visual features : 4000 visual words,standard gray SIFT only.

  • Text features: Bag-of-bigrams , ocr resultsobtained for each image in the dataset.

  • 3 repeats, to compute standard deviations in Avg. Precision.

  • Histogram Intersection Kernel in libsvm.

  • Text only, Visual only and Fused results are compared.

Fine grained building classification results
Fine Fine-Grained Object Classification-GrainedBuildingClassificationResults

ocr : 15.6 ± 0.4




Bow + ocr : 39.0 ± 2.6

Bow : 32.9 ± 1.7

Discount House
















Conclusion Fine-Grained Object Classification

  • Background removal is a

    suitable approach for scene

    text detection

  • A new text detection method,

    using background connectivity

    and, color, contrast and

    objectness cues is proposed.

  • Improvedperformance to scene

    text recognition.

  • Improved Fine-Grained Object

    Classification performance

    with visual and scene text

    information fusion.

DEMO Fine-Grained Object Classification