1 / 43

Détection des textes dans les images issues d’un flux vidéo pour l´indexation sémantique

Détection des textes dans les images issues d’un flux vidéo pour l´indexation sémantique. 1 juillet 2004. Christian Wolf. Christian.wolf@liris.cnrs.fr http://rfv.insa-lyon.fr/~wolf. Laboratoire d'Informatique en Images et Systèmes d'information LIRIS, FRE 2672 CNRS

bbills
Download Presentation

Détection des textes dans les images issues d’un flux vidéo pour l´indexation sémantique

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Détection des textes dans les images issues d’un flux vidéo pour l´indexation sémantique 1 juillet 2004 Christian Wolf Christian.wolf@liris.cnrs.fr http://rfv.insa-lyon.fr/~wolf Laboratoire d'Informatique en Images et Systèmes d'information LIRIS, FRE 2672 CNRS Bât. Jules Verne, INSA de Lyon 69621 Villeurbanne cedex

  2. Plan Features Introduction Evaluation/ Choice of features Text detection Experimental Results Conclusion Introduction Features Evaluation Text detection Results Conclusion

  3. Result Key word keyword-based Search Patrick Mayhew Indexing phase Patrick Mayhew Min. chargé de l´irlande de Nord ISRAEL Jerusalem montage T.Nouel ... ... Introduction Features Evaluation Text detection Results Conclusion Image/video indexing • Content based image retrieval (Master’s degree): • Query by example: • Indexing based on local texture (Gabor) features • Video indexing using semantic descriptors (PhD) : • Text detection, enhancement, segmentation and recognition.

  4. Detection Enhancement Segmentation “Soukaina Oufkir” Introduction Features Evaluation Text detection Results Conclusion Text detection

  5. Heuristics Separate populations (discriminant analysis) Learning a model (SVM, etc.) Reinforcement learning Master’s thesis of Graham Taylor Introduction Features Evaluation Text detection Results Conclusion Detection in an image • Problems: • Which features? • How can the decision be taken (text - non-text)? Contrast and Edge features Geometrical features Corner features Texture features Color features Region/stroke segmentation

  6. Introduction Features Evaluation Text detection Results Conclusion Plan Features Introduction Evaluation/ Choice of features Text detection Experimental Results Conclusion

  7. Introduction Features Evaluation Text detection Results Conclusion Videos vs. scanned documents Temporal aspects Complex and moving background Artificial shadows

  8. Introduction Features Evaluation Text detection Results Conclusion Videos vs. scanned documents • Low resolution • Low quality • Antialising artifacts • Compression artifacts • Color bleeding

  9. Introduction Features Evaluation Text detection Results Conclusion What is text? - character segmentation Scene text Artificial text

  10. Introduction Features Evaluation Text detection Results Conclusion What is text? - texture Original image Filter tuned to the example text Example: Gabor energy features on a text image Gabor energy Thresholded Gabor energy

  11. Introduction Still images Videos Character segmentation Results Indexing What is text? - texture

  12. Derivative 2nd derivative smeared Introduction Features Evaluation Text detection Results Conclusion What is text? - corners Unthresholded “Harris” corner response

  13. Introduction Features Evaluation Text detection Results Conclusion What is text? - contrast & geometry Example image Accumulated horizontal Sobel edges

  14. Sobel on grayscale image Modified Sobel on L*u*v* image Introduction Features Evaluation Text detection Results Conclusion What is text? - color • Special cases of text: • Small contrast in the lumination plane • High(er) contrast in the color plane Original image

  15. Introduction Features Evaluation Text detection Results Conclusion Plan Features Introduction Evaluation/ Choice of features Text detection Experimental Results Conclusion

  16. Introduction Features Evaluation Text detection Results Conclusion Evaluation • A good evaluation algorithm permits: • A simple and intuitive interpretation of the obtained performance • An objective comparison between the different algorithms to evaluate • A good correspondence between the performance measures and the real performance, taking into account the objective of the algorithm (goal oriented approach) • Takes into account only the performance of the algorithm, without side effects of other processing steps

  17. Introduction Features Evaluation Text detection Results Conclusion Evaluation at different levels Lower influence of later stages Lower computational complexity Detection result Ground truth Statistical separation: Bhattacharyya distance Error rate, Recall/Precision on pixel level Recall/Precision on rectangle level Patrick Mayhew Min. chargé de l´irlande de Nord ISRAEL Jerusalem montage T.Nouel ... ... Patrick Mayhew Min. chargé de l´irlande de Nord ISRAEL Jerusalem montage T.Nouel ... ... Goal oriented: Recall/Precision on character level Higher relevance to the application

  18. Detection Ground truth Introduction Features Evaluation Text detection Results Conclusion Evaluation on rectangle level • Pure overlap is ambiguous on multiple images: • 50% of recall could mean: • 50% of the text rectangles have been detected perfectly • 100% of the rectangles have been detected with 50% surface • Anything between the two ...

  19. Introduction Features Evaluation Text detection Results Conclusion Evaluation on rectangle level • Requirements of an evaluation measure: • Tells intuitively how many rectangles have been detected, and how many false alarms • Measures the detection quality • Takes into account one-2-one, one-2-many and many-2-one matches • Scales up to multiple images Problem: Counts number of correctly detected rectangles Measures the detection quality Contradiction

  20. Ground truth Gi Detection Di “Surface” Recall and Precision: Thresholded by different thresholds on recall and precision Introduction Features Evaluation Text detection Results Conclusion Performance graphs For each rectangle, we will know whether it has been detected or not, depending on a quality threshold

  21. Introduction Features Evaluation Text detection Results Conclusion Performance graphs Threshold on surface recall Threshold on surface precision

  22. Introduction Features Evaluation Text detection Results Conclusion Comparison of different detection algorithms Method 1: Local contrast Method 2: SVM Learning

  23. Introduction Features Evaluation Text detection Results Conclusion The influence of the test database Local contrast SVM learning

  24. Introduction Features Evaluation Text detection Results Conclusion Plan Features Introduction Evaluation/ Choice of features Text detection Experimental Results Conclusion

  25. Introduction Still images Videos Character segmentation Results Conclusion The local contrast method Calculate a text probability image according to a text model (1 value/ pixel) F. LeBourgeois Separate the probability values into 2 classes. Fisher/Otsu Post processing • Mathematical morphology • Geometrical constraints • Verification of special cases • Combination of rectangles

  26. Introduction Features Evaluation Text detection Results Conclusion The learning method Learning gray values and edge maps alone may not generalize enough. Texture alone is not reliable, especially if the text is short. Geometry is a valuable feature. State of the art: enforce geometrical constraints in the post-processing step (mathematical morphology) We propose the usage of geometrical features very early in the detection process, i.e. not during post-processing.

  27. Introduction Features Evaluation Text detection Results Conclusion Geometrical features: baseline • Text consists of: • A high density of strokes in direction of the text baseline. • A consistent baseline (a rectangular region with an upper and lower border). • Two detection philosophies: • Detection of the baseline directly before detecting the text region. • Detection of the baseline as the boundary area of the detected text region in order to refine the detection quality.

  28. Introduction Features Evaluation Text detection Results Conclusion Estimation of the text rectangle height Original image Accumulated gradients

  29. Introduction Features Evaluation Text detection Results Conclusion Features Mode width (=rectangle height) Mode height (=Contrast) Difference height left-right Mode mean Mode standard deviation Difference in mode width

  30. Training image database Introduction Features Evaluation Text detection Results Conclusion Learning with Support Vector Machines positive samples negative samples Bootstrapping, cross-validation • Classification step: a reduction of the computational complexity is necessary: • Sub-sampling of the pixels to classify (4x4) • Approximation of the SVM model by SVM-regression.

  31. Introduction Features Evaluation Text detection Results Conclusion Plan Features Introduction Evaluation/ Choice of features Text detection Experimental Results Conclusion

  32. Introduction Features Evaluation Text detection Results Conclusion AIM2 Commercials AIM3 News AIM4 Cartoons, News AIM5 News

  33. Introduction Features Evaluation Text detection Results Conclusion Detection in still images Local contrast SVM learning

  34. Introduction Features Evaluation Text detection Results Conclusion Local contrast SVM learning

  35. Introduction Features Evaluation Text detection Results Conclusion Local contrast SVM learning

  36. Introduction Features Evaluation Text detection Results Conclusion Detection in video sequences

  37. Introduction Features Evaluation Text detection Results Conclusion Character segmentation: examples Original image Fisher/Otsu Fisher/Otsu (windowed) Yanowitz-B. Yanowitz-B. +post-proc. Niblack Sauvola et al. Contrast maximiz.

  38. Bayesian estimation using a Markov random field prior Sauvola et al. MRF Introduction Features Evaluation Text detection Results Conclusion OCR results Local contrast based binarization Recognition by Abby Finereader 5.0

  39. “Oil” “Air plane” “Airline” “Dance” Introduction Features Evaluation Text detection Results Conclusion TREC 2002 “Music” Collaboration with Laboratory LAMP, University of Maryland “Energy Gas”

  40. Introduction Features Evaluation Text detection Results Conclusion Conclusion • The choice of features is primordial in vision. • We developed a new system for detection, tracking, enhancement and binarisation of text. • Detection performance is high due to the integration of several types of features in a very early stage. The learning method is less sensitive to textured noise in the image. • We propose a new evaluation method which allows intuitive visualization of the detection quality by performance graphs.

  41. Introduction Features Evaluation Text detection Results Conclusion Outlook • Possible improvement of the features (e.g. contrast normalization, non-linear texture filters). • Integration of different feature types (statistical, structural, ...) • Usage of a priori knowledge on text in order to decrease the number of false alarms • Integration of the detected text into a indexing/browsing/segmentation framework

  42. Optionalslides

  43. The Bhattacharyya distance

More Related