1 / 43

Motivation

Efforts to understand, manipulate, and archive valuable historical handwritten manuscripts through digitization, increasing accessibility and allowing for automatic processing. Provides insights into tangible and intangible cultural aspects from the past.

thodge
Download Presentation

Motivation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Motivation • Historical handwritten manuscripts are valuable cultural heritage • Providing insights into both tangible and intangible cultural aspects from the past •  Efforts to understand, manipulate and archive historical manuscripts • Digitizationincreases accessibility and allows automatic processing *Courtesy: - wadod.com - Genizah Project

  2. Outline • Background • Challenges • Seam Carving • Text line representation by seams • Energy Map • Seam Generation • Experimental Results • Summary

  3. Image representation N x M (Matrix)

  4. Binarization # pixels intensity

  5. Connectivity & Components • We can define 4- or 8-paths depending on the type of connectivity specified • A set of pixels S is a Connected • Componentiffor each pixel pair • (x1,y1) є S and (x2,y2) є S there • is a path between them such that • every two successive pixels in the path • are in S and are X-neighbors. (X = 4, 8). 8-Neighborhood 4-Neighborhood

  6. Connected Component One word, but 3connected components

  7. Distances • Given 2 points P = (u,v) , Q = (x,y) • Euclidean Distance • City Block Distance • Chessboard Distance • In example: P = (1,8); Q = (4,1)

  8. Distance transform • Given a set of pixels S, calculate the distance of other pixels to S • The pixels in the set S will be considered as reference pixels • Let . We scan the image by a pre-defined connectivity : • First pass: Consider Green pixels (N1)

  9. Distance transform • In reverse scan, consider Blue pixels (N2) First scan Distance transform

  10. Distance transform – (cont’d) Alef Letter - Arabic Printed Handwritten Binary Representation Distance transform Chessboard metric = Reference pixels

  11. Sign Distance transform Alef Letter Printed Handwritten Sign Distance transform chessboard metric

  12. Sign Distance transform – (cont’d) • The brighter the color the larger the distance from reference pixels Original Document Image Sign Distance transform (SDT)

  13. Gradient • A gray-scale image I is defined as a two-dimensional function I(x,y)=gray • The gradient of the image (I ) is given by the formula : Where: • is the derivative of the image in the horizontal direction • is the derivative of the image in the vertical direction • The magnitude of the gradient is defined by:

  14. Gradient

  15. Background Pre-Processing Segmentation Original *Courtesy: Islamic manuscript, Leipzig University Library, Germany

  16. Text-line Extraction Assigning the same color to each text line ب ت ث يــجـ خـ حـ Original Manuscript Processed Manuscript *Courtesy: Juma Al-majid Center for Culture and Heritage, Dubai.

  17. Outline • Background • Challenges • Seam Carving • Text line representation by seams • Energy Map • Seam Generation • Experimental Results • Summary

  18. Challenges Historical handwritten documents pose different challenges than those in machine-printed. • Looser layout format • Line Proximity • Multi-Oriented lines • Touching components • Different slope (within the same line) • Delayed strokes • Overlapping components A 19th century master thesis – SAAB medical Library, American University of Beirut

  19. Outline • Background • Challenges • Seam Carving • Text line representation by seams • Energy Map • Seam Generation • Experimental Results • Summary

  20. Seam Carving • Content-aware image resizing • An energy function defines energy value for each pixel • A seam is an optimal 8-connected path of low energy pixels Original Image Calculated seams Gradient Image Resized

  21. Seam Carving – (cont’d) • let I be an n x m size image. Define a vertical seam to be: where x is a mapping x : [1, . . . ,n] [1, . . . ,m]. • Seam contains one, and only one, pixel in each row of the image, otherwise a distorted image might be obtained. • The pixels of the path of a seam will therefore be : • one can change the value of K in the constraint, and get either a simple column for k = 0 , or even completely disconnected set of pixels.

  22. Seam Carving – (cont’d) • Given an energy function e, the cost of a seam is: • We look for the optimal seam s* that minimizes this cost : • The optimal seam can be found using Dynamic programming

  23. Outline • Background • Challenges • Seam Carving • Text line representation by seams • Energy Map • Seam Generation • Experimental Results • Summary

  24. Text line representation by seams • Human perception of text lines • Tracks text lines by ink concentration and in-between line spaces • Two types of seams have been defined *Courtesy: Wadod Center for masnuscripts.

  25. Text line representation by seams-(cont’) • The medial seam crosses the text area of a text line. • ASeparating seam is a path that passes between two consecutive text lines. Original Document Image Seam Seed Medial Seam Separating Seam Processed *Courtesy: Wadod Center for masnuscripts.

  26. Outline • Background • Challenges • Seam Carving • Text line representation by seams • Energy Map • Seam Generation • Experimental Results • Summary

  27. Energy Map • We use the Sign distance transform (SDT) as an energy map • In SDT, pixels values are assigned according to their distance from the nearest reference pixel • Recall, distance values are negative inside connected components and positivein-between • Intuition: Local minima and maxima points determine the medial and separating seams, respectively Original Document Image Sign Distance Transform (SDT) *Courtesy: Wadod Center for masnuscripts

  28. Outline • Background • Challenges • Seam Carving • Text line representation by seams • Energy Map • Seam Generation • Experimental Results • Summary

  29. Seam Generation – (cont’d) • The SDT is traversed horizontally to compute a cumulative energy map - Seam Map - for all possible connected seams for each entry (i,j): • SDT is traversed with two passes to enhance text line patterns Sign distance transform • Bi-linearly interpolate the resulting two maps Right-to-left pass Left-to-right pass Interpolated map

  30. Seam Generation – (cont’d) • The minimal entry of the last column is detected. • Backtrack from the minimal entry to find the medial seam. Original Document Image Seam Map – One pass Seam Map – Two passes

  31. Seam Generation – (cont’d) • Iteratively, all text lines will be extracted

  32. Seam Generation – (cont’d) • Then, why separating seams are needed? • Avoid recalculation of energy and seam maps after each line extraction • Avoid additional strokes classification (post processing)

  33. Seam Generation – (cont’d) • Separating seams define the boundaries of text lines • Generated with respect to the medial seam of the corresponding text line • Grown from seam seeds toward the two sides of the image guided by the SDT

  34. Seam Generation – (cont’d) • Seam fragment is a connected group of pixels defined as the closest local maxima along the vertical direction • Seam fragments with low priority are discarded • Seeds candidate set is constructed • The seed that generates the optimal (maximal cost) seam was chosen Medial Seam Seam Map Sign Distance Transform

  35. Seam Generation – (cont’d) • The separating seams may diverge from the medial seamdue to the fork of ridges • A spring force anchored at the medial seamguides the separating seams Before After

  36. Touching/Overlapping Components • Usually, crossing overlapping components is avoided gracefully • Touching components are split too, but not necessarily in the optimal position Processed Processed

  37. Outline • Background • Challenges • Seam Carving • Text line representation by seams • Energy Map • Seam Generation • Experimental Results • Summary

  38. Experimental Results

  39. Experimental Results- (cont’d) Table 1: correctness of text line extraction Table 2: crossed components

  40. Experimental Results- (cont’d)

  41. Outline • Background • Challenges • Seam Carving • Text line representation by seams • Energy Map • Seam Generation • Experimental Results • Summary

  42. Summary • Summary • Language independent approach • Dynamic programming was used to find text lines • Saves energy map re-computing after text line extraction • Post processing steps are avoided • Crossing overlapping components was avoided in most cases • Still need more research to split touching components optimally

  43. Thank you

More Related