Connect the Dots

1. Connect the Dots Introduction to Computational and Biological Vision Kerenshemesh

2. Introduction • The game • educative and creational • Motivation & Goal • Kids often play the game alone. • Errors causing false understanding of the numerical system. • Kids can check themselves. • Known topics in computer vision: • OCR • Hough and Circular Hough transform

3. Assumptions • Dot Circles • circle-shaped in range of 5 to 15 • approximately the same size and shape • filled black • All numbers • same font size • All digits • can easily be isolated and extracted • horizontal • the intensity of the components is clearly different from the intensity of the background

4. Principles of the Algorithm: stage 1 • finds all connected dots in the image • Finding all circles using Circular Hough Transform • Creating bounding box • Filter circles • Ratio of white-pixels/pixels in the bounding box < threshold • The center of the circle is black

5. Principles of the Algorithm: stage 1

6. Principles of the Algorithm: stage 2 • finding all numbers in the image • Extracting all connected components

7. Principles of the Algorithm: stage 2 • filtering out the potential connected dots while assuming that a connected dot is not ‘in’ a number And vice versa

8. Principles of the Algorithm: stage 2 • filtering out components not on the average height • filtering out components not on the average width

9. Principles of the Algorithm: stage 2 • Link all digit in the same number

10. Principles of the Algorithm: stage 3 • Interpreting each digit using OCR detection • pre-saved database of font templates for every number 0-9 • the best fit correspondence between a component and the database is calculated using correlation

11. Principles of the Algorithm: stage 3

12. Principles of the Algorithm: stage 4 • matching the dot circles with the numbers by proximity • Match the closest digit of every number to every dot circle • Only the closest circle is matched to each number - removing spare circles • All linked digits are calculated to a number • All connected dots with matched numbers are sorted by the numbers

13. Principles of the Algorithm: stage 4

14. Principles of the Algorithm: stage 5 + 6 • Drawing lines between connected dots with sequential numbers using interpolation • Filing the image with random numbers

15. Results

16. Known issues • False identification of number • False identification of connected dot • Inaccurate OCR detection due to an unknown font • False identification of connected dot and a number

17. Discussion and Conclusions • Correct identification depends on the components in the image • Size • Proximity to others • Font of the numbers

18. The application

19. Questions?