Robust Image Topological Feature Extraction

1. Robust Image Topological Feature Extraction Kārlis Freivalds, Paulis Ķikusts University of Latvia Theory Days at Jõulumäe October 2008

2. Image Analysis • Image analysis steps • Pixels  features • Features  objects • Objects  knowledge • Knowledge  decision • Features • Edge • Ridge • Valley (ravine) • Corner

3. Topological Features • Image is treated as 3D surface • Ridge • Valley • Junction • Peak • Pit • Saddle • (Edge)

4. Ridges in Images

5. Ridge Image Examples Text Line drawings Natural objects Industrial applications

6. Mathematics • Image – smooth function f(x, y) • Derivatives • Directional derivative in direction p = (px, py) • Second derivative in two directions (unit vectors) p and q

7. Principal Orthogonal Directions • Two directions play thekey role • Gradient direction • Tangent direction h = (hx, xy) = (–gy, gx). • fh = 0 • Decision is based on fgg, fhhandfhg

8. Ridge Definition • Point (x, y) is called a ridge point if and only if

9. Analytical example f(x, y) = y2 – x2 K3: xy = 0 gives x = 0 and y = 0 K1: K2: always true Ridge: x = 0

10. Another example f(x, y) = –y2 – 10x2 K3:xy= 0 gives x = 0 and y = 0 K1: always true K2: Ridge: x = 0 Theorem: Ridge of a quadratic surface is always a line.

11. Naive implementation • fx = (f(k + 1, l) – f(k – 1, l)) / 2, • fy = (f(k, l + 1) – f(k, l – 1)) / 2, • g = • gx = fx / g, • gy = fy / g, • hx = – gy, • hy = gx, • fxx = f(k + 1, l) – 2f(k, l) + f(k – 1, l), • fyy = f(k, l + 1) – 2f(k, l – 1) + f(k, l – 1), • fxy = (f(k + 1, l + 1) + f(k – 1, l – 1) – f(k + 1, l – 1) – f(k – 1, l + 1)) / 4, • fgg = fxxgxgx + 2fxygxgy + fyygygy, • fhh = fxxhxhx + 2fxyhxhy + fyyhyhy, • fgh = fxxgxhx + fxy(gxhy + gyhx) + fyygyhy. Ridge conditions:

12. Naive implementation

13. Equivalent Formulation • Hessian matrix • Eigenvalues of H • Eigenvectors of H • Ridge: R. Haralick, “Ridges and Valleys on Digital Images,” Computer Vision, Graphics, and Image Processing, vol. 22, no. 10, pp. 28–38,Apr. 1983.

14. Algorithm • Calculate eigenvalues and eigenvectors • See if • See if there is a sign change of in the direction of • Problems: • Unreliable • Requires interpolation • Relatively Slow

15. Lee and Kim Algorithm • 4 directions of discrete grid are analyzed • Orthogonal directions of greatest curvature are selected • Point is classified as ridge, valley, peak or pit based on sign changes of gradients in these directions. • Problems: • Unreliable • No clear distinction between ridges and peaks Lee, S. and Kim, Y. J. 1995. Direct Extraction of Topographic Features for Gray Scale Character Recognition. IEEE Trans. Pattern Anal. Mach. Intell. 17, 7 (Jul. 1995), 724-729.

16. New Discrete Algorithm • Consider 4 directions of discrete grid • Find direction of minimum second derivative (that will be direction perpendicular to ridge), let fhh be that second derivative. • Test if the current point is a local maximum in that direction • Let fgg be the second derivative in the perpendicular direction. • Test if abs(fhh) >= abs(fgg) • If all tests are true return abs(fhh) as ridge strength • Else return that this is not a ridge point

17. Implementation • // Calculation of ridge pixel strength at the given point, 0 if the point is not a ridge point • // assumes that int *image holds the image values in row by row. • int RidgePixelStrengthOptimized(int x, int y){ • int *p = image + (x + y*width); • int f = *p; • int fpp[4]; int fhh,fhhI, fhh_tmp, fhhI_tmp; • fpp[0] = 2*(f - p[1] - p[-1]); // calculate magnitudes of directional second derivatives • fpp[1] = - p[width+1] - p[-width-1]; • fpp[2] = 2*(f - p[width] - p[-width]); • fpp[3] = - p[width-1] - p[-width+1]; • if (fpp[1]>fpp[0]){fhh = fpp[1];fhhI = 1;} else {fhh = fpp[0];fhhI = 0;}// find the maximum one • if (fpp[2]>fpp[3]){fhh_tmp = fpp[2];fhhI_tmp = 2;} else {fhh_tmp = fpp[3];fhhI_tmp = 3;} • if(fhh_tmp > fhh){fhh = fhh_tmp;fhhI = fhhI_tmp;} • fhh += 2*f; // diagonal • if(fhh<lowerThreshold) return 0 ;// optimization: early exit • int fgg = fpp[(fhhI+2)&3] + 2*f; // second derivative in the orthogonal direction • int f1Ofs[4] = {1, width+1, width, width-1}; • int ofs = f1Ofs[fhhI]; • if(p[ofs] < f && p[-ofs] < f && abs(fgg) <= fhh) return fhh; • return 0; • }

18. Results

19. Sensitivity to Noise

20. Sensitivity to Noise (1)

21. Sensitivity to Noise (2)

22. Sensitivity to Noise (3) Note: All images were processed with the same parameters

23. Results • Connected lines • Thin lines • No artifacts • Low noise sensitivity • High performance 45MPix/s = 108 video frames(720 × 576) per second (AMD Athlon 2.6GHz)

24. Compared to Lee and Kim Algorithm • Simpler • Faster • No spurious ridges Original image Lee & Kim Freivalds & Ķikusts

25. Scale Selection • Gaussian blurring (Geusebroek et.al. 2002) • Scale space and automatic scale selection (Lindeberg 1996) • Adaptive blurring (Elder, Zucker 1998) Geusebroek, J., Smeulders, A. W., and Weijer, J. v. 2002. Fast Anisotropic Gauss Filtering. In Proceedings of the 7th European Conference on Computer Vision-Part I. LNCS, vol. 2350. pp. 99-112. T. Lindeberg: "Edge detection and ridge detection with automatic scale selection", International Journal of Computer Vision, vol 30, number 2, pp. 117--154, 1998. Earlier version presented at IEEE Conference on Pattern Recognition and Computer Vision, CVPR'96, San Francisco, California, pages 465--470, june 1996 Elder, J. and Zucker, S. 1998. Local scale control for edge detection and blur estimation. IEEE Pattern Anal. Machine Intell., 20(7): 699–716

26. Topological Edge Detection • Edge conditions: • fgg= 0 • fggg < 0 • Algorithm • Find the discrete direction of maximum gradient • See if the second derivative has sign change from positive to negative in that direction

27. Comparison to Canny Detector • Similar quality • Simpler • Faster Original Canny Proposed

28. Results – Shooting Target

29. Conclusion • New robust ridge, edge detection algorithms • High quality • High performance; suitable for real-time applications • Simple implementation • Straightforward hardware implementation