A NEW USE OF AUTOMATIC MORPHING TECHNIQUES: TO CORRECT DISTORTION OF ENDOSCOPIC SYSTEMS

1. A NEW USE OF AUTOMATIC MORPHING TECHNIQUES: TO CORRECTDISTORTION OF ENDOSCOPIC SYSTEMS Authors: Tiago Bonini Borchartt Marcos Cordeiro d’Ornellas Aura Conci Alicia Becerra Romero Paulo Henrique de Aguiar

2. Summary • Introduction • Endoscopy Image • Proposed Method • Morphing and Warping • Affine Transformations • OpenGL and GPU’s • Results • Future Works • Conclusion

3. Introduction • Images obtained through endoscopes have some distortion due to lens or all system used; • This distortion affects the diagnostic; • This work proposes a method to correct these distortions by applying techniques of morphing and warping.

4. Endoscopic Image Real examination endoscopic image

5. ProposedMethod • Correct the distortion using Morphing Techniques; • Use linear equations to transform parts of the image separately, dividing the image into a triangular mesh; • Create the mesh from a pattern pre-defined; • Apply the same equations in real images; • Improve the processing performance using GPU – graphic processing unit.

6. Algorithm: Over view:

7. Morphing and Warping Techniques • Morphing Techniques are used to produce special effects in animations, games and movies; • Morphing is often used to depict one person turning into another; • The Automatic Warping technique uses a triangular grid and affine transformations to modify an original triangle in another triangle in a target position.

8. Chess-pattern Fig. 2: Chess-pattern acquired by endoscope Fig. 1: Chess-pattern used to calibration

9. VerifyTheDistortion Fig. 3: Fig. 2 minus Fig. 1 pre-processed (75.3% matching)

10. Triangular Mesh Fig. 4: Mesh generated in Chess-pattern Fig. 5: Mesh generated in endoscope image

11. AffineTransformations Triangle from chess-pattern Triangle from endoscopy image The system calculates the affine transformations that changes the position of the vertices of a triangle in the vertices of another triangle using the equation: wi = M.vi + t Written in two dimensions as:

12. Affine Transformations (2) • Developing the matrix operations for each triangular vertex, we arrive to the following equations:

13. Affine Transformations (3) • Are calculated the values of M and t from each triangle pair source/target • The system applies the transformation to all pixels belonging to the respective triangle in the distorted image; • This process is repeated for all triangles of the mesh; • The same equations can be used to correct images of real endoscopy.

14. ProblemsSolved • If a triangle has its area expanded : • Problem: Provide values for pixels that that presents not corresponded in the destination; • Solution: We increase four times the image size with a bilinear filter, before applying the transform; • If a triangle has its area reduced : • Problem: The possibility of multiple pixels being aggregated in the same region; • Solution: The new color values of each pixel are defined as an arithmetic mean between the all possible values.

15. OpenGL and GPU’s • The system uses texture mapping to render the result image directly into the video card; • The coordinates of the triangular mesh of Fig. 4 is used by system to draw triangles in which the texture is defined by mapping the distorted image using the mesh of Fig. 5 as texture coordinates; • Acceleration of approximately 700x the processing of an image; • Average processing time reduced from 14737 milliseconds (14.737 seconds) to 21.2 milliseconds.

16. Results – Chess-pattern Fig. 6: Result of applying the method in Fig. 2

17. Results – VerifytheDistortion Fig. 7: Fig. 6 minus Fig. 1 pre-processed (94.6% matching)

18. Results – ComparativeImages Fig. 8: Real endoscopic image Fig. 9: Fig. 8 processed by correction method

19. Results – Comparative Images (2) Fig. 11: Fig. 10 processed by correction method Fig. 10: Real endoscopic image

20. Future Works • To improve the results using morphing techniques is necessary to use a non uniform mesh and also test the system using other patterns for calibration; • As future work we can handle the extra brightness in the image caused by light source coupled in the camera and try to optimize the algorithm to apply it directly in real time during the screening process; • Try other techniques such as analytical and processing techniques using polar coordinates.

21. Future Works (cont.) Try other images for calibration like this using polar coordinates:

22. Conclusion • The main advantage of here presented technique of morphing and warping for image correction is that these methods accept the use of several triangular regions in an image and warp each region in a different way; • This method provides greater freedom to handle the image; • Texture mapping is more adequate for real time processing.

23. References • Borchartt, T., 2009. More results and the executable software. In: http://www.ic.uff.br/~uffufsm/?p=software . • Conci, A., Azevedo, E., & Leta, F., 2008. Computac¸ ˜ao Grafica: volume 2. Campus/Elsevier. • Hartley, R. & Kang, S., 2007. Parameter-free radial distortion correction with center of distortion estimation. IEEE Transaction on Pattern Analysis and Machine Intelligence, vol. 29, n. 8, pp. 1309–1321. • Helferty, J., Zhang, C., McLennan, G., & Higgins, W., 2001. Videoendoscopic distortion correction and its application to virtual guidance of endoscopy. IEEE Transaction on Medical Imaging, vol. 20, n. 7, pp. 605–617. • Kannala, J. & Brandt, S., 2006. A generic camera model and calibration method for conventional, wide-angle and fish-eye lenses. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28, n. 8, pp. 1335–1340. • Möller, T. & Haines, E., 2002. Real-Time Redering. Editorial Sales, 2 edition. • Stehle, T., Truhn, D., Aach, T., Trautwein, C., & Tischendorf, J., 2007. Camera calibration for fish-eye lenses in endoscopy with an application to 3d reconstrution. IEEE Transaction on Biomedical Imaging, pp. 1176–1179. Thank you!