1 / 35

Development of Image Processing Based Feedback Systems for Interactive Gaming Using Non-Traditional Controllers

This project explores the development of image processing techniques for interactive gaming using non-traditional controllers, such as Project Natal for Xbox 360. It focuses on full-body motion detection, geometry detection, and color classification.

emcmillan
Download Presentation

Development of Image Processing Based Feedback Systems for Interactive Gaming Using Non-Traditional Controllers

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Development of Image Processing Based Feedback Systems for Interactive Gaming Using Non-Traditional Controllers Adam Hedji Mantas Pulinas Philip San III. Viktorious SSIP 2009, Debrecen, Hungary

  2. Full Body Motion Detection Project Natal X-Box 360 Microsoft Corporation New project that promises to use two cameras for full-body motion detection

  3. CHECKERS!

  4. Overview of Code Board Plane Geometry Detection

  5. Overview of Code Board Plane Geometry Detection Segmentation of Individual Game Squares Identification of Square Status by Colour Classification Relaying of Data with Checkers A.I. Feedback This is blue!

  6. Geometry Detection • 2 approaches were used: • Edge detection and region growing • Hough transforms Find the corners

  7. Geometry Detection – Edge Detection and Region Growing

  8. Geometry Detection – Edge Detection and Region Growing Edge detection with Sobel operator Dilate the image to fill the gaps in the border

  9. Geometry Detection – Edge Detection and Region Growing Edge detection with Sobel operator Dilate the image to fill the gaps in the border Thin the image to get the true border

  10. Geometry Detection – Edge Detection and Region Growing Edge detection with Sobel operator Dilate the image to fill the gaps in the border Thin the image to get the true border Dilate the image several times to remove useless edges

  11. Geometry Detection – Edge Detection and Region Growing Edge detection with Sobel operator Dilate the image to fill the gaps in the border Thin the image to get the true border Dilate the image several times to remove useless edges Region growing Determine the corner tiles Track corners in real-time

  12. Geometry Detection – Projective Correction Edge detection with Sobel operator Dilate the image to fill the gaps in the border Thin the image to get the true border Dilate the image several times to remove useless edges Region growing Determine the corner tiles Track corners in real-time

  13. Geometry Detection – Projective Correction Calculate homography using corner coordinates

  14. Geometry Detection – Hough Transform • Hough transform • We had time so we developed a better solution • Based on Hough transformations • Better real-time line detection

  15. Development of Image Processing Based Feedback Systems for Interactive Gaming Using Non-Traditional Controllers

  16. Colour Classification How can the computer classify the colours? (32,61,105) (171,154,158)

  17. Colour Classification Use image processing algorithms to make the RGB values only 0 or 255 (0,0,255) (255,255,255)

  18. Colour Classification (255,255,255) (255,255,0) (0,0,0) (0,0,255) • Select individual tile • Analyse the predominant colour inside to classify the square state (white, black, blue, yellow) • Sample of pixels used as opposed to whole square

  19. Colour Classification • How to determine if each R, G and B values are 0 or 255? • -Need to choose threshold value e.g. A given pixel of value (15, 19, 250) A threshold of 126 Output is (0,0,255)

  20. Colour Classification - But how to choose the threshold? • Webcams can be of relatively poor quality and provide poor contrast. • For example, blue pieces used were relatively hard to distinguish from the black tiles.

  21. Development of Image Processing Based Feedback Systems for Interactive Gaming Using Non-Traditional Controllers

  22. But how to choose the threshold? • p1=(241,209,210) • p2=(232,204,214) • p3=(240,211,205) • Can instead normalise the image and use a threshold of 127, given by (256/2)-1. • Select a white tile and take an average of the colour values. • Do the same for a black tile. • Use these averages to normalise the image

  23. Colour Classification - Normalisation

  24. Colour Classification - Normalisation

  25. Colour Classification - Normalisation

  26. Colour Classification - Normalisation Then reduce the image to absolute values of 0 and 255 Use a threshold of 126 (half of full intensity value)

  27. Colour Classification - Normalisation

  28. Colour Classification - Normalisation

  29. Summary • Main results • Successfully used 2 approaches to chess board detection • Edge detection and region growing • Hough transform • Removing perspective distortion • Identification of individual tiles and pieces, including classification • Connection to engine interface with feedback system

  30. Improvements • Aim towards real-time 60fps processing • Use a more efficient programming language such as C++ • Use of GPU using CUDA or Open CL programming language • More complex algorithms • Motion detection of hand • Use of overlay of 3D structures onto camera image. • Virtual humans...

  31. Unused Virtual Human

  32. The Team Adam Hedji University of Zagreb, Croatia adam.hedi@fer.hr Philip San University College London, England p.san@ucl.ac.uk Mantas Paulinas Vilnius Gediminas Technical University, Lithuania mantas.paulinas@el.vgut.lt Viktor Blaskovics University of Szeged, Hungary blaskovics.viktor@stud.u-szeged.hu Philip San University College London, England p.san@ucl.ac.uk Viktor Blaskovics University of Szeged, Hungary blaskovics.viktor@stud.u-szeged.hu

More Related