To Design an Interactive Learning System for Child by Integrating Blocks with Kinect

1. To Design an Interactive Learning System for Child by Integrating Blocks with Kinect Tamkang University Taiwan Presenter :FENG-CHIH HSU

2. Outline • Introduction • The Interactive Block-Building System • Experimental Results • e-Block System • Conclusions

3. Outline • Introduction • The Interactive Block-Building System • Experimental Results • e-Block System • Conclusions

4. Introduction • Building blocks • Develop children’s independent thinking • Use children’s hands and fingers to grasp objects • Cultivate children’s hand-eye coordination

5. Introduction • Life of George toys https://www.youtube.com/watch?v=1DHZwSOVKBY

6. Introduction • Life of George toys • These blocks and smart phone windows is smaller for children • The Identification can be used to 2D Identification only • The Identification is affected easily by light and shadow

7. Introduction • Interactive block-building system integrate the • We use bigger Lego blocks , PC windows, children using easy • Depth information of kinect can be used to 3D Identification • Depth information is not affected by light and shadow

8. Outline • Introduction • The Interactive Block-Building System • Experimental Results • e-Block System • Conclusions

9. The Interactive Block-Building System • Step 1. Extracting Depth Information • Our system employs the Kinect to extract the depth information • PC convert the depth information to point cloud dataset with 3D format

10. The Interactive Block-Building System • Step 2. Segmenting the Test Object • We apply algorithm of Random Sample Consensus (RANSAC). • The algorithm remove the point could data of the desk. • Only the point could dataset of assembled object is retained for recognition.

11. The Interactive Block-Building System • Step3.Feature Extraction and Object Recognition • We use the density projection algorithm, the point cloud dataset is projected to the XY, XZ, YZ planes, respectively.

12. The Interactive Block-Building System • Step3.Feature Extraction and Object Recognition • Each plane is then uniformly divided into 16×12 image blocks. • Each image block is counted as the feature to represent the object whose ranges from 0 to 255.

13. Outline • Introduction • The Interactive Block-Building System • Experimental Results • e-Block System • Conclusions

14. Experimental Results • Ten types of assembled objects are used in the system • Total of 460 and 230 images acquired by the Kinect are used as training and testing patterns • The recognition accuracy is 95%

15. Outline • Introduction • The Interactive Block-Building System • Experimental Results • e-Block System • Conclusions

16. e-Block System • http://youtu.be/kgCPUjf4YqQ

17. Outline • Introduction • The Interactive Block-Building System • Experimental Results • e-Block System • Conclusions

18. Conclusions • The system help • Boosts the motivation of children to assemble blocks. • Train children’s differentiate shapes. • Develop children’s independent thinking.

24. Thanks for your attention