1. ECE477 Project MinotaurSoftware Narrative Jon Roose

2. Project Overview • Home Security Robot • Interaction through a Webserver • Robot driven manually through house and spots intruders • Relays video to web interface using kinnect

3. Hardware Hierarchy

4. Software Hierarchy

5. Benefits of Model • Reliance upon open source projects • Ease of access, available to HTML5 browsers • Emphasis on x86 architecture • Modularity • Scalability

6. Software Hierarchy - Web Interface

7. Software Hierarchy - Web Interface • Apache Web Server • Serves interface to user’s web browser • HTML5 and JavaScript WebSockets • Opens efficient communication with C&C Server • Real time commands, removes HTTP reliance • RDP Video Stream • Implemented and configured through FFServer • Connects via an embedded video object

8. Software Hierarchy - C&C Server

9. Software Hierarchy - C&C Gateway • C&C Server (Written in Python) • Mediator for command and control signals • Communicates on internal LAN with robot • Simple websocket interface with User’s browser • Leverages TCP/IP LAN interface to communicate with robot via WiFi • FFServer (Open source application) • Accepts video from Atom board and serves to user, manages RDP video stream

10. Software Hierarchy - Atom Board

11. Software Hierarchy – Atom Board • Intelligent Drive System (C++) • Navigates room and avoids obstacles • Accepts commands on a TCP/IP socket from C&C • Interprets sensor data from Kinect and PIC24 • Converts video into YUV encoding for FFMpeg • FFMpeg (Open source application) • Accepts data via stdin from the Intelligent Drive system and packages it in an MPEG video format

12. IDS Software Architecture • Thread 1 - 30 FPS event loop • Transcodes and Outputs video to FFMpeg • Polls Kinect Video and Depth Sensor • Polls network socket for C&C Commands • Thread 2 - Queue driven event loop • Implements room mapping and human detection • Timers add standard events to queue • Flexible queue design allows adjusting task frequency • Communicates with PIC24 over RS232

13. Software Hierarchy - Microcontroller

14. Software Hierarchy - MicroController • Low Level Peripheral Controller (C) • Implements fail-safe obstacle avoidance • Accepts commands via 5 byte protocol over RS232 • Controls motors and polls IR sensors • Flag driven event loop • Libfreenect & OpenNI (Open source) • Library for communicating with Kinect • Retrieves depth sensor and video data • Recognizes human form

15. Debugging • Hardware/MicroController • Standard Reset pushbutton and debug LED • RJ11 allows for in-circuit debugging via ICD3 • Headers for port pins broken out • Printing strings over SCI • Atom Board • SSH directly into board • Sending metrics and debugging data to C&C • Sending bitmaps of room map