Wiimote kinect lab design
1 / 31

Wiimote / Kinect Lab Design - PowerPoint PPT Presentation

  • Uploaded on

Wiimote / Kinect Lab Design. Senior Design December 2011, Group 16 Adviser: Dr. Tom Daniels Brenton Hankins Rick Hanton Harsh Goel Jeff Kramer. Outline. Problem/Solution Requirements Risks, Costs, and Schedule Wii Hardware WiiWrap and Software Kinect Hardware Software

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Wiimote / Kinect Lab Design' - claude

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Wiimote kinect lab design

Wiimote/Kinect Lab Design

Senior Design December 2011, Group 16

Adviser: Dr. Tom Daniels

Brenton Hankins

Rick Hanton

Harsh Goel

Jeff Kramer


  • Problem/Solution

  • Requirements

  • Risks, Costs, and Schedule

  • Wii

    • Hardware

    • WiiWrap and Software

  • Kinect

    • Hardware

    • Software

  • Test Plan

  • Project Status

  • Work Distribution

  • Future Plans

Problem statement
Problem Statement

The Wii Wrap software used by freshmen Computer Engineering students is simple but needs improvement for installation and use by novice users such as a high school science teachers.  In addition, similar software should be developed that uses the Microsoft Kinect sensor as input to student-written programs.

Functional requirements
Functional Requirements

  • Develop an installer for the Wiimote project.

  • Identify and Remove errors from the Wiimote lab software

  • Correct buffering problems and synchronize user program with the incoming data stream.

  • Make Kinect sensor data streams available to the user in an efficient and real-time manner

  • Provide an interactive application that students can program using the Kinect data.

  • Have an interface to write programs on the received Kinect data and run them.

Non functional requirements
Non-Functional Requirements

  • Reliability: We will refine and package the software that is designed. This pertains to optimization of real-time processes such as data processing. A testing plan shall be developed for this purpose.

  • Portability: The project shall be portable between computers. We will cater mainly to windows, but will ensure that our libraries are extendable in case there is a need to adapt it to another system. Additionally, the software shall be easy to install and remove.

  • Usability: The project shall be easy to use, both for instructors and students. It shall simplify tasks to set up and run projects by means of intuitive GUIs so that students can focus on problem solving aspects as much as possible.


  • The software is designed to be used in a Windows computer lab that has both a bluetooth dongle and a Wiimote or a Kinect sensor at each computer terminal.

  • We are currently assuming that the Microsoft SDK will be released in the coming months, otherwise development will continue based on the OpenKinect software on the Windows platform

What makes us unique
What makes us unique?

  • Using video game hardware for to teach programming in an educational setting

  • Designing software for students with little programming experience to work with 3D video capture & processing functions

  • Interactive API and GUI that makes it easy for students to program without worry about OO programming or managing system/hardware issues.

Existing developed software
Existing Developed Software

  • The Wii Wrap software was previously developed by a student at Iowa State

  • Similar software for the Wii Remote exists such as Wiimote Whiteboard, WiinRemote, or GlovePie

  • OpenKinect is a package of software created by enthusiasts to run Kinect programs on Linux, Mac, and Windows using a variety of languages

  • Within a few weeks Microsoft will release a Kinect SDK specifically for Windows


Risk 1: Temporarily lose a developer from our team for unforeseen reasons.

Mitigation 1: The team will do our best to continue work without the developer while they are gone and rely on a detailed documentation trail. We will use technologies like Skype and email to keep in touch with the developer during their absence.

Risk 2: Unable to keep tasks fully on schedule due to other projects and schoolwork.

Mitigation 2: Set up minor deadlines to make progress on larger tasks.

Risk 3: Important equipment becomes lost or nonfunctional (Wiimote, Kinects)

Mitigation 3: Jeff has several personal Wiimotes available to him for use and there are one or two extra department-owned Kinects Dr. Daniels has for use.

Risk 4: No individuals on the project team have experience programming with graphics APIs.

Mitigation 4: We have built periods of time into our schedule for us to work on developing the necessary experience with OpenGL and SDL environments to allow us to design and develop software for the Kinect efficiently.

Hardware wii
Hardware - Wii

  • Wii Remote / Wiimote

  • 11 buttons + Power

  • Roll/Pitch/Yaw Gyro

  • Infrared Sensor

  • Speaker (unsupported)

  • Optional Nunchuck (unsupported)

Development software selection
Development Software Selection

Wii Wrap

  • Written in C

  • GUI written in Java

    Kinect SDK

  • Written in C++

    Wii Wrap installer

  • MSI

Wii wrap gui1
Wii Wrap GUI

  • Fix Run/Stopped Errors in GUI when the Wii Wrap software doesn’t execute as planned.

  • Discover where the lag in display output is (Java Buffers, Windows I/O)

  • Integrate GUI into Wii Wrap installer

Hardware kinect
Hardware - Kinect

  • RGB Webcam Camera 640x480 @ 30 Hz (center)

  • IR Projector Projects Grid for Depth Mapping

  • IR Camera at 640x480 @ 30 Hz collects depth data (right)

  • Microphones give room audio

  • Small Panning motor turns sensor bar from side to side

  • Onboard image preprocessing

  • Projector power draw requires extra AC power

  • Thanks to iFixit for some specs data

Kinect approaches
Kinect - Approaches

Naïve Approach

  • Develop similar text output program that is based on OpenKinect, using OpenGL, pthreads, and the Windows port of the Linux USB library functions

  • Allow students to generate a spreadsheet of data that they can work on before developing real-time software.

    Con: Slow, lots of text processing and possibly storage

Kinect approaches1
Kinect - Approaches

What we need

  • Provide a GUI environment that would use OpenGL (or an alternative) to show basic output of the Kinect (depth on video) as well as user modifications to the video data (head replacement, skeleton tracking, etc).

  • Want to make use as painless as possible (no pthreads etc to deal with)

Colored image to visualize depth

Kinect design
Kinect - Design

  • Need to evaluate how to best integrate student code

  • Concept A : Process-Update Model

    • Provide a framework of classes that students modify without needing to know how OO programming works

User Models


API for the Student


Called when new frame data is available

* -> function pointers

Called to update the screen after processing


  • Anticipated User View

  • Backend: High-Level Sketch

Acquire Moving Average of Frames

Input Screen

Output Screen

Apply User-Defined Process* Function

Apply User-Defined Update* Function

Apply Changes to Output Screen

Test plan
Test Plan

  • We will use similar test plans for both the Wii and Kinect.

  • Initially we will use black box testing methods on individual modules of the program.

    • Testing boundary input values such as null, negative values, etc.

    • Randomizing input values in case edge case is missed

  • The second phase of testing will be integration testing.

    • Testing the entire system as a whole single unit

    • Making sure all the individual modules function when combined

  • GUI testing can be done alongside integration phase of the testing plan.

    • Consists of making sure GUI is responsive, functions properly (buttons do the correct actions)

Test plan part 2
Test Plan (Part 2)

  • Once the functionality is tested, we will need to do performance testing especially on the GUI.

    • Use Eclipse TPTP to test performance.

    • Test GUI input buffering, load tests, etc.

  • Will have Dr. Daniels use the programs to see if they are satisfactory and what changes need to be made.

    • Make revisions to code based on Daniels’ feedback

Current project status
Current Project Status

  • Have been working on developing a software update for Wii Wrap

  • Finishing installer for Wii Wrap

  • Still need to write documentation for Wii Wrap and provide a distribution setup

  • Started experimenting with the OpenKinect software and communicating with OpenKinect founder about release of Microsoft SDK

  • Currently preparing a test plan for the Kinect

Work distribution fall 11 plans
Work Distribution / Fall ‘11 Plans

  • Now – May 6: Complete Wii Wrap Installer (Brenton)

  • Now – Sept. 1: Identify data stream from Kinect (Rick/Brenton)

  • Now – Sept. 1: Make the data streams accessible to user (Rick/Brenton)

  • Now – Sept. 1: Developer user engine/API, investigate graphics (Harsh/Jeff)

  • Sept. 1 – 7: Document Kinectpackage (Rick – Lead)

  • Sept. 7- 21: Testing of Kinect Software (Jeff – Lead)

  • Sept. 22-Dec: Fix Kinect Software Issues, Complete Documentation (All)