- Baljeet Aulakh -Arnold Csok -Jared Shepherd - Amandeep Singh

1. -BaljeetAulakh-Arnold Csok-Jared Shepherd-Amandeep Singh EEC 490 Spring 2012 Kinect Fitness Trainer

2. Project Overview • Recognition of Body Movements and Exercises • Everything is Voice Controlled • Ability to Calculate Velocity And Acceleration • Write Statistics of Movements to an Output File • User Interface Implementation

3. Requirements • Hardware: Kinect + Laptop(2.66Hz above) • Software: Kinect SDK + Microsoft Speech Platform + C# 2010 • Environment: Quite Room(voice) + Only one thing moving at a time

4. User Interface

5. Step by Step • Voice Recognition • Detection Algorithms • Velocity and Acceleration • Ability to Write Statistics to an Output file • Documentation and Website

6. Voice Recognition • Allow User To Control Kinect By Voice • Ease of Access to the Program • As Simple as- Start && Stop • Say Which Exercise to Count

7. How Voice Recognition Work • Initialize the Audio Source from the Sensors • Initialize Speech Recognition by Speech Recognizer • Create a Speech Recognition Engine with Exercise Names • Listen to User Speech • Respond to User Speech

8. Problem With Voice Recognition • The First Release of Language Pack Doesn't Have a Reliable Confidence Model • Kinect Tries To Match Every Audio Source It Picks Up • Problem with Matching the Right Exercise Because of This

9. Possible Solutions For Voice Recognition • Test Confidence Interval for Best Accuracy • Use Fitness Trainer in a Quiet Environment • Introduce Noise Cancelation • Use of Headset/Bluetooth • Wait for a More Reliable Language Pack

10. Detection Algorithms • Resources • Recognition algorithms • List of Recognized Body Movements • An Example • Velocity and Acceleration

11. 20 Joint Skeleton System • Provided By Microsoft Research Kinect

12. Resources • 20 Joint Skeleton System • Each joint gives x, y and z values • Vector Math to find the angle • Timer functionality • Voice recognition functionality • Flag Variables in programming. • Counters to store the repetition.

13. Recognition algorithms • Simple cases: • Displacement of the Joints • Displacement in the X, Y, Z Direction • Moderate cases: • Calculating Angles Between Joints • Setting Threshold For Some Angles • Complex cases: • Set a Step by Step Routine using all of the above to detect a motion

14. Recognized Exercises • The Following Exercises Are Recognized: • Squats • Upper Left Punch • Upper Right Punch • Right Punch • Left Punch • Right Arm Curl • Left Arm Curl

15. Recognized Exercises: Continued • Left Kick • Right Kick • Bowling • Hip Abduction • Lateral Weight Shift • Hamstring Stretch • Counter For All Of These Exercises

16. An example: Bowling motion

17. Bowling final position

18. JOINT STATISTICS • Each exercise has the average acceleration and velocity of all joints calculated. • The statistics are time stamped and saved into a text file.

19. AVERAGE VELOCITY • Two points • Distance between them • Time to travel between them

20. AVERAGE VELOCITY • Initial and final position of the joint • The distance formula • A Stopwatch

21. INITIAL AND FINAL POSITION • Save reference position(point) of skeleton for use in all calculations in defaultPosition[20]. • getDisplayPosition(data.Joints[JointID.HipCenter]); • getDisplayPosition(data.Joints[JointID.Spine]); • The order is very important

22. DISTANCE FORMULA • Joints lie on a Cartesian plane • However there is a caveat; The position is measured in pixels • Pixels Are Converted Into Centimeters with the conversion factor of 72 DPI • x2 = (x2 * 2.54) / (72);

23. TEXTFILE • For every joint returned from the array generated from GetValues • foreach (JointID joint in Enum.GetValues(typeof(JointID))) • JointID Joint = (JointID)i; • When int is typecasted to JointID it returns the name of the joint at that position, NOT A STRING

24. Display / Interface • The display on the bottom of the screen shows what exercise is being done by the user • The state: Start / Stop • Exercise counter • Exercise to be detected • Exercise that was detected

25. Known Limitations • Blocked by an object • Overlapping joints • Distance from the Kinect Throws Off X, Y, Z coordinates • Two People In Front Of Kinect • Joints Move During Angle Calculation • These Limitations Make The Accuracy of the Exercise Recognition Difficult • E.g. Crossed arms...

26. Future Approaches • Fix Voice Recognition problem • Have a new accurate Skeleton System • Be able to record and replay exercises • More complex body motions • Introduce more exercises • Find solutions to the Limitations for example a solution to the overlapping joints and blocking objects.

27. Conclusion • Learned about Kinect programming • Use with windows • Use of Kinect in different areas other than gaming • Learned and getting used to C#

28. Website And Facebook Page • Link To Our Webpage: “http://www.baljeetaulakh.com” • Link To Our Facebook Page: “https://www.facebook.com/pages/Kinect-Fitness-Trainer/236918233072748”

29. Questions????? OR Suggestions?

30. End of Slides

31. Fun Time!!!!! Demonstration!!!!!