Applejack

1. Applejack A new take on educational video games

2. The Client • Proxemic Technologies • A startup company looking to use interactive technologies to address problems in higher education • Lead by Dr. Brian Moriarty and Dr. Elizabeth Lennon

3. The Problem • The United States currently ranks 5th in overall global competitiveness • 48% of students entering undergraduate STEM (Science, Technology, Engineering, and Math) programs fail to earn their degree • Proxemic believes that a way to improve global competitiveness is to reduce this high exodus rate of students from STEM programs

4. The Solution • Increase student engagement in the subject matter to increase retention of information • Video games provided a familiar and well liked medium • Motion controls through Microsoft’s Kinect provide rich interaction and an increased level of engagement

5. The Game • A dungeon crawler adventure game that uses introductory calculus concepts as a vehicle for combat and puzzle solving • Important for game to feel like a traditional video game, not an educational one • The use of calculus as a gameplay mechanic is justified through the game’s story • The player takes on the role of Isaac Newton in an alternate timeline where his work creating calculus is unknown to the public • The “power” of calculus must be used to stop those who wish to silence Newton’s work • This gives the player a much more engaging experience, because they are not simply solving problems on a computer screen, they are working towards a narrative goal

6. Learning Through Gaming • The game focuses on basic integration concepts, including: • Integration by substitution • Integration by parts • Order of integration • Each level of the game reinforces a specific concept, presenting the player with progressively harder problems as they move from beginning to end • The end of each level includes a progression barrier in the form of a challenging problem • The problem must be solved to progress to the next area, ensuring mastery of the current concept

7. Learning Through Gaming • Exploit the reward structures of video games to incentivize learning and promote exploration in the game world • At the end of the level, the player is given a “report card” with a detailed breakdown of their performance and a ranking within their class • Achieving certain ranks or collecting enough in game currency can be used by the class instructor to exchange for rewards including extra credit on homework and exams • Locked treasure chests and secret areas of each level provide players with valuable rewards but are blocked off by difficult problems • This incentivizes players to replay sections they have completed and further master the concepts

8. Technologies • Unity Game Engine • C# • Microsoft Kinect SDK • ZigFu Development Kit • PrimeSenseNiTE Natural Interaction Middleware

9. Motion Tracking • Joints on the body are tracked by ZigFu middleware • ZigFu produces joint objects for the developer to manipulate • Gestures are created using the location of these joints

10. Motion Tracking • The game is currently tracking the users’ hands and shoulders • The hands will be used like a mouse cursor when below the shoulder • The user can select objects in the scene, and UI

11. Motion Tracking • The hands will be registered as gestures when above the shoulders • Left hand above left shoulder to turn left • Right hand above right shoulder to turn right • Both hands above and between both shoulders to move forward

12. Under the Hood • The framework we created for the game includes a number of classes: • Actor – An entity in the scene that has no logic • AIController– Logic for NPC’s and the player (The player is an AI) • PlayerController– Logic for the player’s Actor, but in our case it contains an AIController. We give commands to our main character • EventBus – A global signaling machine to let everyone know that an event has occurred. i.e., an Actor has died, or an Actor has been attacked (Singleton) • Factory – The Factory is a singleton that contains blue prints for creating actors, items, or anything that may eventually be added to the game • IProblem – A generic problem that will be used in order to solve a puzzle, an Integral in our case • Inventory – A container for items that a player or NPC has

13. Under the Hood • The framework we created for the game includes a number of classes: • Item – An object that the Actor can contain • Inventory – A container for Items that a player or NPC has • StateMachine – An object used to transition between States • States – An action that an Actor can do • UIManager – Manages the UI, and when to display it on the screen • IDrawable – A piece of UI to be drawn • LevelManager– Manages the Level, and how to handle specific events • InputManager – Handles all types of input, in our case input from the Kinect

15. Integral Class • Problem Algorithm: • Take the length of the problem to be generated as the input • For each segment, generate a random number between 1 and 20 • Power of x is determined by position in the problem – 1 • Generate lower bound with random number between 1 and 5 • Generate upper bound with random number between lower bound + 1, and 10

16. Integral Class • Answer Algorithm • Four answers are generated for each problem • One is generated using correct definite integral formula • One is generated without division • One is generated without raising bounds to the new power • One is generated by taking the derivative instead

17. Gameplay • Two categories, combat and puzzles, both of which require solving integrals • Combat: • When the player engages an enemy, they will be able to select a spell from a “spell bar” that runs across the bottom of the screen • The player will then be presented with an integral to solve, and answering correctly will result in the selected spell being cast • Answer incorrectly, and the enemy will attack you

18. Gameplay • Two categories, combat and puzzles, both of which require solving integrals • Combat: • There are three types of spells: fire, water, and electrical • Each enemy type has a specific weakness • Casting a fire spell on a water based enemy results in less damage dealt to the enemy, and requires more correct answers before an enemy is defeated • Casting spells that match an enemy’s weakness presents easier problems to solve, and deals more damage, requiring less correct answers before defeating the enemy • Players thus become more powerful as they master the material

19. Gameplay • Two categories, combat and puzzles, both of which require solving integrals • Puzzles: • The problems that are required to solve puzzles are much more challenging than those during combat • This keeps bonus areas of levels accessible to only more skilled players, and ensures mastery of the topic as players are required to solve a puzzle to complete a level • Since puzzles take place out of combat, there is no risk in breaking up the flow of battle with problems that take a long time to complete

20. Interface

22. Demo

23. Future Development • Future student teams will continue developing the game going forward • The current state of the project is a completed framework for future teams to build on and a basic version of our ultimate vision of the game • New levels can easily be added without having to worry about the backend, and different types of problems, or even different subjects, can be swapped in and out

24. Questions?