1 / 42

Ryan Carlson - Developing Number Sense with Play: Lessons in Data Analysis

Presenter: Ryan Carlson, Research Programmer, Carnegie Learning We developed an iPad racing game to increase students’ math fluency with an architecture that enables rapid prototyping of game features. The game is unique in that it is configurable in both content and behavior, allowing educational researchers, without programming, to quickly build a wide range of experiments to gauge educational gains. For example, researchers can change game parameters, in-game feedback, the reward structure or even make interface changes. Students have been very enthusiastic about the game. Initial tests show big differences in the way students approach the speed/accuracy tradeoff inherent in playing the game. Our hypothesis is that students’ approach reflects both their understanding and motivation. Robust instrumentation enables a deeper understanding of students’ learning and experiences within the game.

SeriousPlayConf
Download Presentation

Ryan Carlson - Developing Number Sense with Play: Lessons in Data Analysis

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Developing Number Sense with Play: Lessons in Data Analysis Ryan Carlson

  2. Overview • New iPad game to practice & develop number sense • In-school user study & pilot study addressing non-cognitive factors • Discussion of instrumentation & evaluation of our game

  3. Outline • Some Definitions • NumberACEr • Designing a configurable game • Instrumentation • User Study • Pilot Study • Data Analysis • Lessons Learned

  4. Who Are We? • Founded 1998 • Develop K-12 mathematics blended curricula • 600k students / year across >2,000 schools • Cognitive Tutor • Model student knowledge • Adaptively present new material

  5. What is Number Sense? “fluidity and flexibility with numbers, the sense of what numbers mean and ability to perform mental mathematics to look at the world and make comparisons” (Gersten & Chard, 1999) Foundation for further mathematics Early number sense predicts later mathematics ability (Geary et. al., 2013) • •

  6. What is Number Sense? 1 2 3 4

  7. What is Number Sense? 4 5 3 4

  8. What is Number Sense? 5 7 2 3

  9. What is Number Sense? 6 13 9 17

  10. Personalization Cognitive What you know (positive results) Non-Cognitive Everything Else (less well understood) Knowledge Tracing Self-Efficacy Grit Learning Orientation • • • •

  11. NumerACEr

  12. NumerACEr

  13. NumerACEr

  14. Configurability • Content • Game Parameters • Level Layout • Scripting System • Dynamic runtime engine • React to game events • Add text / images For non- programmers

  15. Configurability

  16. Configurability

  17. Instrumentation Boring But Important Generate usage data & plan analyses before deployment Logging infrastructure & API • Developers annotate code • Log messages sent to central storage Log everything! • Level started? • Speed changes? • iPad tilt? • Device suspend? • • •

  18. User Study December 2014 in Raleigh, West Virginia • 6thgraders (36) • 8thgraders (7) Played for ~10 min Exit feedback (verbal + written) • • •

  19. User Study

  20. User Study Neutral (High Accuracy) Cautious Neutral Aggressive (Low Accuracy)

  21. User Study Neutral (High Accuracy) Cautious Neutral Aggressive (Low Accuracy)

  22. Pilot Study May 2015 in Savannah, Georgia • 6thgraders (200) Remote Deployment 15 minute play requested Non-Cognitive survey administered prior to play • Self-efficacy • Grit • Mastery learning orientation • • • •

  23. Pilot Study: Interventions Self-Efficacy Low SE  Motivation •

  24. Pilot Study: Interventions Grit High Grit  Increased Difficulty Earning Brains •

  25. Pilot Study: Interventions Learning Orientation Low Mastery  Coach Towards Mastery •

  26. Pilot Study: Interventions Game Strategy Aggressive  Slowdown • Reduce maximum speed Cautious  Speedup • Increase minimum speed Neutral (low acc.)  Instruction • Provide in-game feedback on mistakes • • •

  27. Pilot Study: Interventions

  28. Pilot Study: Interventions

  29. Pilot Study: Game Usage

  30. Pilot Study: Game Usage

  31. Pilot Study: Game Usage

  32. Pilot Study: Game Effectiveness

  33. Pilot Study: Intervention Effectiveness Relationship b/w Self-Efficacy and how much students play the game Self-Efficacy & Grit interact in an interesting way

  34. Pilot Study: Intervention Effectiveness F(1,105)=3.81 p=0.05

  35. Pilot Study: Intervention Effectiveness χ2 test, p<.01

  36. Pilot Study: Intervention Effectiveness F(1,176)=3.46 p<0.10

  37. Pilot Study: Intervention Effectiveness F(1,176)=0.05 p>.10

  38. Pilot Study: Intervention Effectiveness F(1,176)=4.50 p<.05

  39. Pilot Study: Intervention Effectiveness F(1,176)=0.03 p>.10

  40. Lessons Learned & Limitations • Promising directions for adapting to cognitive + non-cognitive factors into games • Gamification serves a purpose • Countdown timer adds pressure • Speed forces rapid comparisons • “pretty fun for a school game” • Desire for common game features • Customization, currency • Assumption: the game is a valid measure of number sense

  41. References • Geary DC, Hoard MK, Nugent L, Bailey DH (2013) “Adolescents’ Functional Numeracy Is Predicted by Their School Entry Number System Knowledge”. PLoS ONE 8(1): e54651. Gersten, R., and D. Chard. “Number Sense: Rethinking Arithmetic Instruction for Students with Mathematical Disabilities.” The Journal of Special Education 33.1 (1999): 18-28 •

  42. Thank you! Questions? rcarlson@carnegielearning.com

More Related