1 / 46

E19.2250 Research on Simulations and Games for Education

E19.2250 Research on Simulations and Games for Education. Jan L. Plass, DMDL. Overview. Playtesting Goals Design Methods Setup. Game Genres. Goals for Games for Learning Literacy Focus on social learning, identity Exploration and knowledge creation

xylia
Download Presentation

E19.2250 Research on Simulations and Games for Education

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. E19.2250 Research on Simulations and Games for Education • Jan L. Plass, DMDL

  2. Overview • Playtesting • Goals • Design • Methods • Setup

  3. Game Genres • Goals for Games for Learning • Literacy • Focus on social learning, identity • Exploration and knowledge creation • Legitimate Participation in community of practice • Apprenticeship (reciprocal) • Learning through design & creating new artifacts

  4. Game Research • Playtesting–Research Goals • Guide Design of New Games • Playtest, Usability of Game Prototypes • Efficacy, Evaluation of Completed Games • Advancement of Theory

  5. Games Research • Playtesting • Obtain user opinions about a game • Questions concern levels of difficulty, fun, engagement • Follows in-house QA beta-testing • Typically involves 100...103 participants

  6. Games Research • Usability Research • Usability Research: evaluate perceived and actual effectiveness of the design • Provide guidance for specific design decisions using a variety of methods involving the target audience • Typically involves 101...102 participants

  7. Games Research • Efficacy Research • Evaluate the ability of the game to have the intended outcomes–cognitive, affective, or skills outcomes • Typically involves >103 participants

  8. Class Activity • What are the function and process of an Expert evaluation/ Heuristic review? • What research questions are addressed? • What data is collected? • What happens with the results?

  9. Playtest • Research Design • Focussed Research Questions • Sampling of Participants • Player Group Sizes and Composition • Location of Testing (Lab, field) • Standardization • Reference Data • Heuristics (experienced-based methods)

  10. Group Activity • Play the Angles Game. Devise 3 research questions for playtesting of this game(15 min., Groups of 3–4) • What research questions are most important? • What function would the answer serve? • How could the question be addressed?

  11. Playtest • Research Methods • Questionnaires • Interviews • Think-Aloud Protocols • Video Observations • User Logs (Events, User actions)

  12. Playtest • Questionnaires • Learner/Player Profile • Learner Variables • Learner Responses • Consider • Types of questions • Question content • Question wording • Response format • Question placement and sequencing

  13. Playtest • Questionnaires • Intrinsic/Extrinsic Motivation Scale

  14. Playtest • Questionnaires • Self Esteem Scale

  15. Playtest • Child Questionnaires

  16. Playtest • Parent Questionnaires

  17. Playtest • Interviews • Structured, Semi-Structured, or Open-Ended • Semi-Structured • How much did you like this game? • Would you play this game again in the future? • What did you like most about this game? • What did you like the least? • Did you make enough progress?

  18. Playtest • Think-Aloud Protocols • Learner utterances during play • (more on that next week)

  19. Playtest • Video Observations • Video Recording of Player Behavior and Game Play • Allows for later analysis of a variety of issues

  20. Playtest • User Logs • Instrumented simulation or game writes game events and user actions to a log file or database • Need to specify what should be recorded • Highest Level: Every detail, allowing play back of game • Lowest Level: Only key events are recorded

  21. Playtest • User Logs • id--Uniquely identifies each row of the table • game-A string identifiying the name of the game (with it's version number) • treatment-"Conceptual" or "Arithmetic" • userid-the user's id.  This is not required to be unique so it's up to us to make sure we don't reuse these, timestamps can help disambiguate • username-The user's name that they typed in themselves • timestamp-The date/time the rows was inserted into the table, rounded to the nearest whole second • location--The chapter&level or screenname where the event took place • who-The name of the entity who issued the event (for example, "ANGLE EDF", or "RIGHT ANGLE BUTTON") • what-The content of the message (for example, "SELECTED", or "Correct") • extra- Additional values pertinent to the message, but is often unused

  22. Playtest • User Logs • CyberBuch Böll Survey Monday, November 18, 1996 • 12:15:38 PM Start using stack • 12:15:38 PM Environment: Annotations: TXTE PIC MOV Comprehension: Off Vocabulary: Off Forced lookup: On • 12:15:38 PM User Name Clarke XXXX • 12:15:40 PM Open Card 1 • 12:18:44 PM ärmlich TextEngl • 12:18:54 PM ärmlich Picture • 12:19:04 PM döst Picture • 12:19:16 PM döst TextEngl • 12:19:49 PM See Picture • 12:19:56 PM See TextEngl • 12:20:11 PM Wellenkämmen TextEngl • 12:20:20 PM Wellenkämmen Movie

  23. Playtest • User Logs

  24. Playtest • User Logs

  25. Playtest • User Logs

  26. Playtest • Research Setup • Lab or Field setting • Equipment to use for study • Role of Experimenter • Internet access

  27. Playtest • Technical Setup • Data Capture from XBox, PS3

  28. Playtest • Technical Setup • Data Capture from PC

  29. Playtest • Playtest Research Plan • Research Questions • Research Design (player groups & composition, location) • Participant profile criteria • Sampling Plan • Method (Survey Instruments, Procedure, Technical Setup) • Analysis of Data

  30. Group Activity • Develop your own research plan for playtesting of the Angles Game! (15 min., Groups of 3–4) • Research Questions • Research Design (player groups & composition, location) • Participant profile criteria • Sampling Plan • Method (Survey Instruments, Procedure, Technical Setup) • Analysis of Data

  31. Statistics • Descriptive Statistics • Frequency Distribution • Measures of Central Tendencies • Variability • Correlations • Mean Comparison • Statistical Power and Effect Size

  32. Statistics • Frequency Distribution

  33. Statistics • Measures of Central Tendency • Mean • Mode • Median

  34. Statistics • Variability • Range, Variance, Standard Deviation

  35. Statistics • Variability • Range, Variance, Standard Deviation

  36. Statistics • Means Comparison • T-test

  37. Statistics • Statistical Power and Effect Size

  38. Out of Class Activity • Play a game! • Play a computer/console game at home • Study a game! • What research questions do you find interesting exploring in the game you played?

  39. Extra Slides • Backup Slides to follow

  40. Cognitive Design Factors • Evidence-centered Design • Define educational goals of game • Define outcomes (e.g., desired knowledge of learners) • Define and identify acceptable evidence for learning • Design activities to learn and provide evidence of learning

  41. Game Research • Research Frameworks

  42. Research Framework

  43. Research Framework • Alternative Knowledge Claims • Postpositivist knowledge • Socially constructed knowledge • Advocacy/Participatory Knowledge • Pragmatic Knowledge

  44. Research Framework • Strategies of Inquiry • Qualitative • Ethnographies • Grounded Theory • Case Studies • Phenomenological Research • Narrative Research • Quantitative • Experiments • Quasi-Experiments • Correlational • Mixed Methods

  45. Research Framework • Methods • Surveys & Questionnaires • Field observations • Open-ended interviews • Knowledge Tests • Psychometric Scales • Log files • Biometrics • EEG, EKG, EMG, NILS, GSR

  46. Generate Learning Ideas Generate Gaming Ideas Analyze Requirements Formalize Ideas Evaluate Results Test Ideas Overview • Learning Game Design Model (Plass, 2010) Brainstorming No problems Physical Prototype Digital Prototype Problems with Design Design Documentation Refine and Polish Quality Assurance Educational Effectiveness Production

More Related