1 / 60

Makahiki and SGSEAM: A Serious Game Framework for Sustainability and Assessment Method

Makahiki and SGSEAM: A Serious Game Framework for Sustainability and Assessment Method. Yongwen Xu Collaborative Software Development Laboratory Department of Information and Computer Science University of Hawai`i at Mānoa. Introduction. Sustainability

ciro
Download Presentation

Makahiki and SGSEAM: A Serious Game Framework for Sustainability and Assessment Method

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Makahiki and SGSEAM: A Serious Game Framework for Sustainability and Assessment Method Yongwen Xu Collaborative Software Development Laboratory Department of Information and Computer Science University of Hawai`i at Mānoa

  2. Introduction • Sustainability • is “development that meets the needs of the present without compromising the ability of future generations to meet their own needs”. • Brundtland Report, 1987

  3. Sustainability Behavior “We are not always rational decision makers” Psychology of Sustainable Behavior, Manning, 2009 • Energy is “invisible” • Water is “everywhere”

  4. Sustainability Education • “Education as an essential tool for achieving sustainability” • Agenda 21, Earth Summit 1992, U.N. • “Surprising lack of sustainability curriculum initiatives in institutions of higher education” • Sendall 2010

  5. Collegiate Sustainability Competitions 160+ Chelsea Hodge, 2010 Campus Sustainability Survey

  6. Information Technology for Sustainability Education?

  7. Harvard Green Cup A. Use a "minimal tech" solution such as a web page and manual posting of data and results

  8. Oberlin Campus Resource Monitoring System B. Out-source to a commercial provider Lucid Design Group

  9. The Kukui Cup 2011University of Hawai`i at Manoa C. Build your own custom in-house solution 3 weeks competition 2 years software development

  10. D. Use an Open Source Serious Game Framework

  11. Research Goals Design a serious game framework for sustainability education Design a method for assessing serious game frameworks

  12. Serious Game “a mental contest, played with a computer in accordance with specific rules that uses entertainment to further government or corporate training, education, health, …..” Zyda, Michael 2005

  13. Serious Games in Learning and Behavior Accomplishment Motivation Social Engaging Enforcement

  14. Related work Power House Reeves, James, 2012

  15. Related work Vermontivate Vermont town energy competition, 2012

  16. Game Framework “a framework comprised of a collection of different tools, utilities, and interfaces that hide the low-level detailsof the various tasks that make up a video game.” - Sherrod 2007

  17. Makahiki: • An Open Source Serious Game Framework for Sustainability

  18. System Design Architecture Responsive Web UI Game and Mechanics Library A P I Water game Energy game Smartgrid game Raffle game Topscore game Manual Data Badge Referral Participation Quest Wattdepot Data Facebook Integration Game Analytics Score Manager Resource Manager Green Button Data Authentication CAS LDAP Django MemCache Database

  19. Whatmakes Makahiki a Framework? • Configurable game elements • Configurable resources • Extending new content and games • Built-in analytics for game evaluation • Mobile support • Flexible local or cloud deployment

  20. 1. Configurable Game Elements

  21. 2. Configurable Resource

  22. 3. Extending new content and games

  23. 4. Built-in Analytics for Game Evaluation

  24. 5. Mobile support Laptop view Mobile view Responsive Web Design

  25. 6. Flexible local or cloud deployment

  26. Experience with Makahiki Framework 2012 2013 Upcoming 2014

  27. Instances of the Framework

  28. Assessing Serious Game Framework How do we know if a serious game framework is correctly designed ?

  29. Related work Serious Game Assessment

  30. Related work Assessment Tools • GEQ (Game Engagement Questionnaire) • QUIS (Questionnaire for User Interaction Satisfaction)

  31. SGSEAM: Serious Game Stakeholder Experience Assessment Method

  32. SGSEAM Overview

  33. Assessment Vs Evaluation • Evaluation: • is to “determine the quality of a program by formulating a judgment” • Hurteau et al., Evaluation 2009 • SGSEAM is an assessment method • not to judge • not compare with others

  34. SGSEAM Goals • Identify the strengths and weaknesses of the framework by assessing the stakeholders’ experiences • for the purpose of self improvement

  35. SGSEAM Serious Game Stakeholders • Players participate in the game • System admins install and maintain the game infrastructure • Game designers design the content and game mechanics • Game managers manage the game during the game period • Game developers extend, enhance the game

  36. Serious Game Stakeholders NOT in SGSEAM • Researchers • Spectators • Funding organizations

  37. SGSEAM Assessment Approaches

  38. Multiple Assessment Approaches • Different levels of rigor • in-lab experiment has more data points • post-hoc interview has data in production • Additive, the more the better

  39. SGSEAM Evaluation Apply SGSEAM to Makahiki Apply SGSEAM to Lucid BuildingOS

  40. Makahiki Assessment Status

  41. Player Pre-post Study Assessment Approach Energy literacy survey • before and after the game Energy usage comparison: • before and after the game

  42. Pre-post Study Result 2011 UH Kukui Cup [2] Energy literacy • 18% (p=0.056) improvement (24 players) Energy usage: • ~2% reduction (20 teams) Assessment Outcome: • effectiveness in literacy improvement • not enough evidence in behavior change [2] Robert Brewer PhD dissertation

  43. Player Engagement Metrics Assessment Approach Analyze game log to calculate: • Participation rate • Daily players • Daily play time • Submission rate • Social interaction rate • Game error rate

  44. Engagement Metrics Assessment Result 2012 UH Kukui Cup (n=1038) Evidence of successful player engagement

  45. In-lab experiments • ICS691 Spring 2013 • Serious game development course • 8 students (CS major) • Assessment: • System admin • Game designer • Game developer

  46. Experiment Design • Student assignments: • Install the framework • Design games • Develop enhancements • Use google form to record: • Time spent • Problem encountered • Write blog posts to describe: • Problems in depth • Experiences

  47. System admin In-lab installation experiment Result Avgtime (minutes) for installation steps -- 2013 Spring ICS691 (n=8) • Weakness: • database configuration, no enough automation

  48. Game designer In-lab design experiment Result Avgtime (minutes) for design tasks -- 2013 Spring ICS691 (n=8) • Weakness: • Smartgrid game predicate design, bugs in design interface

  49. Game developer In-lab development experiment Result completed “Hello world” widget • 8 students completed all 5 enhancements • 1 students Main problem reported • lack of documentation 2013 Spring ICS691 (n=8)

  50. Makahiki SGSEAM Assessment Preliminary Outcome SGSEAM Assessment Report

More Related