1 / 22

Alessandra TESEI, Alessandra BARBIERI NATO - CMRE, Italy Ion ROCEANU, D. BELIGAN

Status of User studies on identified Best Practices within SIG 3.4 – Security, safety & Crisis Management. Alessandra TESEI, Alessandra BARBIERI NATO - CMRE, Italy Ion ROCEANU, D. BELIGAN “CAROL I” National Defense University, Romania. Outline. Plan for User Studies – Overview

coral
Download Presentation

Alessandra TESEI, Alessandra BARBIERI NATO - CMRE, Italy Ion ROCEANU, D. BELIGAN

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Status of User studies on identified Best Practices withinSIG 3.4 – Security, safety & Crisis Management Alessandra TESEI, Alessandra BARBIERI NATO - CMRE, Italy Ion ROCEANU, D. BELIGAN “CAROL I” National Defense University, Romania Madrid Project Meeting 2013

  2. Outline Plan for User Studies – Overview Assessing the learning impact of selected games Studies: “Cultural Awareness - Afghanistan Pre-deployment” “Boarders Ahoy!” Present status & Discussion Madrid Project Meeting 2013

  3. Plan for User Studies Two Best Practice selected during Year 2 Common feature: focus on the Human factor while conducting different kinds of military/security operations: “Cultural Awareness - Afghanistan Pre-deployment” “Boarders Ahoy!” On Case 1: Cooperation with MAN for Having access to the game through the web Conducting evaluation in parallel at MAN and at CMRE on different typologies of players Comparing our own results → Joint publications on the achievements On Case 2: Submitting this game to the same CMRE User Group who has evaluated “Afghanistan” Comparing the results obtained on the two games from the same user group at CMRE Madrid Project Meeting 2013

  4. “Cultural Awareness – Afghanistan” serious game Released by MOD UK (2010) – it could be seen as a game-based training applicationfor military purpose. Aim - develops and reinforces skills in identifying relevant cultural information, generating decision criteria and selecting alternatives Objectives - to familiarize with the specificities of a mission (techniques and tactics and risks); the geography of the area; and, above all, the best approach towards local people. Target group -military personnel who will perform missions in Afghanistan theatre of operation. Usability - can be used inside a Learning Management System (LMS) through the web (or stand alone on a CD) Final skills - acquiring necessary knowledge regarding cultural awareness in Afghanistan and being able to make the best decisions in the game scenarios, using available resources Madrid Project Meeting 2013

  5. “Boarders Ahoy!” serious game Released by NATO ACT (2010) – game-based training applicationfor military purposes (maritime security and anti-terrorism). Aim - to instruct NATO security teams on how to conduct ship boarding operations (e.g., questioning crews) and search a suspect merchant vessel for illegal goods or weapons of mass destruction (WMD) Educational Objectives - Professional training as individual and as a team of NATO forces mainly at operational/tactical level on anti-piracy, anti-terrorism, contraband and maritime security procedures.. Target group -Operational/tactical levels of NATO Forces (security boarding parties) in support to Maritime Interdiction Operations. Adopted at NATO Maritime Interdiction and Operational Training Centre (NMIOTC). Usability - can be used on a PC. A lighter version is available for Android (free app) Final skills - acquiring necessary knowledge in maritime security procedures, with particular attention on how to approach the crew (human factor) Madrid Project Meeting 2013

  6. “Cultural awareness - Afghanistan”Game characteristics as a learning environment Our goal is to provide the most realistic and best training possible Madrid Project Meeting 2013

  7. Game characteristics as a learning environment • Learning is focused on acquiring cultural habits • Players are autonomous.They need to be free to make decisions. • They need to connect learning to a knowledge/experience background (provided in an introduction) • There are different scenarios classified according to various types of missions. • the scenarios present possible real situations and solutions are based on theoretical inputs provided in an introduction • Users can repeat an action as many times as they want, until knowledge is fully achieved • Users have the possibility to access available resources (text windows) while playing • Assessment is given by displaying correct results and a conflict indicator percentage • feedback is provided via virtual human presence embedded in the game (videos) • If the user fails, the culture risk indicator outlines this, and a video explains the risk Madrid Project Meeting 2013

  8. SG user groups • At MAN: • Players are Romanian students at Defense School. • Students are goal-oriented. This SG is part of their pre-deployment training course, and they usually know what goal they want to attain. • Their motivation is high as acquired knowledge is crucial for their future missions and will be applied on the field in near future • Learning has to be achieved through their own effort, to be of value to them. • At CMRE: • Most players are civilian NATO staff members. They do not expect to apply what learnt immediately to real life. However, they are deeply aware of the problem and might be involved in operations on the field (deployment clause in NATO contract) • Some are military officers with direct experience on the field • They have various degrees of familiarity with military operations from different perspectives • At average they have medium experience as video-game players, very low / no experience as serious-game users • Most of them have technical expertise in modeling&simulation or computing Madrid Project Meeting 2013

  9. Assessmentofgame learning impact at MAN • 12 questions in post-briefing questionnaire (on-line) • 88 participants (Students who are attending a course aimed at future actual deployment) • Completed • Main conclusion: most of participants consider that this game is very useful and very well made, representing a real support in terms of theoretical training for the mission. • Most players state: • the interface is very friendly and contains catching audio-visual materials; • the content is attractive, with detailed features which enable a good fixation of knowledge. Madrid Project Meeting 2013

  10. Assessmentofgame learning impact at CMRE • 15Questions in post-briefing questionnaire (on-line through SurveyMonkey) • 40 participants (NATO staff members - Scientists, Engineers, Military and Admin/Managers) • Quick pre-briefing (only instructions on how to access the game) • Still in Course (24 answers over 40) • Preliminary results: • All participants consider this game very useful and its content very interesting • Most judge its man-machine/graphical interface too static, basic and obsolete • About one half of them considers the game boring and not immersive at all, and suggests to significantly improve its graphical interface to make it more appealing • Nevertheless most participants believe that the game enables a good knowledge transfer, quicker than other more traditional means (such as books or ppt slides). • Feedback and assessment are considered appropriate and successful Madrid Project Meeting 2013

  11. Present Status Study 1 (Cultural Awareness – Afghanistan) On-line access to the game made possible by MAN (Many thanks!) User group study: at MAN: completed at CMRE: almost completed Comparison between the respective results (on common questions): TO DO Study 2 (Boarders Ahoy!) The game is already available at CMRE User Group study is planned for May-June Same CMRE user group as for Study 1 Questionnaire ready (Most questions common to the two studies) Comparison between the results obtained on the two games at CMRE: TO DO Madrid Project Meeting 2013

  12. BACK-Up Slides Madrid Project Meeting 2013

  13. CMRE - SurveyMonkey Survey. Question 8 Madrid Project Meeting 2013

  14. CMRE - SurveyMonkey Survey. Question 10 Madrid Project Meeting 2013

  15. MAN Conclusions • SGs - new and modern way for assimilating knowledge with the purpose to determine proper behavioral and attitudinal reactions. • SGs could be used as a sole method of education and training when they satisfy appropriate requirements • Complementary solution to the classical system of education and training. • Serious games are an effective way for users to safely make decisions in different scenarios, even “incorrect” ones, and thus see their possible effects • Users who completed this serious game were more familiar with the specificities and realities from Afghanistan than those who have not accessed it. Madrid Project Meeting 2013

  16. Instructions to play w/ Cultural Awareness - Afghanistan Dear Player, Please find below a set of guidelines to follow in order to play the Serious Game “Cultural Awareness – Afghanistan”. The game has been kindly made available by the Romanian National Defense University (namely, “adlunap”), who is partner of GALA European Network of Excellence on Serious Games, along with CMRE. 1)     Access the web site http://lms.adlunap.ro from any PC connected to internet 2)      Your Username is your own lastname (all small letters); your Password is “987654.” (mind the final fullstop). At the first connection you may change the password using the following path: Personal Desktop / Settings / Password. 3)    Once inside the environment, the SG Cultural Awareness - Afghanistan is located directly on your personal desktop and can be accessed by clicking on it. 4)    The game starts automatically with an Introduction about the game goals and various pieces of information on Afghanistan and its culture. Go through it and answer questions, as it is a good basis to the following actual play session.

  17. Instructions to play w/ Cultural Awareness – Afghanistan (2)  5)    At the end of the Introduction (or at any time, if you click on Menu icon) you will go to the main menu page.  There, you will find several items: a.       A top menu bar with: i.      Introduction (as in 4))  ii.      Resources:  where you can find more detailed info on Afghanistan and its  iii.      How to use the course: a set of slides explaining step by step how to play. Actually the play mechanics is so intuitive that you might not need it  iv.      Exit: to quit b.      A set of red callouts which are enlightened when you pass on them with your mouse. They refer and give access to all the scenarios of the game. Every time you click on one of them you start playing that scenario. Go through each of play sessions in this way. 6)   At the end of the game session, exit from the game and from the website, and access: https://www.surveymonkey.com/s/WQJ6F63 in order to fill the online questionnaire. Please follow instructions. 7)      Please besure the questionnaireiscompletedbyApril 30nd 2013, toallowsufficienttimefor data processing and reporting The collection of these questionnaires (which will remain anonymous) is extremely important for our statistical evaluation of the game, which is one of CMRE tasks in the context of GALA project. Thank you so much for your kind and precious cooperation! the GALA team

  18. Approach to learning sequences • Sequences • Step by step chronological • Simple to complex • Known to unknown • General overview of the detailed learning • How to meet this requirement • Task are always done in a certain order or if one task is depended of another • Topics are sequenced in terms of increasing complexity • Familiar topics are considered before unfamiliar ones • An entire system is introduced before separate tasks. Madrid Project Meeting 2013

  19. Learning phases Phase III Situational Awareness/Understanding Phase II Culturally Specific Rapport Building Skills Situational Awareness Level required at Deployed and Dismounted and Integrated Phase I Principles of Cross-Cultural Communication Skills Cultural Competency Cultural Competency Cultural Competency 1. Expertise 1. Expertise 1. Expertise 2. Understanding 2. Understanding 2. Understanding 3. Awareness 3. Awareness 3. Awareness TIME Cultural Training Road to Deployment STEP III Cultural Education STEP II PME STEP I Madrid Project Meeting 2013

  20. MAN Cumulated survey results Madrid Project Meeting 2013

  21. MAN Cumulated survey results Madrid Project Meeting 2013

  22. MAN Cumulated survey results Madrid Project Meeting 2013

More Related