“Using Simulations to Train Future Project Leaders at NASA” By Lawrence Suda- Serious Play Conference 2012

1. Using Simulations to Train Future Project Leaders at NASA Experience. Lawrence Suda

2. Performance Simulations • People learn by doing in a risk-free environment • Learners: – are encouraged to experiment – learners drive the experience – get realistic and individualized feedback throughout the experience – must master certain behaviors to achieve higher overall performance metrics

3. Business Simulations Project Leadership Experience Adaptive Project Leadership Applied Agile Project Management Global Project Leadership Portfolio & Program Leadership Business Acumen Virtual Project Leadership Change Leadership Operations Management Strategic Business Management Supply Chain Management

4. Simulations Mapped to Levels 4

5. Business Simulations Scope Large-Scale Team Exercise – 24 to 35 + hours to complete Simulation Integrated into Learning Design – Mixed Methodologies Open-ended, Multi-layered, Multi- dimensional Systems and Detailed Levels of Complexity Instructor Facilitated but Learner Driven Classroom or Virtual or Both Teams Are Not Competing

6. Validation V e t t e d o v e r 3 0 y e a r s w i t h w o r l d c l a s s o r g a n i z a t i o n s a n d l i t e r a t u r e r e v i e w – N A S A N A S A ’s P M Mo d e l s P M Mo d e l – N A S A P r o j e c t Mi r r o r : A s s e s s m e n t & D e v e l o p m e n t C e n t e r N A S A P r o j e c t Mi r r o r : A s s e s s m e n t & D e v e l o p m e n t C e n t e r – G E P r o j e c t L e a d e r R e s e a r c h & Mo d e l G E P r o j e c t L e a d e r R e s e a r c h & Mo d e l – N o r t e l N e t wo r k s P M Mo d e l N o r t e l N e t wo r k s P M Mo d e l – MI T R E MI T R E – R o y a l B a n k o f C a n a d a R o y a l B a n k o f C a n a d a – D e p a r t me n t o f E n e r g y D e p a r t me n t o f E n e r g y – Me r c k L e a d e r s h i p Mo d e l Me r c k L e a d e r s h i p Mo d e l – Ma n a g e m e n t R e s e a r c h G r o u p L e a d e r s h i p Mo d e l Ma n a g e m e n t R e s e a r c h G r o u p L e a d e r s h i p Mo d e l – N A S A T e a m S t u d y & A s s e s s me n t N A S A T e a m S t u d y & A s s e s s me n t – N A S A L e a d e r s h i p Mo d e l N A S A L e a d e r s h i p Mo d e l – U S N a v y P r o j e c t E x e c u t i o n S t u d y U S N a v y P r o j e c t E x e c u t i o n S t u d y – P MI C a r e e r D e v e l o p m e n t F r a m e w o r k P MI C a r e e r D e v e l o p m e n t F r a m e w o r k – A l c a t e l - L u c e n t A l c a t e l - L u c e n t – H e w l e t t - P a c k a r d H e w l e t t - P a c k a r d

7. Transitioning Into Project Leadership Program/Project Manager Levels Level 1 Level 2 Level 3 Level 4 High Technical Discipline Actions Program/Project Management Actions Low

8. Dimensions Program/Project Manager Levels Level 1 Level 2 Level 3 Level 4 High • Technical --- Leadership • Technical --- Abstract • Productive --- Practical • Narrow --- Broad • Task Orientation --- World View • Defined Goals --- Moving Goals • Discipline Specific --- Multidisciplinary Technical Discipline Actions Program/Project Management Actions Low

9. Major Clusters Competency Research Focus Groups & Individual Interviews  Personal Leadership  Team Leadership  NASA Agency Acumen  Communication Effectiveness  Strategic Perspective and Thinking  Project Management Skills and Knowledge

10. Team Learning Reflection Public Shared Meaning Learning Team Coordinated Action Planning Joint

11. Workshop Themes and Modules

12. Learning Objectives The overall purpose of the workshop is to improve our understanding of human and project complexities and to develop effective approaches to leading complex projects. More specifically we want to develop skills in: • Improving project team performance • Leading in complex environments • Developing adaptive leadership skills • Developing defensible, flexible, adaptive plans • Using information to make better decisions • Identifying and managing project tradeoffs • Understanding unintended consequences of decisions • Recognizing when to focus on technical versus adaptive problems •Learning from successes and failures

13. Typical Agenda Introduction Plenary Case Diagnosis Case Discussion Theory Planning Execution Debrief Application Simulation (Teams) Plenary Plenary Simulation Teams Simulation Teams Plenary Plenary

14. NASA Project Leadership Lab Day 1 Day 2 Day 3 Day 4 Day 5 Introduction Leadership Context Setting 360 Feedback Film Film Project Review Team Work & Knowledge Transfer Film Scheduling Personal Styles After Action Reviews Team Styles Simulation Feedback To Teams Team leadership Simulation Resource Planning Presentation Reflections Mind Mapping Simulation Simulation Action Planning Lunch Lunch Lunch Lunch Lunch Introduction to project simulation Definition Scheduling Risk Management Project Execution Simulation Simulation Quality & Budget Management Simulation Simulation Complete Simulation Simulation Film

15. 4-Day Learning Design

16. Short Lectures • Defensible Planning • Project Execution – Problem Solving, Decision Making • Team Leadership – Performance, Motivation, Start- up & Maintenance • Adaptive Leadership

17. Planning Phase Hear Lectures & Simulation Planning  Scheduling  Resource  Risks  Quality  Budgeting Create & Defend Plan

18. Example Tools – Scheduling & Resourcing                         Time (week number) CR       22- Mar 1-Apr 8-Apr 12 13 CR TR           CR       CR             CR         CR                                                     15- Jan 3 1- 8- 15- Mar 11 15- Apr 22-Apr 15 1- 8- 15- May 22-May 1-Jun 8-Jun 15-Jun 22-Jun 1-Jul 19 20 21 22         1-Jan 8-Jan 1 22-Jan 1-Feb 8-Feb 15-Feb 22-Feb 4 5 6 8-Jul 26 Mar 9 Mar 10 May 17 May 18 In the workshop participants develop plans using both computer and manual tools Staff Quality H W PR CC   X 2 7 8 14 16 23 24 25 #  ppl SD 2 T W IM   TE C                                                 Task ID       2 2 2                                                                                                                                                           1 X   X                                                                                                                                 2 2 2 3 2     X     X                   2 1 2 1 2 1 2 X 1 2 X 1 6 2         X X         X 1 3 X X 3 X X 3   X X X X 7 1 X   X     X                                                      X X X 8 2           X                                   9 3 X   X     X                                                         2 2 2 2 2 2 2 16 2   X X                                                                           X 3 X 3 X   3   19 2         X X X                                               20 3   X X X   X X                                                                                                      2 2 21   2                   X X X                                               1 1 1 X 1 2 X 1 2 X   X X 2 2 X 2 2 1   X   X                                                                     X X 2 2 X 2 X X 2 2 X 2 4 1       X               5 2   X                           2 2 X X 2 2 X X X X X X 2   X X X X 2   X X X X X 2 10 2/V   X   X X             11 2/V   X   X X             2 2 12 2/V   X     X                             X X X X X X 13 2       X     X                                   14 2   X                     X 15   2       X       X           X                                 4 2 4 2 4 2 17 4     X       X                               X X               18 2         X               X                                                               X X                               Total  staff                               2 2 2 3 3 3 8 8 9 9 9 8 7 5 10 10 10 4 4 3 3 3 2 2     H W PR CC 7 7 9 6 5 6 9 T W IM TE C Note     Senna   Mayes   Franz   Banks   Dance     SD                                                     1 1 1 3 3 3                                         4 4 8 0 8 6 6   1 1 1 3 3 3                                         2 3 7 4 8 0 4   CC CC HW 2 2 2                                         3 7 4 3 4 1 4                                                       7 4 8 3 7 7 8                                                           Bilko 6 5 7 3 6 0 6                                                       Inc Bovis   Brown   Samin   Unter   Ltd Burns   Candy   Mindy                                                                                                                                                                                                                                                                                                                                                                                                                             6 5 6 0 7 0 6                                                       9 6 9 5 10 8 9                                                       10 9 3 9 3 0 7                                                       18

19. Planning 4-person teams (instructor shown) develop an overall strategy and approach. They develop a detailed plan that includes: project priorities, critical path schedule, workload leveling, human resources plan, training, risk management, quality, budgeting. They must defend their strategy and plans. 19

20. Project Execution Phase Hear Lectures & Simulation Control  Problem Solving,  Decision Making  Decision Trade-offs  Re-planning  Control Cycle  Control Tools

21. Implementation Teams make tactical decisions on three levels as they implement their plan: Project, Task and Staff. During the exercise, they must actively manage the project as conditions change as a result of their decisions and unknown events that create more issues and problems. Note: Instructor can introduce more issues and problems. Participants report status on a monthly basis. They have control tools to assist them as they proceed through the simulation exercise. 21

22. Some Typical Disruptions – Resignations & Absenteeism – Resources pull for other projects – Large Scope Change – Scope Creep – Customer wants more meetings – Sponsor wants project sooner – New technology doesn’t work – Team performance is low – Key staff member resigns – Budget cuts – Estimates are wrong – Poor Morale

23. Project Competition

24. Performance Summary KPI PLANNED ACTUAL Final Cost 693,000 875,000 Completion Week / Date June 17 June 20 Open Quality Issues 0 0 – 2 Quality Issues Reported 47 30 – 40 Quality Issues Fixed 47 30 – 40 Labor Cost / Hour 64 74 Technology Cost / Hour 60 75 Overtime Costs 6,315 20,486 Training Costs 10,000 4,000 Utilization % 83 87 Performance % 100 83 Absence Rate % 2 1 Communication % 6 4 Number of Resignations 1 1 Staff Pulled Away Weeks 6-8 5

25. Simulation Debrief • What were your results? (brief overall description) • Did you keep your risk mitigation plan updated? • What contributed to your successes? • What were your major simulation / project challenges? • How did you overcome any challenges? • Describe your team dynamics. • Did you capitalize on team diversity? • Did you notice movement through the team development  process? 25

26. Reported Learning • • • • • • • • Lead in ambiguous, complex environments Improve individual performance Develop team effectiveness and improve team performance Develop defensible, flexible plans Align project work with organization’s strategy Think strategically and conceptually Translate strategies into concrete actions Plan systematically and establishes objectives and priorities Manage risk -- knows when too much risk is being taken, when the right amount is being taken, know the consequence and communicate to the team and to leadership Learn, develop and model new team behaviors Develop personal leadership skills Demonstrate sensitivity and encourage innovation and creativity • • • •

27. NASA Comparison of PM62, PM61, PM60, PM59 Participants’ Mean Ratings 4 5 3.8 Mean Rating Mean Rating 4.8 3.6 4.6 3.4 4.4 3.2 4.2 3 4 2.8 3.8 2.6 3.6 2.4 3.4 2.2 3.2 2 3 6 PM62 7 8 9 10 1 2 3 4 5 PM61 PM60 PM59 PM62 PM61 PM60 PM59 Questions about Instructor and Session 1. Instructor Effectiveness 2. Discussion were effective in reinforcing learning (not asked in PM59) 3. Activities were effective in reinforcing learning (not asked in PM59) 4. Instructor stimulated thinking 5. Overall rating of learning experience Questions about Information Presented in Session 6. Relevant to job 7. Likeliness to apply back on the job 8. Usefulness in increasing knowledge/skill to be an effective Project Manager 9. Value to assist in the achievement of goals as a Project Manager 10. Usefulness back on the job

28. Accreditations • PMI Global Registered Provider Advanced PM = 21 to 35 PDU’s • American Council on Education (ACE) Accredited for Graduate Credits – Stevens Institute of Technology – Drexel University – Duke CE – e-Cornell – EADA (Barcelona Spain)

29. Application • • • • • • • High Potential Leadership Training New Employee Orientation Introduction of New PM Initiatives Kick-off or Wrap-up of Conferences Create Project-based Organization Culture Change PM Certification Capstone

30. Current & Future Challenges  Manager Involvement - Getting Participants’ Managers to be part of the learning process.  Training Time Compression - Pressures to reduce training time and time away from work.  Work Pressures on Participants – Work while at training  Travel Funds – Budgets are reduced.  Budget Pressures – Develop funds for new work reduced.  Training Outsourced to One or Few Vendors.  Measuring Effectiveness – ROI.  Technology Challenges & Learning Design Challenges – Particularly with Large Scale Sims in Virtual Environments  Changing Organization, Program, Project Dynamics – e.g. Agile PM

31. Thank You! Lawrence Suda Palatine Group, Inc. New York City, NY Lsuda@thepalatinegroup.com

32. Useful Links: www.seriousplayconference.com www.seriousgamesdirectory.com www.seriousgamesassociation.com Contact: sbohle@seriousgamesassociation.com