Intelligent Agents for A Hockey Game

1. Intelligent Agents for A Hockey Game Presented by : Wen

2. 1 Outline • Outline • Introduction • 1.1 Proposal • 1.2 Statement of The Problem • 1.3 Solution of The Problem • Background Knowledge • 2.1 Intelligent Agents (IA) • 2.2 Decision Rules • 2.3 Minimax Algorithm • AI in Hockey Game • 3.1 Preprocess of the Playground • 3.2 Simple Reflex Agent Based on Decision Rule Technique • 3.3 Unility-based Agent Based on Minimax Algorithm • 4. Future Work • 5. Demo Time

3. 2 Introduction Introduction 1. Proposal Game AI is one popular area. However, most of classic game AI solves the game that can be described in a compact form using symbolic representations [1]. For the athletics games, most of them are based on a large number of rules, which come from the human experience. In this project, I tried to implement a classical AI algorithm into an athletic game, hockey to see how it works.

4. 3 Introduction 2. Statement of The Problem Goal: obtain a goal Factor: player’s move, passing puck and shooting. 3. Solution of The Problem Shooting: simple reflex agent Move : utility-based agent Pass puck: utility-based agent

5. 4 Background Knowledge 2. Background Knowledge 2.1 Intelligent Agent (IA) [2] Four types of IA: simple reflex agents, reflex agents with state, goal-based agents, utility-based agent. Four elements: percepts, actions, goals, environment Environment types: accessible or inaccessible, deterministic or nondeterministic, episodic or non episodic, static or dynamic, discrete or continuous.

6. 5 Background Knowledge 2.2 Decision Rules [2,4] Format: IF Condition(s) THEN Action. Advantages: intuition and efficiency

7. 6 Background Knowledge 2.3 Minimax Algorithm Depth = 4 Example is from [2].

8. 6 Background Knowledge 2.3 Minimax Algorithm Depth = 4 6 Example is from [2].

9. 6 Background Knowledge 2.3 Minimax Algorithm Depth = 4 6 Example is from [2].

10. 6 Background Knowledge 2.3 Minimax Algorithm 3 3 0 2 Depth = 4 6 Example is from [2].

11. 7 AI in Hockey 3. AI in Hockey Game Intelligent agent Percept: player's position, puck's position, attacking team, defending team. Actions: shooting with 6 possible directions, passing puck with 8 possible directions, moving with 9 possible directions (including stop). Goals: obtain a goal. Environment: playground border, puck moving rules, goal's position.

12. 7 AI in Hockey 3. AI in Hockey Game 3.1 Preprocess of The Playground Environment: accessible, deterministic, non episodic, static and continuous. accessible, deterministic, non episodic, static and discrete. easy to describe and process

13. 8 AI in Hockey

14. Sensors Simple reflex agents Information of players, puck, goal Playground Shoot at the corresponding direction If in shooting areas Then shoot Effectors 9 AI in Hockey 3.2 Simple Reflex Agent for Shooting Action [2]

15. Sensors Utility-based agents Current status of players, puck, goal Game over or limitation satisfied Changed status of players, puck, goal Playground Positions, Attack-defense status Value of the changed status Possible moving or passing Max value for attacking player Min value for defending player Decide moving or passing action Effectors 10 AI in Hockey 3.3 Utility-based Agent for Moving and Passing Puck Action [2]

16. 11 AI in Hockey • One on One Assignment • In real hockey game, usually the players do a “one on one” job. • Image how large the tree, if the decision of the move of one player needs to consider all other 9 players moving. • For reality and simpleness, I assign the players one on one. Problem needs to be noticed: This assignment is not simply according to the distance, e.g.

17. 12 AI in Hockey MiniMax Algorithm [2, 3]: Algorithm 1: Minimax Algorithm with Depth Limitation input : Current status of players, puck and the goal. output: Next moving or passing puck direction. CurStatus Current status of players, puck and the goal; call Max Algorithm (CurStatus) / Min Algorithm (CurStatus); ←

18. 13 AI in Hockey Algorithm 2: Max Algorithm for Minimax Algorithm. input : Status of players, puck and the goal. output: A moving or puck passing direction. if Goal or Limitation achievedthen for each possible move or passdo weight Evaluation this move or pass; end return the move or pass with minimal weight; end else for each possible move or passdo NextStatus The status of players, puck and the goal after this move or pass; call returned move or pass = Min Algorithm (NextStatus); weight Evaluation the returned move or pass; end return the returned move or pass with minimal weight; end ← ← ←

19. 14 AI in Hockey Algorithm 3: MinAlgorithm for Minimax Algorithm. input : Status of players, puck and the goal. output: A moving or puck passing direction. if Goal or Limitation achievedthen for each possible move or passdo weight Evaluation this move or pass; end return the move or pass with maximal weight; end else for each possible move or passdo NextStatus The status of players, puck and the goal after this move or pass; call returned move or pass = Max Algorithm (NextStatus); weight Evaluation the returned move or pass; end return the returned move or pass with maximal weight; end ← ← ←

20. 15 AI in Hockey • Optimization of Minimax Algorithm [2,3] • 1. Depth Limitation (already realized) • Minimax algorithm can bring to the victory. • However, it needs time and is not necessary • Idea that Minimax algorithm might bring to victory is consistent with the concept depth limitation • 2. Alpha-Beta Pruning (not realized)

21. 16 AI in Hockey 4. Future Work • Alpha –Beta pruning for Minimax algorithm • Evaluation function improvement • Fix some bugs • A new version between people and computer

22. 17 Reference Reference: [1] Buckland, M.: Programming Game AI by Example. Wordware Publishing, Inc. 2005. [2] Mouhoub, M.: http://www2.cs.uregina.ca/~mouhoubm/. Nov. 19, 2007. [3] Pinto, P.: Introducing the Min-Max Algorithm. July, 2002. [4] Wikipedia: http://en.wikipedia.org/wiki/decision_rules/. Nov. 18, 2007.