Knowledge acquisition for adative game AI

1. Knowledge acquisition for adative game AI Marc Ponsen et al. Science of Computer programming vol. 67, pp. 59-75, 2007 장수형

2. Outline • Introduction • Related work • Adaptive Script of Wargus • Experiment • Result • Alternative method

3. Introduction • Game • Become increasingly realistic • Graphical presentation • Capabilities of characters ‘living’ • Game AI • Game developers • Encompass techniques such as pathfinding, animation, collision physics • Academic researchers • Intelligent behavior • Inferior quality • Benefit from academic research into commercial games

4. Introduction • Adaptive game AI • Behavior of computer-controlled opponents • Potentially increase the quality of game AI • Incorporate a sufficient amount of correct prior domain knowledge • Dynamic scripting • Offline reinforcement learning technique • Dynamic scripting in a real-time strategy game called Wargus • Ambitious performance task • The quality of the knowledge base is essential

5. Introduction • Knowledge base • Manually encode • Take a long time • Sub-optimal due to analysis • Not generate satisfying result • Semi-automatically • Increase the performance • Machine learning • Added to knowledge bases • Evolution algorithm • Automatically • Evolutionary algorithm • Automatically transfers the domain knowledge

6. Related work • Few studies exist on learning to win complex strategy games • Focusing on simpler tasks • Relational Markov decision process model to some limited Wargus scenarios(Guestrin et al.) • Case-bases plan recognition approach for assisting Wargus player(Cheng and Thawonmas) • Manual knowledge acquisition • Typical RTS games(Age of Empires and Command & Cunquer) • Semi-automatic knowledge acquisition • Pattern recognition technique(Street et al.) • Automatic knowledge acquisition • Neural network for Backgammon, GO, Chess(Kirby)

7. RTS games • Usually focus on military combat • Control armies and defeat all opposing forces that are situated in a virtual battlefiled(often called a map) in real-time • Collecting and managing resources • Determines all decision for a computer opponent over the course of the whole game • Form of scripts which are list of game action that are executed sequentially • Constricting buildings, researching new technologies, and combat

8. Wargus • Clone of the popular RTS game Warcraft II • Open source • Stratagus engine • Strategy • Small Balanced Land Attack • Large Balanced Land Attack • Soldier’s Rush • Knight’s Rush

9. Complexity of Wargus • No single tactic dominates all others • The rock-paper-scissors principle • Large action space • The set of possible actions that can be executed at a particular moment • In Wargus… • A : number of assignments workers can perform • P : average number of workplace • T : number of troops • D : Average number of directions that a unit can moves • S : number of choices for a troop’s stance • B : number of buildings • R : average number of choices for research objectives at a building • C : average number of choice of units to create at a building

10. Complexity of Wargus

11. Complexity of Wargus • Decision complex of each state • Higher than the average number of possible moves in many board game such as chess(30)

12. Dynamic Scripting for Wargus • Game AI for complex games is mostly defines in scripts • Contain weaknesses, which human players can exploit • Dynamic script • Introduced by Spronck et al. • Ability to adapt to a human player’s behavior • The probability that a tactics is selected for a script is an increasing function of its associated weight value • Requirements • The game AI can be scripted • Domain knowledge on the characteristics of a successful script can be collected • Evaluation function can be designed to assess the success of the function’s execution

13. Dynamic Scripting for Wargus • Divide the game into a small number of distinct game states • Each state corresponds to a unique knowledge base

14. Weight adaptation in Wargus • F : The overall fitness • Fi : the stats fitness(state i) • Sd : the score for the dynamic player • So : the score for the player’s opponent

15. Weight adaptation in Wargus • Sx : the score of the dynamic player state x • Mx : the military points for player x • Bx : building points for player x

16. EA(Fitness Function) • Md : Military points for the dynamic player • Mo : Military points for the dynamic player’s opponent • b : break-even point • Ct : game cycle • Cmax : maximum game cycle(the longest time a game is allowed to continue)

17. EA(Encoding) • Construct, research, economy, combat genes..

18. Performance evaluation • Dynamic scripting under three condition • Manually acquired • Semi-automatically acquired • Automatically acquired • The other is controlled by a static script • Four strategy • SBLA, LBLA, SR, KR • Randomization turning point • Number of the first game in which the dynamic player statistically outperforms the static player • A low RTP value indicates good efficiency

19. Result

20. Conclusions • Three alternative for acquiring high-quality domain knowledge used by adaptive game AI • Manual, semi-automatic, automatic • Discussed dynamic scripting • Domain knowledge is crucial factor to the performance of dynamic scripting • The automatic knowledge acquisition approach takes best performance

21. Alternative method • Alternative method of script handling • Bayesian Network • Case study : StarCraft • ‘Adaptive Reasoning Mechanism with Uncertain Knowledge for Improving Performance of Artificial Intelligence in StarCraft

22. 상성파악 • 전략과 유닛의 상성 파악

23. 베이지안 네트워크 설계 • 불확실한 지식정보 • 상대방 진영으로의 정찰 시도 • 지어진 건물들의 구성 • 생산한 유닛의 구성 • 건물과 유닛의 개수 • 위의 정보들을 얻어낸 시각 • 거짓정보는 아니지만 완벽한 정보도 아니다 • 숨겨진 유닛, 숨겨진 건물, 지어지다가 취소된 건물

24. 스크립트 선택 • 정보 추론 후 가장 효과적인 대응 스크립트 선택

25. 결과 • 실험결과

26. E.N.D