Autonomous AI

1. Autonomous AI Steve Hookway 9/13/04

2. Outline • Constraining Autonomous Behavior (3.1) • Simple Techniques for Coordinated Behavior (3.2) • Synthetic Adversaries for Urban Combat Training (Laird et al.)

3. No One Lives Forever 2 • UNITY agent Cate Archer fights the H.A.R.M. Worldwide crime syndicate • Use a combination of stealth and brute force to work through each level

4. No One Lives Forever 2 • Excellent reviews for both game-play and AI Games Domain (5/5) "NOLF 2 is extremely hard to fault. Its presentation is top-notch, its levels are varied and interesting, its AI is convincing, and the whole game drips with love and attention to detail. In fact, it's hard to remember a single-player FPS we enjoyed this much since Deus Ex ... Any first-person shooter fan should consider this an essential purchase." HomeLan Fed (94%) "one of the best single player first person shooter games ever made. With solid AI, great looking levels and graphics, fun weapons and gadgets, excellent voice acting, and a healthy dose of humor, A Spy In H.A.R.M’s way is much better than the original and that’s saying something"

5. Ideal vs Actual Behavior • Ideally completely autonomous AI • Goal based system • Tagged objects • Realistically human behaviors need to be added • Total autonomy does not necessarily lead to believable behavior

6. Human Characteristics • Ownership • Dependency • Responsibility • Relevance • Priority • State of Consciousness • Expected State • Presence of Others

7. Ownership • Without constraint all objects of interest are up for grabs • Implement ownership by grouping objects into sets • Each character has his own set of objects • “Is this Object mine to use?”

8. Dependency • The use of one object depends on the use of another object • Washing hands after using bathroom • Can’t just make one inseparable behavior • Enable, disable objects for dependency • “Is there anything I need to do before using this object?”

9. Responsibility • Responsibility as a function of location • Keeps player from being overwhelmed by enemies • Responsibility as a function of the class of character • Set which objects are available to be used by that class of character • “What is my purpose in life?”

10. Relevance • A minor character’s goals should not have a major influence on the world! • Provide a ceiling to the amount of influence an AI character can have over their surroundings • “How will my actions affect the player’s experience?”

11. Priority • Character presented with multiple opportunities simultaneously • 3 categories of behavior • Aggressive > investigative > relaxed • Relative priorities within categories • “What is most important right now?”

12. State of Consciousness • Alive, Sleeping, Unconscious, Dead • Players emit stimuli depending on state • Can act according to previous or current state • “Am I awake?”

13. Expected State • Consciousness is an example of general concept State • Objects have associated state • Notice when an object is out of place • Notice when a player or object’s state has changed • “Has anything changed?”

14. Presence of Others • Behavior depends on who is present • Characters need to take each other into account • “How will this action affect other characters?”

15. Outline • Constraining Autonomous Behavior (3.1) • Simple Techniques for Coordinated Behavior (3.2) • Synthetic Adversaries for Urban Combat Training (Laird et al.)

16. Coordinate Behavior • Split Up • Get out of my way • Get down • What’s going on

17. “Split Up” • Agents are driven by same logic and therefore behave in the same way upon receiving a stimulus • Same shortest path algorithm • Nearby agents follow same path • Crowding!

18. Split up cont. • Path of waypoints to target • Reservations • Shortest path as least costly path • A node is not released until agent completes path • Reserve the node BEFORE the destination • Different paths give illusion of coordination

19. E E 1 1 1 50 1 1 1 1 1 1 A A B B Pathfinding Reserved

20. Maximum Occupancy • The pathfinding algorithm works well in combat • Firing range prevents crowding • When investigating, crowding becomes a problem • First agent should reserve object under investigation • Other agents maintain distance

21. Blackboard Communication • Post and query public information • Avoids overhead in adding data to actual pathfinding data structures themselves • Makes code more maintainable

22. A Record on the Blackboard Struct BlackboardRecord { BBRecordType eType; ObjectID idPoster; ObjectID idTarget; Int nData; };

23. The Blackboard … Class Cblackboard { public: void PostBBRecord(BBRecordType eType, ObjectID idPoster, ObjectID idTarget, int nData); void RemoveBBRecord(BBRecordType eType, ObjectID idPoster); void RemoveAllBBRecords(BBRecordType eType); int CountBBRecords(BBRecordType eType, ObjectID idTarget); BlackboardRecord* GetBBRecord(BBRecordType eType, ObjectID idTarget); protected: BlackboardRecord* m_aBBRecords[MAX_BB]; };

24. Agent path planning • Clear any previously reserved volumes and reserve volume before destination: g_pBlackboard -> RemoveBBRecord(kBB_ReservedVolume, myID); g_pBlackboard -> PostBBRecord(kBB_ReservedVolume, myID, volumeID,NULL);

25. Agent path planning cont. • Check if volume is reserved and apply cost penalty if it is float fCost = pVolume -> fCost; if(g_pBlackboard->GetBBRecord(kBB_ReservedVolume, volumeID) { fCost *= 50.f; }

26. “Get Out of My Way” • It is inevitable that agents will get in each other’s way • Too much overhead to recalculate optimal position • Better to deal gracefully with obstruction

27. Get out of My Way cont. • The obstructed agent can do nothing • The obstructed agent can fire anyway, either killing or going through his ally • The obstructed agent can move • The obstructed agent’s ally can move

28. Implementation • Obviously options 3 and 4 are the way to go • Requests ally moves • If ally is busy (reloading) agent moves • Leads to a dynamic environment with everyone in motion looking for a clear shot

29. Implementation cont. • Previous method works well with ranged weapons but not well with close-ranged melee weapons • Resort to KungFu style

30. “Get Down!” • Agents have a limited number of actions • We don’t want all agents take same action simultaneously • Use blackboard to post actions • Agent posts action and time • Before taking action agent checks to see if ally has done same action within a threshold time

31. “What’s Going On?” • We don’t want agents to stand idol while a nearby ally is taking damage • The same sensory system that detects enemies can be used to detect each other • When agent detects danger or takes damage, it emits a signal to any other agent in earshot

32. Coordination We have achieved simple coordination without the addition of a group behavior layer!

33. Outline • Constraining Autonomous Behavior (3.1) • Simple Techniques for Coordinated Behavior (3.2) • Synthetic Adversaries for Urban Combat Training (Laird et al.)

34. Synthetic Adversaries • Virtual training of soldiers for Military Operations on Urbanized Terrain (MOUT) • System based on a bot developed to play Quake • Uses Infiltration’s (UT’s) environment

35. Requirements for Synthetic Adversaries • Competence • Taskability • Observational fidelity • Behavior variability • Transparency • Minimal computational footprint

36. Competence • Must perform the actions and behaviors humans perform in this domain • Many different roles • Defending a room • Sentry • Acting as a leader • Focus on a small team interacting

37. Competence • Threat data structure • Used to maintain awareness about enemy • Building map data structure • Mission object • Operators are used to take action in the world

38. Taskability • Must be able to be assigned new missions and change objectives during an exercise • All agent knowledge is independent of the specific mission • MOUTBots can issue a limited amount of commands to change the mission of other bots

39. Observational Fidelity • Model human behavior only in those aspects which can be observed • This is one place we can limit computational time • Subject to making the some “human errors”

40. Behavior Variability • Must not be predictable • Allows for soldiers to train in the same situation but experience a new encounter • Limits the amount of surprises in real life combat • Assign probability to each potential action

41. Transparency • Allow after action review by keeping a record explaining actions • Used an application specific visualization tool to show what the agent believes the status of the map to be • Can look at an agent’s goal; “defend room” goal among agents active goals

42. Minimal Computational Footprint • Targeted for personal computers • Majority of processing reserved for graphics and physics modeling • Have been able to run as many as 20 bots in UT while keeping the game’s frame rate

43. Conclusion • Fully autonomous AI is a long way off, but it is still possible to model human behavior

44. BONUS MATERIAL

45. Creature Smarts Is it possible to build an artificial intelligence that’s as clever, as adaptive and as captivating as the average dog? • C4 and Sheep|Dog • Duncan the Terrier

46. Autonomous AI • Build a creature that can interact with each other and human participants • Self sufficient reasoning • Learning • Action Selection • Goals

47. System Architecture Networked Devices Local Devices Creature 1 Mental Rep World Model Input Server Creature 2 Mental Rep Renderers World State

48. Internal Blackboard Object of Attention Motor Desired Motor Actual Cognitive Architecture The World Sensory System Proprioceptive System Perception System Working Memory Action System Attention Selection Action Selection Navigation System Motor System

49. Sensory System • Sensory honesty • Input from world (creatures point of view)

50. Everything Head Location Shape World Location Sound Utterance Humanoid Sheep Down Sit Away Perception System • Percept-perceive stimulus • Classification hierarchy