A New Kind of Episodic Information and Memory for NPC Design in Computer Games

1. A New Kind of Episodic Information and Memory for NPC Design in Computer Games Wan Ching How.c.ho@herts.ac.ukAdaptive Systems Research GroupUniversity of Hertfordshire

2. Overview • Introduction • Typical techniques used in designing & developing NPCs • Nouvelle AI concepts applying to the design and development process of computer games • Episodic information (memory) for NPCs • Own research work • Narrative autobiographic agents • Looking into the games (case studies) • Black & White • The Sims

3. Introduction (1) • Typical techniques used in designing & developing NPCs: • Finite State Machine (FSM) • Scripting

4. Introduction (2) • How FSM works in controlling the behaviours (states) of NPCs: • A limited number of behaviours, or states, are pre-defined by the game designer. The actual state of the NPC is made to switch with seeming intelligence, depending on criteria also pre-defined by the designer. • To add unpredictability to NPCs, many designers incorporate fuzzy logic or random weighting into the decision-making process.

5. Victory Dance Health > 10% & Player not in view Search Win Wait & Charge Health > 10% & Player in view Attack Health < 10% & Player in view Health < 10% & Player not in view Escape Health = 0% Death

6. Introduction (3) • Scripting – a technique in which the control of game events and NPCs is not hard coded into the game, but defined using a high-level language. • A fixed sequence of actions or dialogs will be triggered when the condition has been matched, e.g. NPC answering a question selected by the player • Good to craft a illusion of AI

7. Introduction (4) • “Most game programmers…don’t know how to take techniques from academic AI research and make them work in a practical way.” (Rabin, AI Game Programming Wisdom) • “Currently the mood of the game industry is pragmatic, rather than rush to new technologies… developers were mostly focused on digesting what they had,” (Woodcock, 2002)

8. Imagine the problem of moving a patrolling guard through a number of rooms in a castle. The designer can sidestep the problem of teaching the guard how to work out the best path between rooms by simply drawing an invisible track for the guard to follow through the castle. • But, if something (or the player) is standing on that invisible track?...

9. Introduction (5) • Nouvelle AI concepts can be applied to the design and development process of computer games: • Embodied agents that are actually situated in realistic worlds. • Behaviour-based control architecture. • Learning techniques used for the simulation of adaptive behaviors produce creatures with intelligent capabilities. • Neural networks and genetic algorithms might seem to be useful…

10. Introduction (6) • Agent with embodiment and situatedness • Embodiment: accurately simulating the body of creatures, notably their interaction with the environment. • Situatedness: Embodied systems are mostly affected by their immediate surroundings. Using senses, only local information is gathered, similar to the way humans or animals interact with their environment.

11. Introduction (7) • Behaviour-based control architectures • Suitable for intelligent control because of their reliability and simplicity, often providing a foundation for more elaborate techniques. • E.g. subsumption control architecture (Brooks, 1985)

12. Introduction (8) • Agents with AI techniques of learning: • Teaching involves humans providing a set of examples that help the agent to behave until it's managed to understand what to do. • Imitation allows the agent to copy another player, who is usually human. It can thereby learn its behavior from a third-party experience. • Shaping sets up successive trials from which the agent can learn. After the agent learns to accomplish simple tasks, more complex ones are presented. • Trial and error places the agent in its environment and expects it to learn by trying out all the different approaches on its own.

13. Introduction (9) • Neural networks – attempts to model how neurons behave in the brain, agents can learn from experience. It allows agents to respond to players and accommodate changes in their behaviour without the need for the designer to explicitly program every possibility into the agent. • Genetic algorithm –uses digital evolution and selection to develop a solution to a problem by trial and error

14. Introduction (10) • But when “something misbehaves with one of these technologies, it’s not easy to fix. You can’t exclude the one thing that’s broken without destroying all of the other beautiful things in there. It’s all or nothing, which is a very difficult situation when deadlines approach,” • “The hardest thing in game AI is just making sure that the game never looks dumb. You’d be better off having an AI that was just above average all the time, rather than one that was brilliant 98 percent of the time and stupid 2 percent of the time.” (Rabin, 2002)

15. Episodic Information • How to increase the intelligence and believability of the characters? • NPC’s behaviour looks unnatural since there is no connection or simply players cannot see the reason when the NPC switches from one behaviour to another one. This is described as "schizophrenic" (Sengers, 2001) • Episodic information can be the historical grounding of the agent • AUTOBIOGRAPHIC MEMORY

16. Autobiographic Agents (1) Psychology[1] Autobiographic memory is a specific kind of episodic memory, may develop in human childhood. Minimal Artificial Life Agents[2] An agent possesses an autobiographic memory if it can create and access information about sequences of actions which it experienced during its lifetime. Autobiographic memory relates to meaningful events. Autobiographic Agents[2] Autobiographic agents are agents which are embodied and situated in a particular environment (including other agents), and which dynamically reconstruct their individual history (autobiography) during their lifetimes. Katherine Nelson (1993) “The Psychological and Social Origins of Autobiographic memory” Kerstin Dautenhahn (1996) “Embodiment in Animals and Artifacts”

17. Autobiographic Agents (2) The first previous work[1] showed how a single agent's survival can benefit from autobiographic memory. [2] 1. Wan Ching Ho, Kerstin Dautenhahn, Chrystopher L. Nehaniv (2003) “Comparing different control architectures for autobiographic agents in static virtual environments” in Intelligent Virtual Agents Workshop (IVA2003).2. VRML Model from www.atom.co.jp

18. Autobiographic Agents (3) • The second previous work[3] showed that autobiographic agents effectively extend their lifespan by embedding an Event-based memory as compared to a Purely Reactive subsumption control architecture, both in single-agent and multi-agent experiments. Multi-agent environmental interference dynamics result in a decreasing average lifespan of agents. 3. Wan Ching Ho, Kerstin Dautenhahn, Chrystopher L. Nehaniv, Rene te Boekhorst (2004) “Sharing Memories: An Experimental Investigation with Multiple Autonomous Autobiographic Agents” in The 8th Conference on Intelligent Autonomous Systems (IAS-8)

19. Autobiographic Agents for NPC Design in PC Games (1) • Nowadays only few behaviour simulation systems have made explicit use of episodic memory as a learning mechanism; where learning means individual adaptation processes that occur throughout a character’s brain (Isla & Blumberg, 2002). • The advantage of using episodic memory for learning, compared to other mechanisms such as reinforcement, neural networks or genetic algorithms, is speed, as the game character can form usable hypotheses for making decisions or selecting behaviours to execute in the future, after just one observation of users or other agents.

20. Autobiographic Agents for NPC Design in PC Games (2) • Story Telling and Memories • Story telling provides ‘empathic resonance’ through transferring own experiences or receiving experiences between autobiographic agents, that is a degree of empathic re-experiencing of (the internal state of) others since agents who received experiences may also reconstruct aspects of individual’s history of others (Dautenhahn, 1997).

21. Recognizing individuals • Although the receiver (autobiographic agent) can selectively choose the information to store into its own autobiographic memory from the sender, however, for the internal states of the receiver, a certain level of trust to a specific sender (another autobiographic agent) can be built up. • For example, the receiver had received many experiences from that sender, which brought benefits (such as avoiding dangers) to the receiver. • Consequently, story telling not only brings changes of the agents’ own memory but also the attitude that towards others, this is the impact of communicating with others. • In addition to this, the usefulness of stories which the agent received from others will be known only after the agent re-experienced them later; it is the same idea with the Evaluation element in the narrative structure (Linde, 1993), but applying in other agents’ stories.

22. Understanding a story from another agent • Not simply matching the explicit contents of the story to the agents’ own stories in its memory • Not using high-level cognitive process from the perspective of human understanding. • The agent has to employ a low-level abstraction process for recognizing the meaning and the contents of this story from its own accumulated stories. • Then the agent is not going to directly merge this story with other old stories and store it into its own memory, but to shape it to the agent’s own story figure, i.e. using its own memory schemata to remember the story.

23. Narrative Influenced by AI in PC Games • Top-down design: Embedded narrative with fixed story line & AI cannot make much influence. • Bottom-up design (like The Sims and Black & White): Emergent narrative with open-ended story & AI can be highly involved when there are much more chances for the players interacting with the characters.

24. Case Studies (1)

25. The Sims • Characters always behave unexpectedly • The intelligence to interact with any object is not built into the Sims themselves. Rather, they are equipped with a few basic needs, for food, say, or entertainment. Objects within the game advertise their ability to satisfy some of these needs to any Sims who wander nearby.

26. The Sims series: • Need strategies to play • Open-ended • Game play soon goes beyond any developer’s capacity to pre-program situations • Have the greatest need for advanced AI technologies

27. Case Studies (2)

28. Black & White • Giving the player an in-game representative - a Creature, which acts autonomously, but it can be trained by a system of punishments and rewards. • It watches the player’s actions and attempts to divine the intent behind them, a technique called empathic learning & imitation.

29. The next big challenge for game AI may be getting a game’s cast of characters better at learning and social interaction. (Cass, 2002)

30. End … Thanks for your attention.