topic 4 agent architectures n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Topic 4: Agent architectures PowerPoint Presentation
Download Presentation
Topic 4: Agent architectures

Loading in 2 Seconds...

play fullscreen
1 / 103

Topic 4: Agent architectures - PowerPoint PPT Presentation


  • 119 Views
  • Uploaded on

Topic 4: Agent architectures. general agent architectures deductive reasoning agents practical reasoning agents reactive agents hybrid agents. Agents: definition. M. Wooldridge An agent is a computer system … … that is situated in some environment, …

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Topic 4: Agent architectures' - dinh


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
topic 4 agent architectures

Topic 4: Agent architectures

general

agent architectures

deductive reasoning agents

practical reasoning agents

reactive agents

hybrid agents

agents definition
Agents: definition
  • M. Wooldridge
    • An agent is
      • a computer system …
      • … that is situated in some environment, …
      • … capable of autonomous action in this environment …
      • … in order to meet its design objectives.

agent

observation

action

environment

slide3
agent properties
      • reactivity

reacts to stimuli (changes in env., communication, …)

      • autonomy

does not require user interaction

      • pro-activeness

aims to achieve its own goals, therefore initiates appropriate actions

      • social ability

cooperates / coordinates / communicates / …

      • embodied

situated in the environment

      • mobile

moves around network sites

      • learning

learn from past experiences

essential

extra

agents versus objects
Agents versus Objects
  • Objects (Java / C++ / C# / Smalltalk / Eiffel / …)
      • encapsulate
        • state “attributes” / “data members” / …
        • behaviour “operations” / “methods” / …
      • represent real-world entity
      • own responsibility
agents versus objects cont

agent 1

agent 2

action

Agents versus Objects (cont.)
  • differences
      • autonomy: who decides to execute a particular “action”
        • objects have control over state (through operations)objects do not have control over their behaviour
          • any object can call any public operation of the object
          • object cannot decide when to execute its behaviour
        • agents request an action from another agent […]
          • control lies entirely within receiving agent
        • cfr. humans
        • “objects do it for free, agents do it for money” because they want to”
agents versus objects cont1

agent

Agents versus Objects (cont.)
  • differences (cont.)
      • behaviour architecture: integration of flexible autonomous behaviour
        • objects
          • operations to offer behaviour
        • agents
          • integrate reactive behaviour social behaviour proactive behaviour …
          • cfr. humans
agents versus objects cont2

agent

Agents versus Objects (cont.)
  • differences (cont.)
      • inherently multi-threaded
        • objects
          • no separate thread of control
          • … active objects
        • agents
          • conceptually different threads of control
agents versus expert systems
Agents versus Expert Systems
  • Expert systems e.g. MYCIN
      • act as computerized consultant
      • for physicians
        • MYCIN knows about blood diseases in humans
          • a wealth of knowledge about blood diseases, in the form of rules
          • a doctor can obtain expert advice about blood diseases by giving MYCIN facts, answering questions, and posing queries
    • differences
      • inherently disembodied
        • do not operate in an environment
      • no reactive/proactive behaviour
        • user-controlled
      • no social behaviour
        • as in cooperation / coordination / negotiation / …
agent architecture how to do the right thing
Agent architecture:how to do the right thing ?
  • Pattie Maes [1991]

‘[A] particular methodology for building [agents]. It specifies how . . . the agent can be decomposed into the construction of a set of component modules and how these modules should be made to interact. The total set of modules and their interactions has to provide an answer to the question of how the sensor data and the current internal state of the agent determine the actions . . . and future internal state of the agent.’

 model of agent machinery

  • abstract architecture
    • elements
      • E set of environment states
      • Ac set of actions
      • Ag: E*  Ac (mapping)
slide10
agents with state

I internal agent state

see E  Per

action I  Ac

next I x Per  I

agent

action

see

next

state

act

observe

environment

concrete agent architectures
Concrete agent architectures
  • Deductive reasoning agents
        • 1956 – present
        • “Agents make decisions about what to do via symbol manipulation. Its purest expression, proposes that agents use explicit logical reasoningin order to decide what to do.”
  • Practical reasoning agents
        • 1990 – present
        • “Agent use practical reasoning (towards actions, not towards beliefs) – beliefs / desires / intentions.”
  • Reactive agents
        • 1985 – present
        • “Problems with symbolic reasoning led to a reaction against this — led to the reactive agentsmovement.”
  • Hybrid agents
        • 1989 – present
        • “Hybrid architectures attempt to combine the best of reasoning and reactive architectures.”
1 deductive reasoning agents
1. Deductive reasoning agents
  • architectures based on ideas of “symbolic AI”
      • symbolic representation
        • environment
        • behaviour
        • goals
      • representation: logical formulae
      • syntactic manipulation: logical deduction / theorem proving

Δ “deliberative agent”

deductive reasoning agents agents as theorem provers
Deductive reasoning agents:Agents as theorem provers
  • deliberative agents
    • databases ofbeliefs are specified using formulae of first-order predicate logic
        • e.g. open (door1) closed (door2) visible (wall32) …
      • but:
        • just beliefs … not certified knowledge …
        • semantic not specified …open (door) may mean something exotic to the agent designer
    • set of deduction rules
deductive reasoning agents agents as theorem provers cont
Deductive reasoning agents:Agents as theorem provers (cont.)
  • agent’s action selection function

for eachaction a

ifprecondition-for-action a can be inferred from current beliefs

return a

for each action a

ifprecondition-for-action a is not excluded by current beliefs

return a

return null

deductive reasoning agents agents as theorem provers cont1

(0,2)

(1,2)

(2,2)

robot

(0,1)

(1,1)

(2,1)

(0,0)

(1,0)

(2,0)

Deductive reasoning agents:Agents as theorem provers (cont.)
  • vacuum example [Russell & Norvig, 1995]

percept: dirt or null.

actions: forward, suck, or turn.

goal: transverse room, remove dirt

slide16
domain predicates

In (x,y) agent is at (x,y)

Dirt (x,y) there is dirt at (x,y)

Facing (d) agent is facing direction d

Wall (x,y) there is a wall at (x,y)

  • deduction rules

x y Wall(x,y)  Free(x,y)

  • cleaning action rule (will take priority)

In (x,y)  Dirt (x,y)  Do (suck)

  • if agent is at location (x,y) and perceives dirt: remove dirtotherwise transverse the world
      • For example...

In(0,0)  Facing(north)  Free (0,1)  Dirt(0,0)  Do(forward)

In(0,1)  Facing(north)  Free (0,2)  Dirt(0,1)  Do(forward)

In(0,2)  Facing(north)  Dirt(0,2)  Do(turn)

In(0,2)  Facing(east)  Free (1,2)  Do(forward)…

deductive reasoning agents agents as theorem provers cont2
Deductive reasoning agents:Agents as theorem provers (cont.)
  • “calculative rationality”
      • the selected action is the result of decision making on state in the beginning of the process of decision making
      • not acceptable in environments that change faster than decision making
deductive reasoning agents agents as theorem provers cont3
Deductive reasoning agents:Agents as theorem provers (cont.)
  • Advantages
      • clean logical semantics
      • expressive
      • well-researched domain of logic
  • Problems
      • how to build internal representation from percepts
        • e.g. image  logical formulae
      • inherent computational complexity of theorem proving timely functions !
        • many (most) search-based symbol manipulation algorithms of interest are highly intractable
concrete agent architectures1
Concrete agent architectures
  • Deductive reasoning agents
        • 1956 – present
        • “Agents make decisions about what to do via symbol manipulation. Its purest expression, proposes that agents use explicit logical reasoningin order to decide what to do.”
  • Practical reasoning agents
        • 1990 – present
        • “Agent use practical reasoning (towards actions, not towards beliefs) – beliefs / desires / intentions.”
  • Reactive agents
        • 1985 – present
        • “Problems with symbolic reasoning led to a reaction against this — led to the reactive agentsmovement.”
  • Hybrid agents
        • 1989 – present
        • “Hybrid architectures attempt to combine the best of reasoning and reactive architectures.”
2 practical reasoning agents
2. Practical Reasoning Agents
  • what is practical reasoning ?
    • “reasoning directed towards actions”
      • “practical reasoning is a matter of weighing conflicting considerations for and against competing options, where the relevant considerations are provided by what the agent desires/values/cares about and what the agent believes.” [Bratman]
    • distinguish practical reasoning from theoretical reasoning:
      • theoretical reasoning is directed towards beliefs
      • practical reasoning is directed towards actions
bdi architectures
BDI architectures
  • BDI - a theory of practical reasoning - Bratman, 1988
  • for “resource-bounded agent”
  • includes
    • means-end analysis
    • weighting of competing alternatives
    • interactions between these two forms of reasoning
  • Core concepts
    • Beliefs = information the agent has about the world
    • Desires = state of affairs that the agent would wish to bring about
    • Intentions = desires (or actions) that the agent has committed to achieve
slide22
BDI particularly compelling because:
  • philosophical component - based on a theory of rational actions in humans
  • software architecture - it has been implemented and successfully used in a number of complex fielded applications
    • IRMA - Intelligent Resource-bounded Machine Architecture
    • PRS - Procedural Reasoning System
  • logical component - the model has been rigorously formalized in a family of BDI logics
    • Rao & Georgeff, Wooldrige
    • (Int Ai )   (Bel Ai)
practical reasoning agents cont
Practical Reasoning Agents (cont.)
  • human practical reasoningpractical reasoning = deliberation + means-ends reasoning
      • deliberation
        • deciding whatstate of affairs you want to achievethe outputs of deliberation are intentions
      • means-ends reasoning
        • deciding howto achieve these states of affairsthe outputs of means-ends reasoning are plans
practical reasoning agents cont1
Practical Reasoning Agents (cont.)
  • deliberation
      • intentions
  • means-ends reasoning
      • planning
  • architecture
practical reasoning agents 1 deliberation intentions and desires
Practical Reasoning Agents:1. Deliberation: Intentions and Desires
  • intentions are stronger than desires
      • “My desire to play basketball this afternoon is merely a potential influencer of my conduct this afternoon. It must vie with my other relevant desires [. . . ] before it is settled what I will do. In contrast, once I intend to play basketball this afternoon, the matter is settled: I normally need not continue to weigh the pros and cons. When the afternoon arrives, I will normally just proceed to execute my intentions.” [Bratman, 1990]
practical reasoning agents intentions
Practical Reasoning Agents: Intentions
  • agents are expected to determine ways of achieving intentions
      • If I have an intention to Φ, you would expect me to devote resources to deciding how to bring about Φ
  • agents cannot adopt intentions which conflict
      • If I have an intention to Φ, you would not expect me to adopt an intention Ψthat was incompatible with Φ
  • agents are inclined to try again if their attempts to achieve their intention fail
      • If an agent’s first attempt to achieve Φfails, then all other things being equal, it will try an alternative plan to achieve Φ
  • agents believe their intentions are possible
      • That is, they believe there is at least some way that the intentions could be brought about.
  • agents do not believe they will not bring about their intentions
      • It would not be rational of me to adopt an intention to Φ if I believed that I would fail with Φ
  • under certain circumstances, agents believe they will bring about their intentions
      • If I intend Φ, then I believe that under “normal circumstances” I will succeed withΦ
  • agents need not intend all the expected side effects of their intentions
      • I may believe that going to the dentist involves pain, and I may also intend to go to the dentist — but this does not imply that I intend to suffer pain!
practical reasoning agents 2 means ends reasoning
Practical Reasoning Agents:2. Means-ends Reasoning

intention

(goal / task)

beliefs

(state of

environment)

possible

actions

planner

means-ends

reasoning

plan to

achieve goal

the blocks world
The Blocks World
  • illustrate means-ends reasoning with reference to the blocks world
  • Contains a robot arm, 3 blocks (A, B, and C) of equal size, and a table-top

A

B

C

the blocks world1
The Blocks World
  • Here is a representation of the blocks world described above:Clear(A) On(A, B) OnTable(B) OnTable(C)
  • Use the closed world assumption: anything not stated is assumed to be false
the blocks world2
The Blocks World
  • A goalis represented as a set of formulae
  • Here is a goal:OnTable(A)  OnTable(B)  OnTable(C)

B

C

A

the blocks world3
The Blocks World
  • Actionsare represented using a technique that was developed in the STRIPS planner
  • Each action has:
      • a namewhich may have arguments
      • a pre-condition listlist of facts which must be true for action to be executed
      • a delete listlist of facts that are no longer true after action is performed
      • an add listlist of facts made true by executing the action

Each of these may contain variables

the blocks world operators
The Blocks World Operators
  • Example 1:The stackaction occurs when the robot arm places the object xit is holding is placed on top of object y.Stack(x, y)preClear(y)  Holding(x)delClear(y)  Holding(x)addArmEmpty  On(x, y)

A

B

the blocks world operators1
The Blocks World Operators
  • Example 2:The unstackaction occurs when the robot arm picks an object xup from on top of another object y.UnStack(x, y)preOn(x, y)  Clear(x)  ArmEmptydelOn(x, y)  ArmEmpty addHolding(x)  Clear(y)Stack and UnStack are inversesof one-another.

A

B

the blocks world operators2
The Blocks World Operators
  • Example 3:The pickupaction occurs when the arm picks up an object xfrom the table.Pickup(x)preClear(x)  OnTable(x)  ArmEmptydelOnTable(x)  ArmEmpty addHolding(x)
  • Example 4:The putdownaction occurs when the arm places the object xonto the table.Putdown(x)preHolding(x)delHolding(x) addClear(x)  OnTable(x)  ArmEmpty
a plan
A Plan
  • What is a plan?A sequence (list) of actions, with variables replaced by constants.

a142

a1

I

G

a17

slide36

percepts

Belief revision

Beliefs

Knowledge

B = brf(B, p)

Opportunity

analyzer

Deliberation process

Desires

D = options(B,D, I)

Intentions

Filter

I = filter(B, D, I)

Means-ends

reasoner

Intentions structured

in partial plans

 = plan(B, I)

Library of plans

Plans

Executor

actions

3. BDI Architecture

practical reasoning agents cont2

what are the options (desires) ?

  • how to choose an option ?
  • incl. filter
  • chosen option  intention …
  • when to reconsider intentions !?
Practical Reasoning Agents (cont.)
  • agent control loop

while true

observe the world;

update internal world model;

deliberate about what intention to achieve next;

use means-ends reasoning to get a plan for the intention;

execute the plan

end while

implementing practical reasoning agents
Implementing Practical Reasoning Agents
  • Let’s make the algorithm more formal:
implementing practical reasoning agents1
Implementing Practical Reasoning Agents
  • this version: optimal behaviour if
      • deliberation and means-ends reasoning take a vanishingly small amount of time;or
      • the world is guaranteed to remain static while the agent is deliberating and performing means-ends reasoning;or
      • an intention that is optimal when achieved at time t0 (the time at which the world is observed) is guaranteed to remain optimal until time t2 (the time at which the agent has found a course of action to achieve the intention).
deliberation
Deliberation
  • The deliberate function can be decomposed into two distinct functional components:
    • option generationin which the agent generates a set of possible alternatives;Represent option generation via a function, options, which takes the agent’s current beliefs and current intentions, and from them determines a set of options (= desires)
    • filteringin which the agent chooses between competing alternatives, and commits to achieving them.In order to select between competing options, an agent uses a filterfunction.
practical reasoning agents cont3
Practical Reasoning Agents (cont.)
  • If an option has successfully passed trough the filter function and is chosen by the agent as an intention, we say that the agent has made a commitment to that option
  • Commitment implies temporal persistence of intentions; once an intention is adopted, it should not be immediately dropped out.

Question: How committed an agent should be to its intentions?

      • degrees of commitments
        • blind commitment
          • ≈ fanatical commitment: continue until achieved
        • single-minded commitment
          • continue until achieved or no longer possible
        • open-minded commitment
          • continue until no longer believed possible
commitment strategies
Commitment Strategies
  • An agent has commitment both
    • to ends(i.e.,the wishes to bring about)
    • and means(i.e., the mechanism via which the agent wishes to achieve the state of affairs)
  • current version of agent control loop is overcommitted, both to means and ends modification: replanif ever a plan goes wrong
slide44

Reactivity, replan

“Blind commitment”

commitment strategies1
Commitment Strategies
  • this version still overcommitted to intentions:
    • never stops to consider whether or not its intentions are appropriate

 modification: stop for determining whether intentions have succeeded or whether they are impossible:

“Single-minded commitment”

single minded commitment
Single-minded Commitment

Dropping intentions

that are impossible

or have succeeded

Reactivity, replan

intention reconsideration
Intention Reconsideration
  • Our agent gets to reconsider its intentions when:
    • it has completely executed a plan to achieve its current intentions; or
    • it believes it has achieved its current intentions; or
    • it believes its current intentions are no longer possible.

 This is limited in the way that it permits an agent to reconsiderits intentions

 modification: Reconsider intentions after executing every action

“Open-minded commitment”

intention reconsideration1
Intention Reconsideration
  • But intention reconsideration is costly!A dilemma:
    • an agent that does not stop to reconsider its intentions sufficiently often will continue attempting to achieve its intentions even after it is clear that they cannot be achieved, or that there is no longer any reason for achieving them
    • an agent that constantlyreconsiders its attentions may spend insufficient time actually working to achieve them, and hence runs the risk of never actually achieving them
  • Solution: incorporate an explicit meta-level controlcomponent, that decides whether or not to reconsider
possible interactions
Possible Interactions
  • The possible interactions between meta-level control and deliberation are:
intention reconsideration2
Intention Reconsideration
  • Situations
    • In situation (1), the agent did not choose to deliberate, and as consequence, did not choose to change intentions.Moreover, if it hadchosen to deliberate, it would not have changed intentions. the reconsider(…) function is behaving optimally.
    • In situation (2), the agent did not choose to deliberate, but if it had done so, it wouldhave changed intentions. the reconsider(…) function is not behaving optimally.
    • In situation (3), the agent chose to deliberate, but did not change intentions. the reconsider(…) function is not behaving optimally.
    • In situation (4), the agent chose to deliberate, and did change intentions. the reconsider(…) function is behaving optimally.
  • An important assumption: cost of reconsider(…) is muchless than the cost of the deliberation process itself.
optimal intention reconsideration
Optimal Intention Reconsideration
  • Kinny and Georgeff’s experimentally investigated effectiveness of intention reconsideration strategies
  • Two different types of reconsideration strategy were used:
    • boldagentsnever pause to reconsider intentions, and
    • cautiousagentsstop to reconsider after every action
  • Dynamismin the environment is represented by the rate of world change, g
optimal intention reconsideration1
Optimal Intention Reconsideration
  • Results (not surprising):
    • If g is low (i.e., the environment does not change quickly),bold agents do well compared to cautious ones.
      • cautious ones waste time reconsidering their commitments while bold agents are busy working towards — and achieving — their intentions.
    • If g is high (i.e., the environment changes frequently),cautious agents tend to outperform bold agents.
      • they are able to recognize when intentions are doomed, and also to take advantage of serendipitous situations and new opportunities when they arise.
implemented bdi agents irma
Implemented BDI Agents: IRMA
  • IRMA – Intelligent Resource-bounded Machine Architecture – Bratman, Israel, Pollack
  • IRMA has four key symbolic data structures:
      • a plan library
      • explicit representations of
        • beliefs: information available to the agent — may be represented symbolically, but may be simple variables
        • desires: those things the agent would liketo make true — think of desires as tasksthat the agent has been allocated;
        • intentions: desires that the agent has chosenand committed to
slide56
IRMA
  • Additionally, the architecture has:
      • a reasoner
        • for reasoning about the world; an inference engine
      • a means-ends analyzer
        • determines which plans might be used to achieve intentions
      • an opportunity analyzer
        • monitors the environment, and as a result of changes, generates new options
      • a filtering process
        • determines which options are compatible with current intentions
      • a deliberation process
        • responsible for deciding upon the ‘best’ intentions to adopt
practical reasoning agents procedural reasoning system prs
Practical Reasoning Agents:Procedural Reasoning System (PRS)
  • “BDI-architecture” (beliefs / desires / intentions)
      • explicit data structures for b/d/i
  • planning
      • no “on-the-fly” planning  plan libraries
        • a plan: goal (post-condition)context (pre-condition)body (sequence of actions / subgoals)
  • intention stack

sensor input

agent

beliefs

plans

interpreter

desires

intentions

action

implemented bdi agents prs
Implemented BDI Agents: PRS
  • Another BDI-based agent architecture
    • PRS – Procedural Reasoning System (Georgeff, Lansky)
      • each agent is equipped with a plan library, representing that agent’s procedural knowledge: knowledge about the mechanisms that can be used by the agent in order to realize its intentions
      • the options available to an agent are directly determined by the plans an agent has: an agent with no plans has no options
      • in addition, agents have explicit representations of beliefs, desires, and intentions, as above
practical reasoning agents cont4
Practical Reasoning Agents (cont.)
  • further implementations
    • Jadex (java)
    • Jason (java – sourceforge.net)
    • JAM (java)
    • JACK (java)
concrete agent architectures2
Concrete agent architectures
  • Deductive reasoning agents
        • 1956 – present
        • “Agents make decisions about what to do via symbol manipulation. Its purest expression, proposes that agents use explicit logical reasoningin order to decide what to do.”
  • Practical reasoning agents
        • 1990 – present
        • “Agent use practical reasoning (towards actions, not towards beliefs) – beliefs / desires / intentions.”
  • Reactive agents
        • 1985 – present
        • “Problems with symbolic reasoning led to a reaction against this — led to the reactive agentsmovement.”
  • Hybrid agents
        • 1989 – present
        • “Hybrid architectures attempt to combine the best of reasoning and reactive architectures.”
3 reactive agents
3. Reactive Agents
  • “symbolism”
        • “intelligence requires a symbolic representation of the world”
  • “connectionism”
        • intelligent behaviour is product of interaction with environment and emerges from simple behaviours
        • example: social behaviour (e.g. social insects)

environment

slide63
Brooks’ subsumption architecture
  • PENGI
  • Situated Automata RULER / GAPPS
  • Pattie Maes: “behaviour network architecture”
  • Free-flow architecture
rodney brooks
Rodney Brooks
  • two basic ideas
      • situatedness and embodiment
          • “real” intelligence is situated in the world, not in disembodied systems such as theorem provers or expert systems
      • intelligence and emergence
          • “intelligent” behaviour arises as a result of an agent’s interaction with its environment
          • intelligence is ‘in the eye of the beholder’, it is not an innate, isolated property
  • two key theses
      • intelligence without representation
          • intelligent behaviour can be achieved without explicit representations of the kind that symbolic AI proposes
      • intelligence without reasoning
          • intelligent behaviour can be achieved without explicit abstract reasoning of the kind that symbolic AI proposes
rodney brooks subsumption architecture
Rodney Brooks:Subsumption Architecture
  • subsumption architecture
    • “a hierarchy of task-accomplishing behaviours”
    • each rule / behaviour:

if situation then action

 map perception input directly to actions

    • each behaviour ‘competes’ with others to exercise control over the agent
    • actions selection: by organising rules in layers
      • lower layers inhibit (“subsume”) higher layers
      • lower layers represent more primitive kinds of behaviour (such as avoiding obstacles), and have precedence over layers further up the hierarchy
  • computationally: very simple
  • “some of the robots do tasks that would be impressive if they were accomplished by symbolic AI systems”
rodney brooks subsumption architecture cont
Rodney Brooks:Subsumption Architecture (cont.)
  • a behaviour is a pair (c,a) where c  P is a set of percepts called the condition and a  A is an action
  • a behaviour (c,a) can fire when the environment is in state s  S iff see(s)  c
  • Beh = {(c,a) ¦ c  P and a  A} be the set of all such rules
  • associated with an agent’s set of behaviour rules R  Beh is a binary inhibition relation on the set of behaviours <  R  RThis relation is a total ordering on R
  • b1 < b2 iff (b1,b2) < “b1 inhibits b2” “b1 subsumes b2” (b1 is lower in the hierarchy than b2, and will get priority over b2)
  • the “action selection function” is then defined as follows...
rodney brooks subsumption architecture cont1

compute all behaviours that can fire

determine if some behaviours subsume others

return appropriate action or null

Rodney Brooks:Subsumption Architecture (cont.)

function action (p : P) : A

var fired : (R)

var selected : A

begin

fired := {(c,a) ¦ (c,a)  R and p  c}

for each (c,a)  fired do

if ((c’,a’)  fired such that (c’,a’) < (c,a)) then

return a

end-if

end for

return null

end function action

slide68

Subsumption Architecture:example – Mars explorer

  • Mars explorer (L. Steels)
    • objective
      • to explore a distant planet, and in particular, to collect sample of a precious rockthe location of the samples is not known in advance, but it isknown that they tend to be clustered
      • mother ship broadcasts radio signal
        • weakens with distance
      • no map available
      • collaborative
slide69

Mother ship

autonomous vehicle

precious rock

subsumption architecture example mars explorer cont
Subsumption Architecture:example – Mars explorer (cont.)
  • single explorer solution:
    • behaviours / rules

1. if obstacle then change direction

2. if carrying samples and at basethen drop them

3. if carrying samples and not at basethen travel up the gradient field of base’s signal

4. if detect sample then pick it up

5. if true then walk randomly

    • total order relation
        • 1 < 2 < 3 < 4 < 5
subsumption architecture example mars explorer cont1
Subsumption Architecture:example – Mars explorer (cont.)
  • multiple explorer solution ?
    • think about it …
    • if one agent found a cluster of rocks – communicate ?
        • range ?
        • position ?
        • how to deal with such messages ? may be far off …
    • indirect communication:
      • each agent carries “radioactive crumbs”, which can be dropped, picked up and detected by passing robots
      • communication via environment is called stigmergy
subsumption architecture example mars explorer cont2
Subsumption Architecture:example – Mars explorer (cont.)
  • solution inspired by ant foraging behaviour
      • agent creates a “trail” of radioactive crumbs back to the mother ship whenever it finds a rock sample
      • if another agent comes across a trail, it can follow it to the sample cluster
  • refinement:
      • agents following trail to the samples picks up some crumbs to make the trail fainter
      • the trail leading to the empty cluster will finally be removed
subsumption architecture example mars explorer cont3
Subsumption Architecture:example – Mars explorer (cont.)
  • modified rule set
    • if detect an obstacle then change direction
    • if carrying samples and at the basethen drop samples
    • if carrying samples and not at the basethendrop 2 crumbs and travel up gradient
    • if detect a sample then pick up sample
    • if sense crumbs then pick up 1 crumb and travel down gradient
    • if true then move randomly (nothing better to do)
  • order relation: 1 < 2 < 3 < 4 < 5 < 6
  • achieves near optimal performance in many situations
  • cheap solution and robust (the loss of a single agent is not critical).
  • L. Steels argues that (deliberative) agents are “entirely unrealistic” for this problem.
subsumption architecture example mars explorer cont4
Subsumption Architecture:example – Mars explorer (cont.)
  • advantages
    • simple
    • economic
    • computationally tractable
    • robust against failure
  • disadvantages
    • agents act short-term since they use only local information
    • no learning
    • how to engineer such agents ? Difficult if more than 10 rules interact
    • no formal tools to analyse and predict
maes behaviour network for situated agents
Maes’ Behaviour Network for Situated Agents
  • observations
    • “deliberative approach does not work in real dynamic environment”
      • brittleness, inflexibility, slow response time
    • goal-oriented behaviour
      • explicit notion of goal (does not imply planning!)
      • inertia in behaviour
    • agents: multiple behaviours at a time in a given situation
      • parallel
    • behaviours conflict
      • use same mechanism (action) or shared resource
    • agents have “competing” behaviours or actions
maes behaviour network for situated agents cont
Maes’ Behaviour Network for Situated Agents (cont.)
  • an agent is a set of “competence modules” (behaviours)
  • each competence module:
      • pre-condition (situation)
      • post-condition (addition / deletion)
      • activation level (~ relevance in current situation)
  • competence modules are linked
      • according to pre-/post-conditions
      • activation levels change
      • activation levels influence action selection
maes behaviour network for situated agents cont1

B1

a

w

b

y

c

B3

w

r

x

s

y

B2

c

x

d

z

e

Maes’ Behaviour Network for Situated Agents (cont.)
  • behaviour network: a graph
    • nodes: behaviours
    • edges: three types of links
      • successor
      • predecessor
      • conflicter
maes behaviour network for situated agents cont2
Maes’ Behaviour Network for Situated Agents (cont.)
  • which action (behaviour) to perform ?
    • “activation energy” flow
      • global goals  built-in source of motivation
      • environment  situational relevance
      • behaviours  (forward) activation by successors (backward) activation by predecessors
      • inhibition by conflicters
maes behaviour network for situated agents cont3

= energy flow

state

state

backward

put-down

hammer

pick-up

paint brush

forward

backward

backward

forward

put-down

paint brush

paint

inhibitor

goal

Maes’ Behaviour Network for Situated Agents (cont.)
  • current situation
    • agent has hammer in hand
    • the paint brush lies in front of the table
maes behaviour network for situated agents cont4
Maes’ Behaviour Network for Situated Agents (cont.)
  • action selection algorithm
    • do forever
      • add external activation energy from goals & environment
      • spread activation/inhibition among behaviours
        • forward activation via successor links
        • backward activation via predecessor links
        • backward inhibition via conflicter links
      • decay: total activation in system is constant
      • behaviour is selected if
        • it is executable (all preconditions are satisfied)
        • its activation level is over a threshold (theta)
        • its activation is the highest among all executable/activated behaviours
      • if one behaviour executes
        • its activation is set to zero.
        • threshold value is reset to default.
      • ifno behaviour executes, reduce threshold value by x%
        • agent “thinks” for a round and tries again next cycle
maes behaviour network for situated agents cont5
Maes’ Behaviour Network for Situated Agents (cont.)
  • tuning the dynamics
    • action selection emerges from the dynamics of activation spreading
    • tuneable parameters:
      • amount of activation injected by environment
      • amount of activation energy injected by goals
      • the threshold value
maes behaviour network for situated agents cont6
Maes’ Behaviour Network for Situated Agents (cont.)
  • evaluation / characteristics:
      • goal-oriented
      • reactive and fast
      • situation-oriented and opportunistic
      • somewhat inert: biased to ongoing goal/plan
      • goals interact and avoid conflicts
      • robust
free flow architecture
Free-flow architecture
  • esp. suitable for multi-objective behaviour
    • Tyrell / Bryson
    • used in AGV case study
      • proceed to location
      • obstacle avoidance
      • collision avoidance
  • principle
    • tree structure of behaviour / sub-behaviour / primitive actions
    • “energy flow”, from:
      • fixed amount for each action selection
      • perceptions
    • primitive actions: winner-takes-all
free flow architecture1
Free-flow architecture

live

avoid obstacles

find target

a1

a2

a3

a4

a5

a6

a7

a8

a9

a10

free flow architecture2
Free-flow architecture

live

distance to obstacle

need energy

avoid obstacles

find target

perc.y

perc.x

a1

a2

a3

a4

a5

a6

a7

a8

a9

a10

concrete agent architectures3
Concrete agent architectures
  • Deductive reasoning agents
        • 1956 – present
        • “Agents make decisions about what to do via symbol manipulation. Its purest expression, proposes that agents use explicit logical reasoningin order to decide what to do.”
  • Practical reasoning agents
        • 1990 – present
        • “Agent use practical reasoning (towards actions, not towards beliefs) – beliefs / desires / intentions.”
  • Reactive agents
        • 1985 – present
        • “Problems with symbolic reasoning led to a reaction against this — led to the reactive agentsmovement.”
  • Hybrid agents
        • 1989 – present
        • “Hybrid architectures attempt to combine the best of reasoning and reactive architectures.”
4 hybrid agents
4. Hybrid agents
  • best of both worlds ?
    • deliberative
    • reactive
  • obvious approach:
    • build an agent out of two (or more) subsystems:
      • a deliberative one, containing a symbolic world model, which develops plans and makes decisions in the way proposed by symbolic AI
      • a reactive one, which is capable of reacting to events without complex reasoning
  • mostly
    • the reactive component is given precedence over the deliberative one
  • this kind of structuring leads naturally to the idea of a layered architecture
hybrid agents cont
Hybrid agents (cont.)
  • horizontal layering
      • layers are directly connected to sensory input and action output
  • vertical layering
      • sensory input and action output are each dealt with by at most one layer each
slide95

m possible actions suggested by each layer, n layers

m2(n-1) interactions

mn interactions

not fault-tolerant to layer failure

introduces bottleneckin central control system

horizontally layered architecture ferguson touringmachines
Horizontally Layered Architecture:Ferguson – TouringMachines
  • perception and action subsystems interface directly with the agent’s environment
  • three activity producing layers
      • reactive layer
        • +- immediate response to changes in the environmentsituation-action rules
      • planning layer
        • pro-active behaviour
        • uses a library of plan skeletons (schemas)
      • modelling layer
        • represents the various entities in the world
        • predicts conflicts and generates new goals to resolve these conflicts
  • a control subsystem, a set of control rules for deciding which of the layers should have control over the agent
slide97

sensor

input

modelling layer

action

subsystem

perception

subsystem

planning layer

reactive layer

control subsystem

action

output

slide98
reactive layera set of situation-action rulesexample:rule-1: kerb-avoidance if is-in-front (Kerb, Observer) and speed (Observer) > 0 and separation (Kerb, Observer) < KerbThreshHold then change-orientation (KerbAvoidanceAngle)
  • planning layerplans and selects actions in order to achieve agent’s goals
  • modelling layersymbolic repr. of ‘cognitive state’ of other entities in the agent’s environment
  • layers communicate with each other and are embedded in a control framework, which use control rulesexample:censor-rule-1: if entity(obstacle-6) in perception-buffer then remove-sensory-record(layer-R, entity(obstacle-6))
horizontally layered architecture cont
Horizontally Layered Architecture (cont.)
  • advantages
    • conceptual simplicity
      • if we need an agent to exhibit n different types of behaviour, then we implement n different layers!
  • disadvantages
    • competing layers  no guarantee for coherent behaviour ?
      • to avoid this: mediator function
          • makes decisions about which layer has control at a given time
      • the need for such “central control” can be problematic
          • agent designer must potentially consider all possible interactions between layers!
vertically layered architecture m ller interrap
Vertically Layered Architecture:Müller –InteRRaP

 vertically layered, two-pass architecture

cooperation layer

social knowledge

planning knowledge

plan layer

behaviour layer

world model

world interface

perceptual input

action output

vertically layered architecture m ller interrap cont
Vertically Layered Architecture:Müller –InteRRaP (cont.)
  • three layers
      • behaviour based layer reactive behaviour
      • local planning layer for everyday planning
      • co-operative planning for social interactions
  • control between layers:
      • bottom-up activation
        • if layer not competent to deal with situation: pass control to higher layer
      • top-down execution
        • when a higher layer makes use of the facilities provided by a lower layer to achieve one of its goals
  • agent control flow
      • perceptual input arrives at the lowest layer
      • ifreactive layer can deal with this input: it will do sootherwise: bottom-up activation (control passed to local planning layer)
      • iflocal planning layer can handle the situation: it will do so (making use of top-down execution)otherwise: bottom-up activation to pass control to the highest layer
vertically layered architecture cont
Vertically Layered Architecture (cont.)
  • advantage
    • complexity of interactions between layers is reduced
      • n-1 interfaces between n layers

 if each layer is capable of suggesting m actions at most m2 (n-1) interactions

 much simpler than the horizontal case (mn interactions)

  • disadvantages
    • this simplicity comes at the cost of some flexibility
      • control must pass between each different layernot fault tolerant:
          • failures in any one layer can have serious consequences for agent performance
conclusion
Conclusion
  • general:
    • agent architectures: engines for agent behaviour
  • families of agent architectures
    • Deductive reasoning agents
        • “Agents make decisions about what to do via symbol manipulation. Its purest expression, proposes that agents use explicit logical reasoningin order to decide what to do.”
    • Practical reasoning agents
        • “Agent use practical reasoning (towards actions, not towards beliefs) – beliefs / desires / intentions.”
    • Reactive agents
        • “Problems with symbolic reasoning led to a reaction against this — led to the reactive agentsmovement.”
    • Hybrid agents
        • “Hybrid architectures attempt to combine the best of reasoning and reactive architectures.”