agents n.
Skip this Video
Loading SlideShow in 5 Seconds..
Agents PowerPoint Presentation
Download Presentation

Loading in 2 Seconds...

play fullscreen
1 / 79

Agents - PowerPoint PPT Presentation

  • Uploaded on

Agents. What is an agent?. Agenthood = 4 dimensions: autonomy proactiveness embeddedness distributedness. Autonomy. Programs are controlled by user interaction Agents take action without user control monitor: does anyone offer a cheap phone?

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Agents' - rusk

Download Now An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
what is an agent
What is an agent?
  • Agenthood = 4 dimensions:
    • autonomy
    • proactiveness
    • embeddedness
    • distributedness

Programs are controlled by user interaction

Agents take action without user control

  • monitor: does anyone offer a cheap phone?
  • answer requests: do you want to buy a phone?
  • negotiate: negotiate delivery date.
  • can even refuse requests.
techniques for autonomy
Techniques for Autonomy

Procedures are replaced by behaviors:

map situation  action

Programming agents = defining

  • behaviors
  • control architecture

Agents can act on users' behalf, for example in looking for products or bidding in auctions (e­bay!)

User might not always be available (mobile phones)

Agents can represent users' interest, for example by choosing the best offers.


Programs are activated by commands: run ...

Agents take action by themselves:

  • to react to events in their environment
  • to propagate changes or distribute information
  • to take advantage of opportunities in the environment
techniques for proactive agents
Techniques for proactive agents

Agents must have explicit goals

Goals are linked to plans for achieving them.

Plans are continously reevaluated; new opportunities lead to replanning


React to ranges of conditions rather than a set of foreseen situations

Gain flexibility in information systems

Proactive agents really act on user's behalf


Programs take as long as they take

Agents act under deadlines imposed by the real world:

  • scheduling, planning: plan must be finished before first step
  • trading: must follow the market
  • negotiation: limited response time

and with limited resources (time, memory, communication)

Asymptotic complexity analysis insufficient:

does not give bounds for particular cases!

techniques for resource bounded reasoning
Techniques for resource­bounded reasoning

1) ''Anytime'' algorithms:

quick and suboptimal solution

iterative refinement

2) reasoning about resource usage:

estimate computation time

 choose suitable computation parameters

3) learning, compilation


Agent can integrate in the real world:

  • driving a car
  • bidding in an auction
  • interacting with a user or other agents

Programs have common data structures and algorithms

Multi­agent systems model distributed systems; agents are independent entities and may:

  • be programmed by different people.
  • function according to different principles.
  • be added and removed during operation.
  • be unknown to other agents in the system.
techniques for distributed agent systems
Techniques for distributed agent systems

Agents run on platforms:

  • runtime environment/interfaces
  • communication langugages
  • support for mobility

Agent system reflects structure of the real system:

  • controlled by their owners
  • local decision making with local information
  • fault tolerant: no central authority

Agents = situated software:

  • reacts to environment
  • under real time constraints
  • in distributed environment
from agents to intelligent agents
From Agents to Intelligent Agents

People understand agents to have intentions:

John studied because he wanted to get a diploma.

and also:

The system is asking for a filename because it wants to save the data.

Modeling intentions: reasoning + intelligence!

situated intelligence
Situated Intelligence

Agent interacts with its environment:

  • observe effects of actions
  • discovery
  • interaction with a user

 particular software architectures




  • Simplest form of situated intelligence: feedback control


Robot following a wall

Backup every new file


Behaviors should adapt themselves

  • people leaving the house
  • robot hitting the end of the wall
  • backup unit broken down

 Reasoning layer





communication cooperation

Agents need to be instructed

Multiple agents need to cooperate:

  • heating in different rooms
  • robots running into each other
  • several agents backing up the same file

 cooperation layer





Other agents


importance of reasoning planning layer
Importance of reasoning/planning layer

Behaviors operate at level of sensors/effectors:

Goto position (35.73,76.14)

Communication is symbolic:

Go to the corner of the room!

 reasoning layer translates between them!

intelligent agents
Intelligent Agents

Intelligence has (at least) 4 dimensions:

  • rationality: reasoning/planning layer
  • symbolic communication about beliefs, goals, etc.
  • adaptivity
  • learning
rational agents
Rational Agents

Programs/Algorithms = always do the same thing

rm ­r * wipes out operating system

Rational agents = do the right thing

rm ­r * will keep essential files

rationality goals
Rationality: goals

File manager:

  • satisfy user's wish
  • keep a backup of all major file versions
  • ...
  • keep one version of all essential operating system files

 action serves to satisfy the goals!

Complex behavior, intelligence:

adapt behavior to changing conditions

Negotiation/self­interest requires explicit goals!

Learning and using new knowledge requires explicit structures

techniques for implementing rationality
Techniques for implementing rationality
  • symbolic reasoning
  • planning
  • constraint satisfaction

 knowledge systems!

communicating agents
Communicating agents

Programs/objects  procedure call:

  • predefined set of possibilities

Agents  communication language:

  • no predefined set of messages or entities

Communication is about:

  • beliefs, when passing information
  • goals and plans, when cooperating and negotiating
agent communication languages
Agent Communication Languages

Language =

  • syntax: predefined set of message types
  • semantics: common ontologies: sets of symbols and meanings

Examples of languages: KQML, ACL

needed for
Needed for

Coordination, cooperation and negotiation among agents

Communicate about intentions, self­interest

ACL provides a higher abstraction layer that allows heterogeneous agents to communicate

Add/remove agents in a running multi­agent system

adaptivity learning

Adapt to user:

  • by explicit but highly flexible customization
  • automatically by observing behavior

Learn from the environment:

  • know objects and operations
  • continuously improve behavior
techniques for adaptive learning agents
Techniques for adaptive/learning agents

Knowledge systems: explicit representation of goals, operators, plans easy to modify

Automatic adaptation by machine learning or case­based reasoning

Information gathering/machine learning techniques for learning about the environment

Reinforcement learning, genetic algorithms for learning behaviors

why is this important
Why is this important?

Every user is different  requires different agent behavior

Impractical to program a different agent for everyone

Programmers cannot foresee all aspects of environment

Agent knows its environment better than a programmer

summary what are intelligent agents
Summary: what are intelligent agents?

Agents are a useful metaphor for computer science:

  • autonomous/proactive: behaviors
  • embedded: real­time
  • distributed
  • intentional: with explicit beliefs, goals, plans
  • communicative: through general ACL
  • self­adaptation/learning
Smart Agents Collaborative

Learning Agents

Collaborative Interface Agents





Autonomous agents

Computational agents

Biological agents

Robotic agents

Artificial life


Software agents






implementing agents
Implementing agents...

Computers always execute algorithms

  • how can anyone implement agents?

Agents are a metaphor, implementation is limited:

  • limited sensors  limited adaptativity
  • limited ontologies  limited communication language
  • ...
technologies for intelligent agents
Technologies for Intelligent Agents

Methods for simple behaviors:

  • behaviors
  • reinforcement learning
  • distributed CSP

Methods for controlling behaviors:

  • planning
  • case­based reasoning
technologies for intelligent agents1
Technologies for Intelligent Agents

Formalisms for cooperation:

  • auctions
  • negotiation
  • BDI (Belief-Desire-Intention)
  • Ontologies
technologies for intelligent agents2
Technologies for Intelligent Agents

Theories of agent systems:

  • self­interestedness
  • competition/economies
  • behavior of agent systems


  • auctions, markets (negotiation, contracts)
  • multi­agent platforms
  • mobile agent platforms
agent communication languages1
Agent Communication Languages

Structure: performatives + content language


Criteria for content languages


Communication among heterogeneous agents:

  • no common data structures
  • no common messages
  • no common ontology

but common communication language:

  • protocol
  • reference
structure of an acl
Structure of an ACL

Vocabulary (words): e.g. reference to objects

Messages (sentences): e.g. request for an action

Distributed Algorithms (conversations): e.g. negotiating task sharing

levels of acl
Levels of ACL

Object sharing (Corba, RPC, RMI, Splice): shared objects, procedures, data structures

Knowledge sharing (KQML, FIPA ACL): shared facts, rules, constraints, procedures and knowledge

Intentional sharing: shared beliefs, plans, goals and intentions

Cultural sharing: shared experiences and strategies

human communication intentional cultural sharing
Human communication: intentional/cultural sharing

Ideal example of a heterogeneous agent system: human society

See agents as intentional systems:

all actions and communication are motivated by beliefs and intentions

Allows modeling agent behavior in a human­understandable way

problems with intentional sharing
Problems with intentional sharing

BDI model requires modal logics

Many modal logics pose unrealistic computational requirements:

  • all consequences known
  • combinatorial inference

 BDI model too general as a basis for agent cooperation

a feasible solution knowledge sharing
A feasible solution: knowledge sharing

ACL = 2 components:

  • performative: request, inform, etc.
  • content: a piece of knowledge

Allows formulating distributed algorithms in a heterogeneous agent society

Basis: human communication/speech acts

speech acts
Speech acts

Language =

  • content (e.g., read a book) +
  • speech act (e.g., I want to, I want you to,...)


  • locution: physical utterance
  • illocution: act of conveying intentions
  • perlocutions: actions that occur as a result
kqml knowledge query and manipulation language
KQML (Knowledge Query and Manipulation Language)

(tell :sender Amazon

:receiver John

:in­reply­to id7.24.97.45391

:ontology lk1

:language Prolog

:content ''price(ISBN348291, 24.95)'‘


Represents a single speech act (tell)

which defines the relevant attributes

kqml keywords
KQML keywords

:sender sender of the message

:receive receiver of the message

:from actual origin if forwarded

:to actual destination if to be forwarded

:in­reply­to reference if this is a reply

:reply­with reference is a reply is expected

:language language used for content

:ontology ontology used for content

:content content itself

kmql queries









KMQL queries





Subscribe( Ask-one(P))



tell(P1, P2, …)


requests achieve unachieve



Requests: achieve & unachieve

Achieve: make a proposition true

Unachieve: undo the previous achieve





















content in kqml
Content in KQML

Most commonly: Knowledge Interchange Format (KIF)

= predicate calculus in Lisp form

 agents are logical reasoners (EPILOG system available from Stanford University)

Several agent systems use KIF as a basis, e.g. IBM's ABE (Agent Building Environment)

programming conversations with kif
Programming conversations with KIF

Responding to queries = decide if KIF expression

  • is consistent with current beliefs
  • is entailed by current beliefs

Contracting tasks (achieve) = defining subgoals in a plan

Querying information = asking for information for completing a proof

 in general first­order logic, all these tasks are computable, but not decidable

better formalisms for content
Better formalisms for content

Content languages must make formulating and responding to performatives decidable!

  • restrict content language to be less expressive than predicate calculus


  • description logics (LOOM)
  • constraint content language (CCL)
programming agent communcication in description logics
Programming agent communcication in Description Logics

Description logic =

reduced form of predicate calculus so that subsumption (an expression consistent with a certain class) can be efficiently decided

  • can efficiently answer and formulate queries

Usefulness for planning and negotiation less clear

programming agent communication in ccl
Programming agent communication in CCL

CCL allows expressing constraint satisfaction problems:

variables, domains, relations and constraints

 answering an information query = deciding whether

  • a variable assignment is consistent with constraints (linear time)
  • a combined CSP is consistent (NP­complete)
  • possibly: intermediate notions of consistency/entailment

Other advantages: more efficient to formulate complex protocols, especially negotiation and coordinated plans


Agent Communication languages are important for building heterogeneous agent systems.

2 levels: performative + content

Issue with content: is there an efficient mechanism for programming agent conversations around them?

implementing embedded agents




Implementing embedded agents

Behavior: conventional real­time programming

Planning/Reasoning: with limited resources

reasoning with limited resources
Reasoning with limited resources

Options for reasoning under time constraints:

  • meta­reasoning about resource consumption
  • competitive algorithms
  • anytime algorithms
  • learning and compilation
reasoning about resource consumption
Reasoning about resource consumption

Estimate computation time  set parameters of method


  • times are very diffcult to estimate
  • what to do when no method is fast enough?

 not very promising in practice

competitive algorithms
Competitive algorithms

Idea: run several methods in parallel

First one to find a solution wins


  • breadth­first/depth­first search
  • different problem solving heuristics

Can be very successful!

anytime algorithms
Anytime algorithms

Idea: algorithm first finds rough solution, then improves with more computation time.

Example: iterative deepening search

bubble sort

depth first search
Depth-first search

Not necessarily shortest path

Limited memory requirement

breadth first search
Breadth-first search

Finds shortest path

Large memory requirement

depth limited search
Depth­limited search

Usually, breadth­first search requires too much memory to be practical.

Main problem with depth­first search:

  • can follow a dead­end path very far before this is discovered.

 impose a depth limit l:

never explore nodes at depth > l

What is the right depth limit?

iterative deepening
Iterative Deepening

Idea: depth­limited search with increasing limit

Some repeated work, but:

tree with n leaves always has < n intermediate nodes

  • effort of search with limit l always at least double that of search with limit l -1
  • effort of previous searches always less than current

 complexity no more than double of straight DFS

iterative deepening as an anytime algorithm
Iterative deepening as an anytime algorithm

As time allows, increase the search depth

 Solution improves with more computation time

learning and compilation
Learning and Compilation

Idea: precompute solutions for the most current problems

 customized agent which works well only in a certain environment


  • reinforcement learning
  • case­based reasoning
reinforcement learning
Reinforcement learning

Consider game:

  • Finite set of states S1, …, Sn
  • Finite set of actions A(S)= A1, …, Am depending on state
  • Payoffs r(A,S)

Learn a strategy p, mapping S  A(S) such that averagepayoff is maximized

optimization criteria
Optimization criteria

Game theory: optimize only current step

But actions also determine future state  future payoffs!

Instead: optimize:

  • finite­horizon: S r(S(t))
  • infinite­horizon: S gt r(S(t))
  • average reward: lim h  inf E( h-1S r(S(t)))
learning with known model
Learning with known model

Model =

  • state transition function T (S; A; S’ )
  • payoff function R(S; A)

 optimal strategy can be computed in two steps:

  • assign values V (S) to states (off­line)
  • select optimal action (on­line)
learning with unknown models
Learning with unknown models

Often, model is not known a­priori

  • agent can only deduce model from experience!


    • learn the model, then the strategy
    • learn both at the same time

First approach tends to be unstable!


Biggest limitation of reinforcement learning:

space of states, actions must be finite, small

Main difficulty: modeling real­world problems to fit the framework!


Challenge for embedded behavior:

limited resources

Two successful methods:

anytime algorithms

reinforcement learning  try it at: