This presentation is the property of its rightful owner.
Sponsored Links
1 / 79

Agents PowerPoint PPT Presentation

  • Uploaded on
  • Presentation posted in: General

Agents. What is an agent?. Agenthood = 4 dimensions: autonomy proactiveness embeddedness distributedness. Autonomy. Programs are controlled by user interaction Agents take action without user control monitor: does anyone offer a cheap phone?

Download Presentation


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript



What is an agent

What is an agent?

  • Agenthood = 4 dimensions:

    • autonomy

    • proactiveness

    • embeddedness

    • distributedness



Programs are controlled by user interaction

Agents take action without user control

  • monitor: does anyone offer a cheap phone?

  • answer requests: do you want to buy a phone?

  • negotiate: negotiate delivery date.

  • can even refuse requests.

Techniques for autonomy

Techniques for Autonomy

Procedures are replaced by behaviors:

map situation  action

Programming agents = defining

  • behaviors

  • control architecture



Agents can act on users' behalf, for example in looking for products or bidding in auctions (e­bay!)

User might not always be available (mobile phones)

Agents can represent users' interest, for example by choosing the best offers.



Programs are activated by commands: run ...

Agents take action by themselves:

  • to react to events in their environment

  • to propagate changes or distribute information

  • to take advantage of opportunities in the environment

Techniques for proactive agents

Techniques for proactive agents

Agents must have explicit goals

Goals are linked to plans for achieving them.

Plans are continously reevaluated; new opportunities lead to replanning



React to ranges of conditions rather than a set of foreseen situations

Gain flexibility in information systems

Proactive agents really act on user's behalf



Programs take as long as they take

Agents act under deadlines imposed by the real world:

  • scheduling, planning: plan must be finished before first step

  • trading: must follow the market

  • negotiation: limited response time

    and with limited resources (time, memory, communication)

    Asymptotic complexity analysis insufficient:

    does not give bounds for particular cases!

Techniques for resource bounded reasoning

Techniques for resource­bounded reasoning

1) ''Anytime'' algorithms:

quick and suboptimal solution

iterative refinement

2) reasoning about resource usage:

estimate computation time

 choose suitable computation parameters

3) learning, compilation



Agent can integrate in the real world:

  • driving a car

  • bidding in an auction

  • interacting with a user or other agents



Programs have common data structures and algorithms

Multi­agent systems model distributed systems; agents are independent entities and may:

  • be programmed by different people.

  • function according to different principles.

  • be added and removed during operation.

  • be unknown to other agents in the system.

Techniques for distributed agent systems

Techniques for distributed agent systems

Agents run on platforms:

  • runtime environment/interfaces

  • communication langugages

  • support for mobility



Agent system reflects structure of the real system:

  • controlled by their owners

  • local decision making with local information

  • fault tolerant: no central authority



Agents = situated software:

  • reacts to environment

  • under real time constraints

  • in distributed environment

From agents to intelligent agents

From Agents to Intelligent Agents

People understand agents to have intentions:

John studied because he wanted to get a diploma.

and also:

The system is asking for a filename because it wants to save the data.

Modeling intentions: reasoning + intelligence!

Situated intelligence

Situated Intelligence

Agent interacts with its environment:

  • observe effects of actions

  • discovery

  • interaction with a user

     particular software architectures





  • Simplest form of situated intelligence: feedback control


    Robot following a wall

    Backup every new file



Behaviors should adapt themselves

  • people leaving the house

  • robot hitting the end of the wall

  • backup unit broken down

     Reasoning layer





Communication cooperation


Agents need to be instructed

Multiple agents need to cooperate:

  • heating in different rooms

  • robots running into each other

  • several agents backing up the same file

     cooperation layer





Other agents


Importance of reasoning planning layer

Importance of reasoning/planning layer

Behaviors operate at level of sensors/effectors:

Goto position (35.73,76.14)

Communication is symbolic:

Go to the corner of the room!

 reasoning layer translates between them!

Intelligent agents

Intelligent Agents

Intelligence has (at least) 4 dimensions:

  • rationality: reasoning/planning layer

  • symbolic communication about beliefs, goals, etc.

  • adaptivity

  • learning

Rational agents

Rational Agents

Programs/Algorithms = always do the same thing

rm ­r * wipes out operating system

Rational agents = do the right thing

rm ­r * will keep essential files

Rationality goals

Rationality: goals

File manager:

  • satisfy user's wish

  • keep a backup of all major file versions

  • ...

  • keep one version of all essential operating system files

     action serves to satisfy the goals!


Complex behavior, intelligence:

adapt behavior to changing conditions

Negotiation/self­interest requires explicit goals!

Learning and using new knowledge requires explicit structures

Techniques for implementing rationality

Techniques for implementing rationality

  • symbolic reasoning

  • planning

  • constraint satisfaction

     knowledge systems!

Communicating agents

Communicating agents

Programs/objects  procedure call:

  • predefined set of possibilities

    Agents  communication language:

  • no predefined set of messages or entities

    Communication is about:

  • beliefs, when passing information

  • goals and plans, when cooperating and negotiating

Agent communication languages

Agent Communication Languages

Language =

  • syntax: predefined set of message types

  • semantics: common ontologies: sets of symbols and meanings

    Examples of languages: KQML, ACL

Needed for

Needed for

Coordination, cooperation and negotiation among agents

Communicate about intentions, self­interest

ACL provides a higher abstraction layer that allows heterogeneous agents to communicate

Add/remove agents in a running multi­agent system

Adaptivity learning


Adapt to user:

  • by explicit but highly flexible customization

  • automatically by observing behavior

    Learn from the environment:

  • know objects and operations

  • continuously improve behavior

Techniques for adaptive learning agents

Techniques for adaptive/learning agents

Knowledge systems: explicit representation of goals, operators, plans easy to modify

Automatic adaptation by machine learning or case­based reasoning

Information gathering/machine learning techniques for learning about the environment

Reinforcement learning, genetic algorithms for learning behaviors

Why is this important

Why is this important?

Every user is different  requires different agent behavior

Impractical to program a different agent for everyone

Programmers cannot foresee all aspects of environment

Agent knows its environment better than a programmer

Summary what are intelligent agents

Summary: what are intelligent agents?

Agents are a useful metaphor for computer science:

  • autonomous/proactive: behaviors

  • embedded: real­time

  • distributed

  • intentional: with explicit beliefs, goals, plans

  • communicative: through general ACL

  • self­adaptation/learning


Smart Agents Collaborative

Learning Agents

Collaborative Interface Agents






Autonomous agents

Computational agents

Biological agents

Robotic agents

Artificial life


Software agents






Implementing agents

Implementing agents...

Computers always execute algorithms

  • how can anyone implement agents?

    Agents are a metaphor, implementation is limited:

  • limited sensors  limited adaptativity

  • limited ontologies  limited communication language

  • ...

Technologies for intelligent agents

Technologies for Intelligent Agents

Methods for simple behaviors:

  • behaviors

  • reinforcement learning

  • distributed CSP

    Methods for controlling behaviors:

  • planning

  • case­based reasoning

Technologies for intelligent agents1

Technologies for Intelligent Agents

Formalisms for cooperation:

  • auctions

  • negotiation

  • BDI (Belief-Desire-Intention)


  • Ontologies

Technologies for intelligent agents2

Technologies for Intelligent Agents

Theories of agent systems:

  • self­interestedness

  • competition/economies

  • behavior of agent systems


  • auctions, markets (negotiation, contracts)

  • multi­agent platforms

  • mobile agent platforms

Agent languages

Agent languages

Agent communication languages1

Agent Communication Languages

Structure: performatives + content language


Criteria for content languages



Communication among heterogeneous agents:

  • no common data structures

  • no common messages

  • no common ontology

    but common communication language:

  • protocol

  • reference

Structure of an acl

Structure of an ACL

Vocabulary (words): e.g. reference to objects

Messages (sentences): e.g. request for an action

Distributed Algorithms (conversations): e.g. negotiating task sharing

Levels of acl

Levels of ACL

Object sharing (Corba, RPC, RMI, Splice): shared objects, procedures, data structures

Knowledge sharing (KQML, FIPA ACL): shared facts, rules, constraints, procedures and knowledge

Intentional sharing: shared beliefs, plans, goals and intentions

Cultural sharing: shared experiences and strategies

Human communication intentional cultural sharing

Human communication: intentional/cultural sharing

Ideal example of a heterogeneous agent system: human society

See agents as intentional systems:

all actions and communication are motivated by beliefs and intentions

Allows modeling agent behavior in a human­understandable way

Problems with intentional sharing

Problems with intentional sharing

BDI model requires modal logics

Many modal logics pose unrealistic computational requirements:

  • all consequences known

  • combinatorial inference

     BDI model too general as a basis for agent cooperation

A feasible solution knowledge sharing

A feasible solution: knowledge sharing

ACL = 2 components:

  • performative: request, inform, etc.

  • content: a piece of knowledge

    Allows formulating distributed algorithms in a heterogeneous agent society

    Basis: human communication/speech acts

Speech acts

Speech acts

Language =

  • content (e.g., read a book) +

  • speech act (e.g., I want to, I want you to,...)


  • locution: physical utterance

  • illocution: act of conveying intentions

  • perlocutions: actions that occur as a result

Kqml knowledge query and manipulation language

KQML (Knowledge Query and Manipulation Language)

(tell:sender Amazon

:receiver John

:in­reply­to id7.24.97.45391

:ontology lk1

:language Prolog

:content ''price(ISBN348291, 24.95)'‘


Represents a single speech act (tell)

which defines the relevant attributes

Kqml keywords

KQML keywords

:sendersender of the message

:receivereceiver of the message

:fromactual origin if forwarded

:toactual destination if to be forwarded

:in­reply­toreference if this is a reply

:reply­withreference is a reply is expected

:languagelanguage used for content

:ontologyontology used for content

:contentcontent itself

Kmql queries









KMQL queries





Subscribe( Ask-one(P))



tell(P1, P2, …)


Requests achieve unachieve



Requests: achieve & unachieve

Achieve: make a proposition true

Unachieve: undo the previous achieve






















Content in kqml

Content in KQML

Most commonly: Knowledge Interchange Format (KIF)

= predicate calculus in Lisp form

 agents are logical reasoners (EPILOG system available from Stanford University)

Several agent systems use KIF as a basis, e.g. IBM's ABE (Agent Building Environment)

Programming conversations with kif

Programming conversations with KIF

Responding to queries = decide if KIF expression

  • is consistent with current beliefs

  • is entailed by current beliefs

    Contracting tasks (achieve) = defining subgoals in a plan

    Querying information = asking for information for completing a proof

     in general first­order logic, all these tasks are computable, but not decidable

Better formalisms for content

Better formalisms for content

Content languages must make formulating and responding to performatives decidable!

  • restrict content language to be less expressive than predicate calculus


  • description logics (LOOM)

  • constraint content language (CCL)

Programming agent communcication in description logics

Programming agent communcication in Description Logics

Description logic =

reduced form of predicate calculus so that subsumption (an expression consistent with a certain class) can be efficiently decided

  • can efficiently answer and formulate queries

    Usefulness for planning and negotiation less clear

Programming agent communication in ccl

Programming agent communication in CCL

CCL allows expressing constraint satisfaction problems:

variables, domains, relations and constraints

 answering an information query = deciding whether

  • a variable assignment is consistent with constraints (linear time)

  • a combined CSP is consistent (NP­complete)

  • possibly: intermediate notions of consistency/entailment

    Other advantages: more efficient to formulate complex protocols, especially negotiation and coordinated plans



Agent Communication languages are important for building heterogeneous agent systems.

2 levels: performative + content

Issue with content: is there an efficient mechanism for programming agent conversations around them?

Embedded agents

Embedded agents

Implementing embedded agents




Implementing embedded agents

Behavior: conventional real­time programming

Planning/Reasoning: with limited resources

Reasoning with limited resources

Reasoning with limited resources

Options for reasoning under time constraints:

  • meta­reasoning about resource consumption

  • competitive algorithms

  • anytime algorithms

  • learning and compilation

Reasoning about resource consumption

Reasoning about resource consumption

Estimate computation time  set parameters of method


  • times are very diffcult to estimate

  • what to do when no method is fast enough?

     not very promising in practice

Competitive algorithms

Competitive algorithms

Idea: run several methods in parallel

First one to find a solution wins


  • breadth­first/depth­first search

  • different problem solving heuristics

    Can be very successful!

Anytime algorithms

Anytime algorithms

Idea: algorithm first finds rough solution, then improves with more computation time.

Example:iterative deepening search

bubble sort

Depth first search

Depth-first search

Not necessarily shortest path

Limited memory requirement

Breadth first search

Breadth-first search

Finds shortest path

Large memory requirement

Depth limited search

Depth­limited search

Usually, breadth­first search requires too much memory to be practical.

Main problem with depth­first search:

  • can follow a dead­end path very far before this is discovered.

     impose a depth limit l:

    never explore nodes at depth > l

    What is the right depth limit?

Iterative deepening

Iterative Deepening

Idea: depth­limited search with increasing limit

Some repeated work, but:

tree with n leaves always has < n intermediate nodes

  • effort of search with limit l always at least double that of search with limit l -1

  • effort of previous searches always less than current

     complexity no more than double of straight DFS

Iterative deepening as an anytime algorithm

Iterative deepening as an anytime algorithm

As time allows, increase the search depth

 Solution improves with more computation time

Learning and compilation

Learning and Compilation

Idea: precompute solutions for the most current problems

 customized agent which works well only in a certain environment


  • reinforcement learning

  • case­based reasoning

Reinforcement learning

Reinforcement learning

Consider game:

  • Finite set of states S1, …, Sn

  • Finite set of actions A(S)= A1, …, Am depending on state

  • Payoffs r(A,S)

    Learn a strategy p, mapping S  A(S) such that averagepayoff is maximized

Optimization criteria

Optimization criteria

Game theory: optimize only current step

But actions also determine future state  future payoffs!

Instead: optimize:

  • finite­horizon: S r(S(t))

  • infinite­horizon: S gt r(S(t))

  • average reward: lim h  inf E( h-1S r(S(t)))

Learning with known model

Learning with known model

Model =

  • state transition function T (S; A; S’ )

  • payoff function R(S; A)

     optimal strategy can be computed in two steps:

  • assign values V (S) to states (off­line)

  • select optimal action (on­line)

Learning with unknown models

Learning with unknown models

Often, model is not known a­priori

  • agent can only deduce model from experience!


    • learn the model, then the strategy

    • learn both at the same time

      First approach tends to be unstable!



Biggest limitation of reinforcement learning:

space of states, actions must be finite, small

Main difficulty: modeling real­world problems to fit the framework!



Challenge for embedded behavior:

limited resources

Two successful methods:

anytime algorithms

reinforcement learning  try it at:

  • Login