Agents
This presentation is the property of its rightful owner.
Sponsored Links
1 / 79

Agents PowerPoint PPT Presentation


  • 125 Views
  • Uploaded on
  • Presentation posted in: General

Agents. What is an agent?. Agenthood = 4 dimensions: autonomy proactiveness embeddedness distributedness. Autonomy. Programs are controlled by user interaction Agents take action without user control monitor: does anyone offer a cheap phone?

Download Presentation

Agents

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Agents

Agents


What is an agent

What is an agent?

  • Agenthood = 4 dimensions:

    • autonomy

    • proactiveness

    • embeddedness

    • distributedness


Autonomy

Autonomy

Programs are controlled by user interaction

Agents take action without user control

  • monitor: does anyone offer a cheap phone?

  • answer requests: do you want to buy a phone?

  • negotiate: negotiate delivery date.

  • can even refuse requests.


Techniques for autonomy

Techniques for Autonomy

Procedures are replaced by behaviors:

map situation  action

Programming agents = defining

  • behaviors

  • control architecture


Result

Result

Agents can act on users' behalf, for example in looking for products or bidding in auctions (e­bay!)

User might not always be available (mobile phones)

Agents can represent users' interest, for example by choosing the best offers.


Proactiveness

Proactiveness

Programs are activated by commands: run ...

Agents take action by themselves:

  • to react to events in their environment

  • to propagate changes or distribute information

  • to take advantage of opportunities in the environment


Techniques for proactive agents

Techniques for proactive agents

Agents must have explicit goals

Goals are linked to plans for achieving them.

Plans are continously reevaluated; new opportunities lead to replanning


Result1

Result

React to ranges of conditions rather than a set of foreseen situations

Gain flexibility in information systems

Proactive agents really act on user's behalf


Embeddedness

Embeddedness

Programs take as long as they take

Agents act under deadlines imposed by the real world:

  • scheduling, planning: plan must be finished before first step

  • trading: must follow the market

  • negotiation: limited response time

    and with limited resources (time, memory, communication)

    Asymptotic complexity analysis insufficient:

    does not give bounds for particular cases!


Techniques for resource bounded reasoning

Techniques for resource­bounded reasoning

1) ''Anytime'' algorithms:

quick and suboptimal solution

iterative refinement

2) reasoning about resource usage:

estimate computation time

 choose suitable computation parameters

3) learning, compilation


Result2

Result

Agent can integrate in the real world:

  • driving a car

  • bidding in an auction

  • interacting with a user or other agents


Distributedness

Distributedness

Programs have common data structures and algorithms

Multi­agent systems model distributed systems; agents are independent entities and may:

  • be programmed by different people.

  • function according to different principles.

  • be added and removed during operation.

  • be unknown to other agents in the system.


Techniques for distributed agent systems

Techniques for distributed agent systems

Agents run on platforms:

  • runtime environment/interfaces

  • communication langugages

  • support for mobility


Result3

Result

Agent system reflects structure of the real system:

  • controlled by their owners

  • local decision making with local information

  • fault tolerant: no central authority


Summary

Summary

Agents = situated software:

  • reacts to environment

  • under real time constraints

  • in distributed environment


From agents to intelligent agents

From Agents to Intelligent Agents

People understand agents to have intentions:

John studied because he wanted to get a diploma.

and also:

The system is asking for a filename because it wants to save the data.

Modeling intentions: reasoning + intelligence!


Situated intelligence

Situated Intelligence

Agent interacts with its environment:

  • observe effects of actions

  • discovery

  • interaction with a user

     particular software architectures


Behaviors

behaviors

world

Behaviors

  • Simplest form of situated intelligence: feedback control

    Thermostat

    Robot following a wall

    Backup every new file


Layers

Layers...

Behaviors should adapt themselves

  • people leaving the house

  • robot hitting the end of the wall

  • backup unit broken down

     Reasoning layer


Agents

Planning/reasoning

behaviors

world


Communication cooperation

Communication/cooperation

Agents need to be instructed

Multiple agents need to cooperate:

  • heating in different rooms

  • robots running into each other

  • several agents backing up the same file

     cooperation layer


Agents

Planning/reasoning

behaviors

world

Other agents

cooperation


Importance of reasoning planning layer

Importance of reasoning/planning layer

Behaviors operate at level of sensors/effectors:

Goto position (35.73,76.14)

Communication is symbolic:

Go to the corner of the room!

 reasoning layer translates between them!


Intelligent agents

Intelligent Agents

Intelligence has (at least) 4 dimensions:

  • rationality: reasoning/planning layer

  • symbolic communication about beliefs, goals, etc.

  • adaptivity

  • learning


Rational agents

Rational Agents

Programs/Algorithms = always do the same thing

rm ­r * wipes out operating system

Rational agents = do the right thing

rm ­r * will keep essential files


Rationality goals

Rationality: goals

File manager:

  • satisfy user's wish

  • keep a backup of all major file versions

  • ...

  • keep one version of all essential operating system files

     action serves to satisfy the goals!


Agents

Complex behavior, intelligence:

adapt behavior to changing conditions

Negotiation/self­interest requires explicit goals!

Learning and using new knowledge requires explicit structures


Techniques for implementing rationality

Techniques for implementing rationality

  • symbolic reasoning

  • planning

  • constraint satisfaction

     knowledge systems!


Communicating agents

Communicating agents

Programs/objects  procedure call:

  • predefined set of possibilities

    Agents  communication language:

  • no predefined set of messages or entities

    Communication is about:

  • beliefs, when passing information

  • goals and plans, when cooperating and negotiating


Agent communication languages

Agent Communication Languages

Language =

  • syntax: predefined set of message types

  • semantics: common ontologies: sets of symbols and meanings

    Examples of languages: KQML, ACL


Needed for

Needed for

Coordination, cooperation and negotiation among agents

Communicate about intentions, self­interest

ACL provides a higher abstraction layer that allows heterogeneous agents to communicate

Add/remove agents in a running multi­agent system


Adaptivity learning

Adaptivity/Learning

Adapt to user:

  • by explicit but highly flexible customization

  • automatically by observing behavior

    Learn from the environment:

  • know objects and operations

  • continuously improve behavior


Techniques for adaptive learning agents

Techniques for adaptive/learning agents

Knowledge systems: explicit representation of goals, operators, plans easy to modify

Automatic adaptation by machine learning or case­based reasoning

Information gathering/machine learning techniques for learning about the environment

Reinforcement learning, genetic algorithms for learning behaviors


Why is this important

Why is this important?

Every user is different  requires different agent behavior

Impractical to program a different agent for everyone

Programmers cannot foresee all aspects of environment

Agent knows its environment better than a programmer


Summary what are intelligent agents

Summary: what are intelligent agents?

Agents are a useful metaphor for computer science:

  • autonomous/proactive: behaviors

  • embedded: real­time

  • distributed

  • intentional: with explicit beliefs, goals, plans

  • communicative: through general ACL

  • self­adaptation/learning


Agents

Smart Agents Collaborative

Learning Agents

Collaborative Interface Agents

Agents

Learn

Cooperate

Autonomous


Agents

Autonomous agents

Computational agents

Biological agents

Robotic agents

Artificial life

agents

Software agents

Entertainment

agents

Viruses

Task-specific

agents


Implementing agents

Implementing agents...

Computers always execute algorithms

  • how can anyone implement agents?

    Agents are a metaphor, implementation is limited:

  • limited sensors  limited adaptativity

  • limited ontologies  limited communication language

  • ...


Technologies for intelligent agents

Technologies for Intelligent Agents

Methods for simple behaviors:

  • behaviors

  • reinforcement learning

  • distributed CSP

    Methods for controlling behaviors:

  • planning

  • case­based reasoning


Technologies for intelligent agents1

Technologies for Intelligent Agents

Formalisms for cooperation:

  • auctions

  • negotiation

  • BDI (Belief-Desire-Intention)

  • ACL/KQML

  • Ontologies


Technologies for intelligent agents2

Technologies for Intelligent Agents

Theories of agent systems:

  • self­interestedness

  • competition/economies

  • behavior of agent systems

    Platforms:

  • auctions, markets (negotiation, contracts)

  • multi­agent platforms

  • mobile agent platforms


Agent languages

Agent languages


Agent communication languages1

Agent Communication Languages

Structure: performatives + content language

KQML

Criteria for content languages


Setting

Setting

Communication among heterogeneous agents:

  • no common data structures

  • no common messages

  • no common ontology

    but common communication language:

  • protocol

  • reference


Structure of an acl

Structure of an ACL

Vocabulary (words): e.g. reference to objects

Messages (sentences): e.g. request for an action

Distributed Algorithms (conversations): e.g. negotiating task sharing


Levels of acl

Levels of ACL

Object sharing (Corba, RPC, RMI, Splice): shared objects, procedures, data structures

Knowledge sharing (KQML, FIPA ACL): shared facts, rules, constraints, procedures and knowledge

Intentional sharing: shared beliefs, plans, goals and intentions

Cultural sharing: shared experiences and strategies


Human communication intentional cultural sharing

Human communication: intentional/cultural sharing

Ideal example of a heterogeneous agent system: human society

See agents as intentional systems:

all actions and communication are motivated by beliefs and intentions

Allows modeling agent behavior in a human­understandable way


Problems with intentional sharing

Problems with intentional sharing

BDI model requires modal logics

Many modal logics pose unrealistic computational requirements:

  • all consequences known

  • combinatorial inference

     BDI model too general as a basis for agent cooperation


A feasible solution knowledge sharing

A feasible solution: knowledge sharing

ACL = 2 components:

  • performative: request, inform, etc.

  • content: a piece of knowledge

    Allows formulating distributed algorithms in a heterogeneous agent society

    Basis: human communication/speech acts


Speech acts

Speech acts

Language =

  • content (e.g., read a book) +

  • speech act (e.g., I want to, I want you to,...)

    Reference:

  • locution: physical utterance

  • illocution: act of conveying intentions

  • perlocutions: actions that occur as a result


Kqml knowledge query and manipulation language

KQML (Knowledge Query and Manipulation Language)

(tell:sender Amazon

:receiver John

:in­reply­to id7.24.97.45391

:ontology lk1

:language Prolog

:content ''price(ISBN348291, 24.95)'‘

)

Represents a single speech act (tell)

which defines the relevant attributes


Kqml keywords

KQML keywords

:sendersender of the message

:receivereceiver of the message

:fromactual origin if forwarded

:toactual destination if to be forwarded

:in­reply­toreference if this is a reply

:reply­withreference is a reply is expected

:languagelanguage used for content

:ontologyontology used for content

:contentcontent itself


Kmql queries

A

A

A

A

B

B

B

B

KMQL queries

Ask-one(P)

Ask-if(P)

tell(P)

sorry

Subscribe( Ask-one(P))

Ask-all(P)

tell(P1)

tell(P1, P2, …)

tell(P2)


Requests achieve unachieve

A

B

Requests: achieve & unachieve

Achieve: make a proposition true

Unachieve: undo the previous achieve

Achieve(P)

Reply()


Facilitation

Facilitation

Advertise(ask(P))

Broker(ask(P))

ask(P))

C

A

B

tell(P))

tell(P))

Advertise(ask(P))

recommend(ask(P))

C

A

B

tell(P))

forward(advertise(ask((P)))

ask(P))

tell(P))


Content in kqml

Content in KQML

Most commonly: Knowledge Interchange Format (KIF)

= predicate calculus in Lisp form

 agents are logical reasoners (EPILOG system available from Stanford University)

Several agent systems use KIF as a basis, e.g. IBM's ABE (Agent Building Environment)


Programming conversations with kif

Programming conversations with KIF

Responding to queries = decide if KIF expression

  • is consistent with current beliefs

  • is entailed by current beliefs

    Contracting tasks (achieve) = defining subgoals in a plan

    Querying information = asking for information for completing a proof

     in general first­order logic, all these tasks are computable, but not decidable


Better formalisms for content

Better formalisms for content

Content languages must make formulating and responding to performatives decidable!

  • restrict content language to be less expressive than predicate calculus

    Examples:

  • description logics (LOOM)

  • constraint content language (CCL)


Programming agent communcication in description logics

Programming agent communcication in Description Logics

Description logic =

reduced form of predicate calculus so that subsumption (an expression consistent with a certain class) can be efficiently decided

  • can efficiently answer and formulate queries

    Usefulness for planning and negotiation less clear


Programming agent communication in ccl

Programming agent communication in CCL

CCL allows expressing constraint satisfaction problems:

variables, domains, relations and constraints

 answering an information query = deciding whether

  • a variable assignment is consistent with constraints (linear time)

  • a combined CSP is consistent (NP­complete)

  • possibly: intermediate notions of consistency/entailment

    Other advantages: more efficient to formulate complex protocols, especially negotiation and coordinated plans


Summary1

Summary

Agent Communication languages are important for building heterogeneous agent systems.

2 levels: performative + content

Issue with content: is there an efficient mechanism for programming agent conversations around them?


Embedded agents

Embedded agents


Implementing embedded agents

Planning/reasoning

behaviors

world

Implementing embedded agents

Behavior: conventional real­time programming

Planning/Reasoning: with limited resources


Reasoning with limited resources

Reasoning with limited resources

Options for reasoning under time constraints:

  • meta­reasoning about resource consumption

  • competitive algorithms

  • anytime algorithms

  • learning and compilation


Reasoning about resource consumption

Reasoning about resource consumption

Estimate computation time  set parameters of method

Problems:

  • times are very diffcult to estimate

  • what to do when no method is fast enough?

     not very promising in practice


Competitive algorithms

Competitive algorithms

Idea: run several methods in parallel

First one to find a solution wins

Examples:

  • breadth­first/depth­first search

  • different problem solving heuristics

    Can be very successful!


Anytime algorithms

Anytime algorithms

Idea: algorithm first finds rough solution, then improves with more computation time.

Example:iterative deepening search

bubble sort


Depth first search

Depth-first search

Not necessarily shortest path

Limited memory requirement


Breadth first search

Breadth-first search

Finds shortest path

Large memory requirement


Depth limited search

Depth­limited search

Usually, breadth­first search requires too much memory to be practical.

Main problem with depth­first search:

  • can follow a dead­end path very far before this is discovered.

     impose a depth limit l:

    never explore nodes at depth > l

    What is the right depth limit?


Iterative deepening

Iterative Deepening

Idea: depth­limited search with increasing limit

Some repeated work, but:

tree with n leaves always has < n intermediate nodes

  • effort of search with limit l always at least double that of search with limit l -1

  • effort of previous searches always less than current

     complexity no more than double of straight DFS


Iterative deepening as an anytime algorithm

Iterative deepening as an anytime algorithm

As time allows, increase the search depth

 Solution improves with more computation time


Learning and compilation

Learning and Compilation

Idea: precompute solutions for the most current problems

 customized agent which works well only in a certain environment

Techniques:

  • reinforcement learning

  • case­based reasoning


Reinforcement learning

Reinforcement learning

Consider game:

  • Finite set of states S1, …, Sn

  • Finite set of actions A(S)= A1, …, Am depending on state

  • Payoffs r(A,S)

    Learn a strategy p, mapping S  A(S) such that averagepayoff is maximized


Optimization criteria

Optimization criteria

Game theory: optimize only current step

But actions also determine future state  future payoffs!

Instead: optimize:

  • finite­horizon: S r(S(t))

  • infinite­horizon: S gt r(S(t))

  • average reward: lim h  inf E( h-1S r(S(t)))


Learning with known model

Learning with known model

Model =

  • state transition function T (S; A; S’ )

  • payoff function R(S; A)

     optimal strategy can be computed in two steps:

  • assign values V (S) to states (off­line)

  • select optimal action (on­line)


Learning with unknown models

Learning with unknown models

Often, model is not known a­priori

  • agent can only deduce model from experience!

    Approaches:

    • learn the model, then the strategy

    • learn both at the same time

      First approach tends to be unstable!


Limitations

Limitations...

Biggest limitation of reinforcement learning:

space of states, actions must be finite, small

Main difficulty: modeling real­world problems to fit the framework!


Summary2

Summary

Challenge for embedded behavior:

limited resources

Two successful methods:

anytime algorithms

reinforcement learning  try it at:

http://www.fe.dis.titech.ac.jp/~gen/robot/robodemo.html


  • Login