Agents
Download
1 / 79

Agents - PowerPoint PPT Presentation


  • 154 Views
  • Uploaded on

Agents. What is an agent?. Agenthood = 4 dimensions: autonomy proactiveness embeddedness distributedness. Autonomy. Programs are controlled by user interaction Agents take action without user control monitor: does anyone offer a cheap phone?

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Agents' - rusk


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

What is an agent
What is an agent?

  • Agenthood = 4 dimensions:

    • autonomy

    • proactiveness

    • embeddedness

    • distributedness


Autonomy
Autonomy

Programs are controlled by user interaction

Agents take action without user control

  • monitor: does anyone offer a cheap phone?

  • answer requests: do you want to buy a phone?

  • negotiate: negotiate delivery date.

  • can even refuse requests.


Techniques for autonomy
Techniques for Autonomy

Procedures are replaced by behaviors:

map situation  action

Programming agents = defining

  • behaviors

  • control architecture


Result
Result

Agents can act on users' behalf, for example in looking for products or bidding in auctions (e­bay!)

User might not always be available (mobile phones)

Agents can represent users' interest, for example by choosing the best offers.


Proactiveness
Proactiveness

Programs are activated by commands: run ...

Agents take action by themselves:

  • to react to events in their environment

  • to propagate changes or distribute information

  • to take advantage of opportunities in the environment


Techniques for proactive agents
Techniques for proactive agents

Agents must have explicit goals

Goals are linked to plans for achieving them.

Plans are continously reevaluated; new opportunities lead to replanning


Result1
Result

React to ranges of conditions rather than a set of foreseen situations

Gain flexibility in information systems

Proactive agents really act on user's behalf


Embeddedness
Embeddedness

Programs take as long as they take

Agents act under deadlines imposed by the real world:

  • scheduling, planning: plan must be finished before first step

  • trading: must follow the market

  • negotiation: limited response time

    and with limited resources (time, memory, communication)

    Asymptotic complexity analysis insufficient:

    does not give bounds for particular cases!


Techniques for resource bounded reasoning
Techniques for resource­bounded reasoning

1) ''Anytime'' algorithms:

quick and suboptimal solution

iterative refinement

2) reasoning about resource usage:

estimate computation time

 choose suitable computation parameters

3) learning, compilation


Result2
Result

Agent can integrate in the real world:

  • driving a car

  • bidding in an auction

  • interacting with a user or other agents


Distributedness
Distributedness

Programs have common data structures and algorithms

Multi­agent systems model distributed systems; agents are independent entities and may:

  • be programmed by different people.

  • function according to different principles.

  • be added and removed during operation.

  • be unknown to other agents in the system.


Techniques for distributed agent systems
Techniques for distributed agent systems

Agents run on platforms:

  • runtime environment/interfaces

  • communication langugages

  • support for mobility


Result3
Result

Agent system reflects structure of the real system:

  • controlled by their owners

  • local decision making with local information

  • fault tolerant: no central authority


Summary
Summary

Agents = situated software:

  • reacts to environment

  • under real time constraints

  • in distributed environment


From agents to intelligent agents
From Agents to Intelligent Agents

People understand agents to have intentions:

John studied because he wanted to get a diploma.

and also:

The system is asking for a filename because it wants to save the data.

Modeling intentions: reasoning + intelligence!


Situated intelligence
Situated Intelligence

Agent interacts with its environment:

  • observe effects of actions

  • discovery

  • interaction with a user

     particular software architectures


Behaviors

behaviors

world

Behaviors

  • Simplest form of situated intelligence: feedback control

    Thermostat

    Robot following a wall

    Backup every new file


Layers
Layers...

Behaviors should adapt themselves

  • people leaving the house

  • robot hitting the end of the wall

  • backup unit broken down

     Reasoning layer


Planning/reasoning

behaviors

world


Communication cooperation
Communication/cooperation

Agents need to be instructed

Multiple agents need to cooperate:

  • heating in different rooms

  • robots running into each other

  • several agents backing up the same file

     cooperation layer


Planning/reasoning

behaviors

world

Other agents

cooperation


Importance of reasoning planning layer
Importance of reasoning/planning layer

Behaviors operate at level of sensors/effectors:

Goto position (35.73,76.14)

Communication is symbolic:

Go to the corner of the room!

 reasoning layer translates between them!


Intelligent agents
Intelligent Agents

Intelligence has (at least) 4 dimensions:

  • rationality: reasoning/planning layer

  • symbolic communication about beliefs, goals, etc.

  • adaptivity

  • learning


Rational agents
Rational Agents

Programs/Algorithms = always do the same thing

rm ­r * wipes out operating system

Rational agents = do the right thing

rm ­r * will keep essential files


Rationality goals
Rationality: goals

File manager:

  • satisfy user's wish

  • keep a backup of all major file versions

  • ...

  • keep one version of all essential operating system files

     action serves to satisfy the goals!


Complex behavior, intelligence:

adapt behavior to changing conditions

Negotiation/self­interest requires explicit goals!

Learning and using new knowledge requires explicit structures


Techniques for implementing rationality
Techniques for implementing rationality

  • symbolic reasoning

  • planning

  • constraint satisfaction

     knowledge systems!


Communicating agents
Communicating agents

Programs/objects  procedure call:

  • predefined set of possibilities

    Agents  communication language:

  • no predefined set of messages or entities

    Communication is about:

  • beliefs, when passing information

  • goals and plans, when cooperating and negotiating


Agent communication languages
Agent Communication Languages

Language =

  • syntax: predefined set of message types

  • semantics: common ontologies: sets of symbols and meanings

    Examples of languages: KQML, ACL


Needed for
Needed for

Coordination, cooperation and negotiation among agents

Communicate about intentions, self­interest

ACL provides a higher abstraction layer that allows heterogeneous agents to communicate

Add/remove agents in a running multi­agent system


Adaptivity learning
Adaptivity/Learning

Adapt to user:

  • by explicit but highly flexible customization

  • automatically by observing behavior

    Learn from the environment:

  • know objects and operations

  • continuously improve behavior


Techniques for adaptive learning agents
Techniques for adaptive/learning agents

Knowledge systems: explicit representation of goals, operators, plans easy to modify

Automatic adaptation by machine learning or case­based reasoning

Information gathering/machine learning techniques for learning about the environment

Reinforcement learning, genetic algorithms for learning behaviors


Why is this important
Why is this important?

Every user is different  requires different agent behavior

Impractical to program a different agent for everyone

Programmers cannot foresee all aspects of environment

Agent knows its environment better than a programmer


Summary what are intelligent agents
Summary: what are intelligent agents?

Agents are a useful metaphor for computer science:

  • autonomous/proactive: behaviors

  • embedded: real­time

  • distributed

  • intentional: with explicit beliefs, goals, plans

  • communicative: through general ACL

  • self­adaptation/learning


Smart Agents Collaborative

Learning Agents

Collaborative Interface Agents

Agents

Learn

Cooperate

Autonomous


Autonomous agents Collaborative

Computational agents

Biological agents

Robotic agents

Artificial life

agents

Software agents

Entertainment

agents

Viruses

Task-specific

agents


Implementing agents
Implementing agents... Collaborative

Computers always execute algorithms

  • how can anyone implement agents?

    Agents are a metaphor, implementation is limited:

  • limited sensors  limited adaptativity

  • limited ontologies  limited communication language

  • ...


Technologies for intelligent agents
Technologies for Intelligent Agents Collaborative

Methods for simple behaviors:

  • behaviors

  • reinforcement learning

  • distributed CSP

    Methods for controlling behaviors:

  • planning

  • case­based reasoning


Technologies for intelligent agents1
Technologies for Intelligent Agents Collaborative

Formalisms for cooperation:

  • auctions

  • negotiation

  • BDI (Belief-Desire-Intention)

  • ACL/KQML

  • Ontologies


Technologies for intelligent agents2
Technologies for Intelligent Agents Collaborative

Theories of agent systems:

  • self­interestedness

  • competition/economies

  • behavior of agent systems

    Platforms:

  • auctions, markets (negotiation, contracts)

  • multi­agent platforms

  • mobile agent platforms


Agent languages

Agent languages Collaborative


Agent communication languages1
Agent Communication Languages Collaborative

Structure: performatives + content language

KQML

Criteria for content languages


Setting
Setting Collaborative

Communication among heterogeneous agents:

  • no common data structures

  • no common messages

  • no common ontology

    but common communication language:

  • protocol

  • reference


Structure of an acl
Structure of an ACL Collaborative

Vocabulary (words): e.g. reference to objects

Messages (sentences): e.g. request for an action

Distributed Algorithms (conversations): e.g. negotiating task sharing


Levels of acl
Levels of ACL Collaborative

Object sharing (Corba, RPC, RMI, Splice): shared objects, procedures, data structures

Knowledge sharing (KQML, FIPA ACL): shared facts, rules, constraints, procedures and knowledge

Intentional sharing: shared beliefs, plans, goals and intentions

Cultural sharing: shared experiences and strategies


Human communication intentional cultural sharing
Human communication: intentional/cultural sharing Collaborative

Ideal example of a heterogeneous agent system: human society

See agents as intentional systems:

all actions and communication are motivated by beliefs and intentions

Allows modeling agent behavior in a human­understandable way


Problems with intentional sharing
Problems with intentional sharing Collaborative

BDI model requires modal logics

Many modal logics pose unrealistic computational requirements:

  • all consequences known

  • combinatorial inference

     BDI model too general as a basis for agent cooperation


A feasible solution knowledge sharing
A feasible solution: knowledge sharing Collaborative

ACL = 2 components:

  • performative: request, inform, etc.

  • content: a piece of knowledge

    Allows formulating distributed algorithms in a heterogeneous agent society

    Basis: human communication/speech acts


Speech acts
Speech acts Collaborative

Language =

  • content (e.g., read a book) +

  • speech act (e.g., I want to, I want you to,...)

    Reference:

  • locution: physical utterance

  • illocution: act of conveying intentions

  • perlocutions: actions that occur as a result


Kqml knowledge query and manipulation language
KQML (Knowledge Query and Manipulation Language) Collaborative

(tell :sender Amazon

:receiver John

:in­reply­to id7.24.97.45391

:ontology lk1

:language Prolog

:content ''price(ISBN348291, 24.95)'‘

)

Represents a single speech act (tell)

which defines the relevant attributes


Kqml keywords
KQML keywords Collaborative

:sender sender of the message

:receive receiver of the message

:from actual origin if forwarded

:to actual destination if to be forwarded

:in­reply­to reference if this is a reply

:reply­with reference is a reply is expected

:language language used for content

:ontology ontology used for content

:content content itself


Kmql queries

A Collaborative

A

A

A

B

B

B

B

KMQL queries

Ask-one(P)

Ask-if(P)

tell(P)

sorry

Subscribe( Ask-one(P))

Ask-all(P)

tell(P1)

tell(P1, P2, …)

tell(P2)


Requests achieve unachieve

A Collaborative

B

Requests: achieve & unachieve

Achieve: make a proposition true

Unachieve: undo the previous achieve

Achieve(P)

Reply()


Facilitation
Facilitation Collaborative

Advertise(ask(P))

Broker(ask(P))

ask(P))

C

A

B

tell(P))

tell(P))

Advertise(ask(P))

recommend(ask(P))

C

A

B

tell(P))

forward(advertise(ask((P)))

ask(P))

tell(P))


Content in kqml
Content in KQML Collaborative

Most commonly: Knowledge Interchange Format (KIF)

= predicate calculus in Lisp form

 agents are logical reasoners (EPILOG system available from Stanford University)

Several agent systems use KIF as a basis, e.g. IBM's ABE (Agent Building Environment)


Programming conversations with kif
Programming conversations with KIF Collaborative

Responding to queries = decide if KIF expression

  • is consistent with current beliefs

  • is entailed by current beliefs

    Contracting tasks (achieve) = defining subgoals in a plan

    Querying information = asking for information for completing a proof

     in general first­order logic, all these tasks are computable, but not decidable


Better formalisms for content
Better formalisms for content Collaborative

Content languages must make formulating and responding to performatives decidable!

  • restrict content language to be less expressive than predicate calculus

    Examples:

  • description logics (LOOM)

  • constraint content language (CCL)


Programming agent communcication in description logics
Programming agent communcication in Description Logics Collaborative

Description logic =

reduced form of predicate calculus so that subsumption (an expression consistent with a certain class) can be efficiently decided

  • can efficiently answer and formulate queries

    Usefulness for planning and negotiation less clear


Programming agent communication in ccl
Programming agent communication in CCL Collaborative

CCL allows expressing constraint satisfaction problems:

variables, domains, relations and constraints

 answering an information query = deciding whether

  • a variable assignment is consistent with constraints (linear time)

  • a combined CSP is consistent (NP­complete)

  • possibly: intermediate notions of consistency/entailment

    Other advantages: more efficient to formulate complex protocols, especially negotiation and coordinated plans


Summary1
Summary Collaborative

Agent Communication languages are important for building heterogeneous agent systems.

2 levels: performative + content

Issue with content: is there an efficient mechanism for programming agent conversations around them?


Embedded agents

Embedded agents Collaborative


Implementing embedded agents

Planning/reasoning Collaborative

behaviors

world

Implementing embedded agents

Behavior: conventional real­time programming

Planning/Reasoning: with limited resources


Reasoning with limited resources
Reasoning with limited resources Collaborative

Options for reasoning under time constraints:

  • meta­reasoning about resource consumption

  • competitive algorithms

  • anytime algorithms

  • learning and compilation


Reasoning about resource consumption
Reasoning about resource consumption Collaborative

Estimate computation time  set parameters of method

Problems:

  • times are very diffcult to estimate

  • what to do when no method is fast enough?

     not very promising in practice


Competitive algorithms
Competitive algorithms Collaborative

Idea: run several methods in parallel

First one to find a solution wins

Examples:

  • breadth­first/depth­first search

  • different problem solving heuristics

    Can be very successful!


Anytime algorithms
Anytime algorithms Collaborative

Idea: algorithm first finds rough solution, then improves with more computation time.

Example: iterative deepening search

bubble sort


Depth first search
Depth-first search Collaborative

Not necessarily shortest path

Limited memory requirement


Breadth first search
Breadth-first search Collaborative

Finds shortest path

Large memory requirement


Depth limited search
Depth­limited search Collaborative

Usually, breadth­first search requires too much memory to be practical.

Main problem with depth­first search:

  • can follow a dead­end path very far before this is discovered.

     impose a depth limit l:

    never explore nodes at depth > l

    What is the right depth limit?


Iterative deepening
Iterative Deepening Collaborative

Idea: depth­limited search with increasing limit

Some repeated work, but:

tree with n leaves always has < n intermediate nodes

  • effort of search with limit l always at least double that of search with limit l -1

  • effort of previous searches always less than current

     complexity no more than double of straight DFS


Iterative deepening as an anytime algorithm
Iterative deepening as an anytime algorithm Collaborative

As time allows, increase the search depth

 Solution improves with more computation time


Learning and compilation
Learning and Compilation Collaborative

Idea: precompute solutions for the most current problems

 customized agent which works well only in a certain environment

Techniques:

  • reinforcement learning

  • case­based reasoning


Reinforcement learning
Reinforcement learning Collaborative

Consider game:

  • Finite set of states S1, …, Sn

  • Finite set of actions A(S)= A1, …, Am depending on state

  • Payoffs r(A,S)

    Learn a strategy p, mapping S  A(S) such that averagepayoff is maximized


Optimization criteria
Optimization criteria Collaborative

Game theory: optimize only current step

But actions also determine future state  future payoffs!

Instead: optimize:

  • finite­horizon: S r(S(t))

  • infinite­horizon: S gt r(S(t))

  • average reward: lim h  inf E( h-1S r(S(t)))


Learning with known model
Learning with known model Collaborative

Model =

  • state transition function T (S; A; S’ )

  • payoff function R(S; A)

     optimal strategy can be computed in two steps:

  • assign values V (S) to states (off­line)

  • select optimal action (on­line)


Learning with unknown models
Learning with unknown models Collaborative

Often, model is not known a­priori

  • agent can only deduce model from experience!

    Approaches:

    • learn the model, then the strategy

    • learn both at the same time

      First approach tends to be unstable!


Limitations
Limitations... Collaborative

Biggest limitation of reinforcement learning:

space of states, actions must be finite, small

Main difficulty: modeling real­world problems to fit the framework!


Summary2
Summary Collaborative

Challenge for embedded behavior:

limited resources

Two successful methods:

anytime algorithms

reinforcement learning  try it at:

http://www.fe.dis.titech.ac.jp/~gen/robot/robodemo.html


ad