CS483/683 Multi-Agent Systems

1 / 16

# CS483/683 Multi-Agent Systems - PowerPoint PPT Presentation

Computer Science & Engineering, University of Nevada, Reno. CS483/683 Multi-Agent Systems. Lectures 5-6: From Satisfaction to Optimization ADOPT: Asynchronous Distributed Optimization. 2-4 February 2010 Instructor: Kostas Bekris. Distributed Constrained Optimization. DCOP:

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Computer Science & Engineering, University of Nevada, RenoCS483/683 Multi-Agent Systems
• Lectures 5-6:
• From Satisfaction to Optimization

2-4 February 2010

Instructor: Kostas Bekris

Distributed Constrained Optimization
• DCOP:
• A set of n variables V={X1, ..., Xn}
• Each variable has a discrete domain: D1, ..., Dn
• Each variable is assigned to an agent
• Only the agent who is assigned a variable has control over its value
• And knowledge of its domain
• Assign values to variable so as to optimize a global objective function
• The optimization of the function satisfies a set of constraints
• Requirements:
• Distributed computation using only local communication
• Fast, asynchronous computation - agents should work in parallel
• Quality guarantees are needed
• Provably optimal solutions whenever possible
• Trade-off between computation and solution quality
• Bounded-error approximation: Guarantee solution within a distance from optimal, less time than the optimal
• The objective function is a summation over a set of cost functions
• F(A)=∑Xi,Xj∈V fij(di,dj)
• Xi←di, Xj←dj in A
• We want to find A* that minimizes F(A)
• e.g. F( { (X1,0), (X2,0), (X3,0), (X4,0) } ) = 4
• F( { (X1,1), (X2,1), (X3,1), (X4,1) } ) = 0

X1

X2

X3

X4

Constraint Graph

Assumptions
• 1. Summation operation over cost function
• Associative
• Commutative
• Monotonic
• Cost of a solution can only increase as more costs are aggregated (i.e., we cannot have negative cost)
• 2. Constraints are at most binary
• There are ways to extend to constraints that involve a larger number of variables
• 3. Each agent is assigned a single variable
• There is a way to extend to the case that an agent must handle multiple variables
• 1. Opportunistic best-first search
• Agents are prioritized in a tree structure
• an agent has a single parent and multiple children
• Each agent keeps on choosing the best value based on the current available information
• i.e., chooses the variable which implies the smallest lower bound
• lower bounds do not need global information to be estimated
• Each agent maintains a lower and an upper bound for the cost of its subtrees
• and informs its parent about its own bounds
• Strategy allows agents to abandon partial solutions which have not been proven to be suboptimal
• they may have to reconsider the same assignments into the future

X1

X2

X3

X4

Communication Graph

• 2. Backtrack Threshold
• When an agent knows from previous search experiences that lb is a lower bound for its subtree
• inform the subtree agents not to bother searching for a solution whose cost is lower than lb
• In the general case, remembering these lower bounds for past assignments requires exponential space
• Approach remembers only one value and then cost is subdivided to children arbitrarily and adapted on the fly as new computations are executed
• 3. Built-in Termination Detection
• Keeping track of bounds (lower and upper bound for the cost function) on each agent
• allows to keep track of the progress towards the optimum solution
• and automatically terminates when necessary
Messages and Data Structures

X1

• VALUE message (like ok?)
• Send selected value to children along the
• constraint graph
• COST message (like NoGood)
• Send to parents along the communication graph
• THRESHOLD message
• Send to children along the communication graph
• Each agent maintains the “context”
• (like the “agent_view”)
• A recorf of higher priority neighbors’ current variable assignment
• Two contexts are compatible if they do not disagree on any variable assignment

X2

X3

X4

Constraint Graph

X1

X2

X3

X4

Communication Graph

COST message
• Xk transmits COST message to Xi
• Message contains
• context of Xk
• lb of Xk
• up of Xk
• When Xi receives the message it stores
• lb(d,Xk)
• ub(d,Xk)
• where d is the assignment of Xi in Xk’s context
• If context of Xi is incompatible with the context of Xk:
• lb(d,Xk) = 0
• ub(d,Xk) = ∞

Xi

Xk

Costs and Bounds

X1

• Local cost: δ(di) = ∑(Xj,dj) fij(di,dj)
• di : assignment of agent Xi
• Xj : higher priority neighbors than Xi
• Lower bound for value d:
• ∀ d ∈ Di:
• LB(d) = δ(d) + ∑ Xk ∈ children lb(d,Xk)
• similarly for the upper bound for value d
• Lower bound:
• LB = min d ∈ Di LB(d)
• similarly for the upper bound
• For leaves: LB(d) = UB(d) = δ(d)
• If not a leaf but has not get received a COST message: LB = δ(d) and UB = ∞

X2

X3

X4

Constraint Graph

X1

X2

X3

X4

Communication Graph

When does Xi change value?
• Whenever LB(di) exceeds the backtrack threshold value, Xi changes its variable value to one with smaller lower bound
• The threshold is updated with the following three ways:
• Its value can increase whenever Xi determines that LB is greater than the current threshold
• guarantees that there is always a variable with a lower bound than the threshold
• Its value can decrease whenever Xi determines that UB is lower than the current threshold
• Invariant: LB ≤ threshold ≤ UB
• Its value is also updated whenever a THRESHOLD message is received from a parent
• a parent subdivides its own threshold value among its children
• t(d,Xk): the threshold on cost allocated by parent Xi to child Xk
• then the value of t(d,Xk) respects the following invariants:
• threshold = δ(di) + ∑ Xk ∈ children t(di,Xk)
• ∀ d ∈ Di, ∀ Xk ∈ children: lb(d,Xk) ≤ t(d,Xk) ≤ ub(d,Xk)
Example
• All agents begin concurrently choosing 0.
• Each agents send a VALUE message to lower priority agents along the constraint graph
• We will follow one specific execution path - there are many possible

X1

X1

X2

X2

X3

X4

X3

X4

Constraint Graph

Communication Graph

Example

X1=0

• X2 receives X1’s VALUE message
• and records this value to its context
• X2’s context: {X1=0}
• Then it computes bounds:
• LB(0) = δ(0) + lb(0,X3) + lb(0,X4) = 1
• LB(1) = δ(1) + lb(1,X3) + lb(1,X4) = 2
• LB(0) < LB(1) ⇒ LB = LB(0) = 1
• Similarly: UB = ∞
• threshold was set to LB(0), equal to 1 so the invariant holds
• Transmits a COST message to X1:
• COST( {X1=0}, 1, ∞ )

X2=0

X4=0

X3=0

Example

X1=0

• X3 receives X1’s and X2’s VALUE messages
• and records these values to its context
• X3’s context: {X1=0, X2=0}
• Then it computes bounds:
• LB(0) = δ(0) = 1 + 1 = 2
• LB(1) = δ(1) = 2 + 2 = 4
• LB(0) < LB(1) ⇒ LB = LB(0) = 2
• Similarly: UB = 2
• threshold was LB(0) = 2 so the invariant holds
• Transmits a COST message to X2:
• COST( {X1=0, X2=0}, 2, 2 )
• Similarly with X4... but no reference to X1

X2=0

X4=0

X3=0

Example

X1=1

• X1 receives X2’s COST message
• COST( {X1=0}, 1, ∞ )
• test if compatible with its current context
• store: lb(0,X2) = 1 and ub(0,X2) = ∞
• Then it computes bounds:
• LB(0) = δ(0) + lb(0,X2) = 0 + 1 = 1
• LB(1) = δ(1) + lb(1,X2) = 0 + 0 = 0
• LB(1) < LB(0) ⇒ LB = LB(1) = 0
• Similarly: UB = ∞
• threshold was 0, but LB(0) = 1:
• violation of the invariant, change assignment
• Send VALUE messages to children

X2=0

X4=0

X3=0

Example

X1=1

• Assume COST messages from X3 and X4 are delayed... instead VALUE message from X1 arrives first at X2
• Current context at X2: {X1=1}
• When X2 receives the COST messages from X3 its context will be incompatible with X2’s
• the bounds in the message will not be stored
• The message from X4 is not incompatible:
• store lb(0,x4) = 1 and up(0,x4) = 1
• Then it computes bounds:
• LB(0) = δ(0) + lb(0,X3) + lb(0,X4) = 2+ 0 +1 = 3
• LB(1) = δ(1) + lb(1,X3) + lb(1,X4) = 0 +0 +0 = 0
• LB(1) < LB(0) ⇒ LB = LB(1) = 1
• Similarly: UB = ∞

X2=1

X4=0

X3=0

Example

X1=1

• X2 will inform X3 and X4 about the changes.
• Similar changes will take place on X3 and X4:
• 1 will be selected as the value
• COST messages will be transmitted:
• from X4 to X2: ( {X2=1}, 0, 0 )
• from X3 to X1 and X2: ( {X1=1,X2=1}, 0, 0 )
• from X2 to X1: ( {X1=1}, 0, 0)
• this is after receiving the COST messages from X3 and X4
• Upon receipt of the COST message of X2 at X1:
• LB = UB = threshold = 0
• X1 sends TERMINATE messages to other agents.

X2=1

X4=1

X3=1