market driven multi agent collaboration in robot soccer domain l.
Skip this Video
Loading SlideShow in 5 Seconds..
Market-Driven Multi-Agent Collaboration in Robot Soccer Domain PowerPoint Presentation
Download Presentation
Market-Driven Multi-Agent Collaboration in Robot Soccer Domain

Loading in 2 Seconds...

play fullscreen
1 / 39

Market-Driven Multi-Agent Collaboration in Robot Soccer Domain - PowerPoint PPT Presentation

  • Uploaded on

Market-Driven Multi-Agent Collaboration in Robot Soccer Domain Today’s Presentation Multi-Agent Systems Robot Soccer The Market Methodology Market-Driven Approach Reinforcement-Based Market-Driven Approach A “New Approach” Multi-Agent Systems Multi-Agent Systems

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

Market-Driven Multi-Agent Collaboration in Robot Soccer Domain

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
today s presentation
Today’s Presentation
  • Multi-Agent Systems
  • Robot Soccer
  • The Market Methodology
  • Market-Driven Approach
  • Reinforcement-Based Market-Driven Approach
  • A “New Approach”
multi agent systems4
Multi-Agent Systems
  • Why use multi-agent systems?
  • Multi-agent systems are becoming more popular than complex single agent systems because they eliminate the problem of single point of failure.
multi agent systems5
Multi-Agent Systems
  • How do they work?
  • Multi-agent systems work by decomposing a complex task into several low-level actions which can then be assigned to the individual team members.
multi agent systems6
Multi-Agent Systems
  • How to assign tasks?
  • This is a key problem, the system must break up the tasks and coordinate the team such that the team collectively completes the overall task.
multi agent systems7
Multi-Agent Systems
  • How to assign tasks?
  • The system must keep track of each robot’s capabilities (trivial in a homogeneous team, but more complicated in a heterogeneous team)
robot soccer9
Robot Soccer
  • Problem Domain?
  • We will look at robot soccer as the problem domain as it provides a very good real world domain for developing multi-agent systems.
robot soccer domain
Robot Soccer Domain
  • Robot soccer is a well-defined environment which provides a good test-bed for developing multi-agent strategies. Each robot has simple, clearly defined actions available and the overall task easy to understand –Beat the other team.
robot soccer domain11
Robot Soccer Domain
  • Robot soccer provides a good way of comparing two systems/strategies. The two systems can simply be played against each other and see which team wins the most matches.
the problem
The Problem
  • We need a way of coordinating the robots to each perform a task/fulfil a role (ie attack, support, defend, goalie etc).
  • The Market-Driven Approach for coordinating the multi-agent system is based on the way free-markets maximize profits.
the market methodology14
The Market Methodology
  • The main goal in free-markets is the maximization of the overall profit. The theory is that if each participant in the market tries to maximize its profit, the overall profit should increase.
the market driven approach
The Market-Driven Approach
  • The Market-Driven Approach splits up the main task into simple tasks and an auction is then held for each task. The robots work out the cost for them to perform a task and then put in their best bid to the auctioneer. The robot which puts in the lowest bid gets the assignment.
the market driven approach18
The Market-Driven Approach
  • In Robot Soccer an auction is held for each of the different roles. The robots calculate the cost of fulfilling those roles (based on distance to ball etc) and bid on them. The robots with the best bid on each role will be assigned the role.
the market driven approach19
The Market-Driven Approach
  • Two (or more) robots may get the same assignment where they must cooperate to perform the task (ie a robot with the ball attacks the goal and another robot supports it by driving close behind)
the market driven approach20
The Market-Driven Approach
  • An advantage of the Market-Driven Approach is that each robot calculates the cost of performing each role and communicates that cost to the other robots. This cost value is much easier and quicker to communicate rather than sending all of the metrics to the other robots.
the market driven approach21
The Market-Driven Approach
  • What about how the auction is run?
  • Centralized
  • Distributed
  • Hybrid
  • There exists a master agent (auctioneer) that controls the auctions and assigns the roles.
  • The master agent receives offers from all other agents for each task and sends the auction results back.
  • Computationally efficient.
  • Prone to single point failures.
  • No master agent.
  • Every agent broadcasts its offer for every task.
  • Every agent runs the same auction mechanism and parallely computes the auction results.
  • Robust against single point failures
  • Requires more computation in total.
  • There exists a master agent
  • There is also an auction for the task of being the master
  • Robust against single point failures
  • Computationly efficient
  • Still not implemented, no test results.
the market driven approach25
The Market-Driven Approach
  • Problem – How to calculate the costs?
  • Each robot must be able to calculate the cost of filling a particular role. The settings for the cost calculations must be calibrated, the performance of the system depends on the calibrations being correct.
  • Eg. - Cattacker = M2*distBall + M2*distOppGoal
reinforcement learning
Reinforcement Learning
  • Reinforcement-Based Market-Driven Approach makes use of Reinforcement Learning (RL) to learn the role assignment process. RL is used when the agent is informed about the consequences of its actions. RL replaces the role assignment as described above.
reinforcement learning28
Reinforcement Learning
  • With the RL system, the robot closest to the ball assigns itself as the attacker and the remaining agents (excluding the static goalie) assign themselves according to a state vector. (see next slide)
reinforcement learning29
Reinforcement Learning
  • The Rules:
  • Goalie is statically assigned.
  • The Robot closest to the ball is assigned the role of attacker.
  • The other robots are assigned roles by a state vector.
  • State vector metrics – distances to the ball, goals, robots, the cost values and the closest player to the ball.
reinforcement learning30
Reinforcement Learning

Broadcast Position and Cost Data

Calculate Attack Cost Array

Calculate Defence Cost Array

Closest to Ball







Role Assigned According to Cost Value

Pass To Cheapest

new approach32
New Approach
  • The New Approach is effectively a simplified version of the Reinforcement Learning system. However instead of using the exact positions of the robots, the field is divided into a grid.
new approach34
New Approach
  • The system can now use this grid to make a decision on what role the robot should be performing. To assign roles, the system uses a state vector with the following metrics: Ball Position (grid number), Ball Possession, Current Role assigned by the Market-Driven strategy, Teammate positions and Opponent positions.
new approach35
New Approach
  • This approach combines the Market-Driven Approach and the Reinforcement Learning based team with the grid separation of the board to keep the number of variables in the state vector to a minimum.
  • The New Approach which combines the Market-Driven, RL and grid system out performs all of the other teams consistently over 90 matches.

Kose, H., Kaplan, K., Mericli, C., Tatlidede, U. & Akin, L. (2005). Market-Driven Multi-Agent Collaboration in Robot Soccer Domain. Cutting Edge Robotics, 407-416.

Kurt, B. (2007). Bogazici University Robotics Server. Retrieved September 09, 2007, from