Market driven multi agent collaboration in robot soccer domain
1 / 39

Daniel - PowerPoint PPT Presentation

  • Updated On :

Market-Driven Multi-Agent Collaboration in Robot Soccer Domain Today’s Presentation Multi-Agent Systems Robot Soccer The Market Methodology Market-Driven Approach Reinforcement-Based Market-Driven Approach A “New Approach” Multi-Agent Systems Multi-Agent Systems

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Daniel' - JasminFlorian

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Today s presentation l.jpg
Today’s Presentation Domain

  • Multi-Agent Systems

  • Robot Soccer

  • The Market Methodology

  • Market-Driven Approach

  • Reinforcement-Based Market-Driven Approach

  • A “New Approach”

Multi agent systems4 l.jpg
Multi-Agent Systems Domain

  • Why use multi-agent systems?

  • Multi-agent systems are becoming more popular than complex single agent systems because they eliminate the problem of single point of failure.

Multi agent systems5 l.jpg
Multi-Agent Systems Domain

  • How do they work?

  • Multi-agent systems work by decomposing a complex task into several low-level actions which can then be assigned to the individual team members.

Multi agent systems6 l.jpg
Multi-Agent Systems Domain

  • How to assign tasks?

  • This is a key problem, the system must break up the tasks and coordinate the team such that the team collectively completes the overall task.

Multi agent systems7 l.jpg
Multi-Agent Systems Domain

  • How to assign tasks?

  • The system must keep track of each robot’s capabilities (trivial in a homogeneous team, but more complicated in a heterogeneous team)

Robot soccer l.jpg
Robot Soccer Domain

Robot soccer9 l.jpg
Robot Soccer Domain

  • Problem Domain?

  • We will look at robot soccer as the problem domain as it provides a very good real world domain for developing multi-agent systems.

Robot soccer domain l.jpg
Robot Soccer Domain Domain

  • Robot soccer is a well-defined environment which provides a good test-bed for developing multi-agent strategies. Each robot has simple, clearly defined actions available and the overall task easy to understand –Beat the other team.

Robot soccer domain11 l.jpg
Robot Soccer Domain Domain

  • Robot soccer provides a good way of comparing two systems/strategies. The two systems can simply be played against each other and see which team wins the most matches.

The problem l.jpg
The Problem Domain

  • We need a way of coordinating the robots to each perform a task/fulfil a role (ie attack, support, defend, goalie etc).

  • The Market-Driven Approach for coordinating the multi-agent system is based on the way free-markets maximize profits.

The market methodology14 l.jpg
The Market Methodology Domain

  • The main goal in free-markets is the maximization of the overall profit. The theory is that if each participant in the market tries to maximize its profit, the overall profit should increase.

The market driven approach l.jpg
The Market-Driven Approach Domain

  • The Market-Driven Approach splits up the main task into simple tasks and an auction is then held for each task. The robots work out the cost for them to perform a task and then put in their best bid to the auctioneer. The robot which puts in the lowest bid gets the assignment.

The market driven approach18 l.jpg
The Market-Driven Approach Domain

  • In Robot Soccer an auction is held for each of the different roles. The robots calculate the cost of fulfilling those roles (based on distance to ball etc) and bid on them. The robots with the best bid on each role will be assigned the role.

The market driven approach19 l.jpg
The Market-Driven Approach Domain

  • Two (or more) robots may get the same assignment where they must cooperate to perform the task (ie a robot with the ball attacks the goal and another robot supports it by driving close behind)

The market driven approach20 l.jpg
The Market-Driven Approach Domain

  • An advantage of the Market-Driven Approach is that each robot calculates the cost of performing each role and communicates that cost to the other robots. This cost value is much easier and quicker to communicate rather than sending all of the metrics to the other robots.

The market driven approach21 l.jpg
The Market-Driven Approach Domain

  • What about how the auction is run?

  • Centralized

  • Distributed

  • Hybrid

Centralized l.jpg
Centralized Domain

  • There exists a master agent (auctioneer) that controls the auctions and assigns the roles.

  • The master agent receives offers from all other agents for each task and sends the auction results back.

  • Computationally efficient.

  • Prone to single point failures.

Distributed l.jpg
Distributed Domain

  • No master agent.

  • Every agent broadcasts its offer for every task.

  • Every agent runs the same auction mechanism and parallely computes the auction results.

  • Robust against single point failures

  • Requires more computation in total.

Hybrid l.jpg
Hybrid Domain

  • There exists a master agent

  • There is also an auction for the task of being the master

  • Robust against single point failures

  • Computationly efficient

  • Still not implemented, no test results.

The market driven approach25 l.jpg
The Market-Driven Approach Domain

  • Problem – How to calculate the costs?

  • Each robot must be able to calculate the cost of filling a particular role. The settings for the cost calculations must be calibrated, the performance of the system depends on the calibrations being correct.

  • Eg. - Cattacker = M2*distBall + M2*distOppGoal

Reinforcement learning l.jpg
Reinforcement Learning Domain

  • Reinforcement-Based Market-Driven Approach makes use of Reinforcement Learning (RL) to learn the role assignment process. RL is used when the agent is informed about the consequences of its actions. RL replaces the role assignment as described above.

Reinforcement learning28 l.jpg
Reinforcement Learning Domain

  • With the RL system, the robot closest to the ball assigns itself as the attacker and the remaining agents (excluding the static goalie) assign themselves according to a state vector. (see next slide)

Reinforcement learning29 l.jpg
Reinforcement Learning Domain

  • The Rules:

  • Goalie is statically assigned.

  • The Robot closest to the ball is assigned the role of attacker.

  • The other robots are assigned roles by a state vector.

  • State vector metrics – distances to the ball, goals, robots, the cost values and the closest player to the ball.

Reinforcement learning30 l.jpg
Reinforcement Learning Domain

Broadcast Position and Cost Data

Calculate Attack Cost Array

Calculate Defence Cost Array

Closest to Ball







Role Assigned According to Cost Value

Pass To Cheapest

New approach l.jpg
New Approach Domain

New approach32 l.jpg
New Approach Domain

  • The New Approach is effectively a simplified version of the Reinforcement Learning system. However instead of using the exact positions of the robots, the field is divided into a grid.

New approach33 l.jpg
New Approach Domain

New approach34 l.jpg
New Approach Domain

  • The system can now use this grid to make a decision on what role the robot should be performing. To assign roles, the system uses a state vector with the following metrics: Ball Position (grid number), Ball Possession, Current Role assigned by the Market-Driven strategy, Teammate positions and Opponent positions.

New approach35 l.jpg
New Approach Domain

  • This approach combines the Market-Driven Approach and the Reinforcement Learning based team with the grid separation of the board to keep the number of variables in the state vector to a minimum.

Results l.jpg
Results Domain

Results37 l.jpg
Results Domain

  • The New Approach which combines the Market-Driven, RL and grid system out performs all of the other teams consistently over 90 matches.

Questions l.jpg
Questions? Domain

References l.jpg
References Domain

Kose, H., Kaplan, K., Mericli, C., Tatlidede, U. & Akin, L. (2005). Market-Driven Multi-Agent Collaboration in Robot Soccer Domain. Cutting Edge Robotics, 407-416.

Kurt, B. (2007). Bogazici University Robotics Server. Retrieved September 09, 2007, from