Market-Driven Multi-Agent Collaboration in Robot Soccer Domain - PowerPoint PPT Presentation

Market driven multi agent collaboration in robot soccer domain l.jpg
Download
1 / 39

Market-Driven Multi-Agent Collaboration in Robot Soccer Domain Today’s Presentation Multi-Agent Systems Robot Soccer The Market Methodology Market-Driven Approach Reinforcement-Based Market-Driven Approach A “New Approach” Multi-Agent Systems Multi-Agent Systems

Related searches for Market-Driven Multi-Agent Collaboration in Robot Soccer Domain

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

Download Presentation

Market-Driven Multi-Agent Collaboration in Robot Soccer Domain

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Market driven multi agent collaboration in robot soccer domain l.jpg

Market-Driven Multi-Agent Collaboration in Robot Soccer Domain


Today s presentation l.jpg

Today’s Presentation

  • Multi-Agent Systems

  • Robot Soccer

  • The Market Methodology

  • Market-Driven Approach

  • Reinforcement-Based Market-Driven Approach

  • A “New Approach”


Multi agent systems l.jpg

Multi-Agent Systems


Multi agent systems4 l.jpg

Multi-Agent Systems

  • Why use multi-agent systems?

  • Multi-agent systems are becoming more popular than complex single agent systems because they eliminate the problem of single point of failure.


Multi agent systems5 l.jpg

Multi-Agent Systems

  • How do they work?

  • Multi-agent systems work by decomposing a complex task into several low-level actions which can then be assigned to the individual team members.


Multi agent systems6 l.jpg

Multi-Agent Systems

  • How to assign tasks?

  • This is a key problem, the system must break up the tasks and coordinate the team such that the team collectively completes the overall task.


Multi agent systems7 l.jpg

Multi-Agent Systems

  • How to assign tasks?

  • The system must keep track of each robot’s capabilities (trivial in a homogeneous team, but more complicated in a heterogeneous team)


Robot soccer l.jpg

Robot Soccer


Robot soccer9 l.jpg

Robot Soccer

  • Problem Domain?

  • We will look at robot soccer as the problem domain as it provides a very good real world domain for developing multi-agent systems.


Robot soccer domain l.jpg

Robot Soccer Domain

  • Robot soccer is a well-defined environment which provides a good test-bed for developing multi-agent strategies. Each robot has simple, clearly defined actions available and the overall task easy to understand –Beat the other team.


Robot soccer domain11 l.jpg

Robot Soccer Domain

  • Robot soccer provides a good way of comparing two systems/strategies. The two systems can simply be played against each other and see which team wins the most matches.


The problem l.jpg

The Problem

  • We need a way of coordinating the robots to each perform a task/fulfil a role (ie attack, support, defend, goalie etc).

  • The Market-Driven Approach for coordinating the multi-agent system is based on the way free-markets maximize profits.


The market methodology l.jpg

The Market Methodology


The market methodology14 l.jpg

The Market Methodology

  • The main goal in free-markets is the maximization of the overall profit. The theory is that if each participant in the market tries to maximize its profit, the overall profit should increase.


Market driven approach l.jpg

Market-Driven Approach


The market driven approach l.jpg

The Market-Driven Approach

  • The Market-Driven Approach splits up the main task into simple tasks and an auction is then held for each task. The robots work out the cost for them to perform a task and then put in their best bid to the auctioneer. The robot which puts in the lowest bid gets the assignment.


The market driven approach17 l.jpg

The Market-Driven Approach


The market driven approach18 l.jpg

The Market-Driven Approach

  • In Robot Soccer an auction is held for each of the different roles. The robots calculate the cost of fulfilling those roles (based on distance to ball etc) and bid on them. The robots with the best bid on each role will be assigned the role.


The market driven approach19 l.jpg

The Market-Driven Approach

  • Two (or more) robots may get the same assignment where they must cooperate to perform the task (ie a robot with the ball attacks the goal and another robot supports it by driving close behind)


The market driven approach20 l.jpg

The Market-Driven Approach

  • An advantage of the Market-Driven Approach is that each robot calculates the cost of performing each role and communicates that cost to the other robots. This cost value is much easier and quicker to communicate rather than sending all of the metrics to the other robots.


The market driven approach21 l.jpg

The Market-Driven Approach

  • What about how the auction is run?

  • Centralized

  • Distributed

  • Hybrid


Centralized l.jpg

Centralized

  • There exists a master agent (auctioneer) that controls the auctions and assigns the roles.

  • The master agent receives offers from all other agents for each task and sends the auction results back.

  • Computationally efficient.

  • Prone to single point failures.


Distributed l.jpg

Distributed

  • No master agent.

  • Every agent broadcasts its offer for every task.

  • Every agent runs the same auction mechanism and parallely computes the auction results.

  • Robust against single point failures

  • Requires more computation in total.


Hybrid l.jpg

Hybrid

  • There exists a master agent

  • There is also an auction for the task of being the master

  • Robust against single point failures

  • Computationly efficient

  • Still not implemented, no test results.


The market driven approach25 l.jpg

The Market-Driven Approach

  • Problem – How to calculate the costs?

  • Each robot must be able to calculate the cost of filling a particular role. The settings for the cost calculations must be calibrated, the performance of the system depends on the calibrations being correct.

  • Eg. - Cattacker = M2*distBall + M2*distOppGoal


Reinforcement based market driven approach l.jpg

Reinforcement-Based Market-Driven Approach


Reinforcement learning l.jpg

Reinforcement Learning

  • Reinforcement-Based Market-Driven Approach makes use of Reinforcement Learning (RL) to learn the role assignment process. RL is used when the agent is informed about the consequences of its actions. RL replaces the role assignment as described above.


Reinforcement learning28 l.jpg

Reinforcement Learning

  • With the RL system, the robot closest to the ball assigns itself as the attacker and the remaining agents (excluding the static goalie) assign themselves according to a state vector. (see next slide)


Reinforcement learning29 l.jpg

Reinforcement Learning

  • The Rules:

  • Goalie is statically assigned.

  • The Robot closest to the ball is assigned the role of attacker.

  • The other robots are assigned roles by a state vector.

  • State vector metrics – distances to the ball, goals, robots, the cost values and the closest player to the ball.


Reinforcement learning30 l.jpg

Reinforcement Learning

Broadcast Position and Cost Data

Calculate Attack Cost Array

Calculate Defence Cost Array

Closest to Ball

Cheapest

Yes

Yes

Shoot

No

No

Role Assigned According to Cost Value

Pass To Cheapest


New approach l.jpg

New Approach


New approach32 l.jpg

New Approach

  • The New Approach is effectively a simplified version of the Reinforcement Learning system. However instead of using the exact positions of the robots, the field is divided into a grid.


New approach33 l.jpg

New Approach


New approach34 l.jpg

New Approach

  • The system can now use this grid to make a decision on what role the robot should be performing. To assign roles, the system uses a state vector with the following metrics: Ball Position (grid number), Ball Possession, Current Role assigned by the Market-Driven strategy, Teammate positions and Opponent positions.


New approach35 l.jpg

New Approach

  • This approach combines the Market-Driven Approach and the Reinforcement Learning based team with the grid separation of the board to keep the number of variables in the state vector to a minimum.


Results l.jpg

Results


Results37 l.jpg

Results

  • The New Approach which combines the Market-Driven, RL and grid system out performs all of the other teams consistently over 90 matches.


Questions l.jpg

Questions?


References l.jpg

References

Kose, H., Kaplan, K., Mericli, C., Tatlidede, U. & Akin, L. (2005). Market-Driven Multi-Agent Collaboration in Robot Soccer Domain. Cutting Edge Robotics, 407-416.

Kurt, B. (2007). Bogazici University Robotics Server. Retrieved September 09, 2007, from http://robot.cmpe.boun.edu.tr/robsem/ailab_market.ppt


  • Login