ISPA 2008 APDCT Workshop
Download
1 / 44

Reinforcement Learning applied to Meta -scheduling in grid environments Bernardo Costa Inês Dutra - PowerPoint PPT Presentation


  • 76 Views
  • Uploaded on

Reinforcement Learning applied to Meta -scheduling in grid environments Bernardo Costa Inês Dutra Marta Mattoso. Outline. Introduction Algorithms Experiments Conclusions and Future work. Introduction Algorithms Experiments Conclusions and Future work. Introduction. Relevance:

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Reinforcement Learning applied to Meta -scheduling in grid environments Bernardo Costa Inês Dutra' - corin


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

ISPA 2008 APDCT Workshop

Reinforcement Learning applied to Meta-scheduling in grid environments

Bernardo Costa

Inês Dutra

Marta Mattoso


Outline

ISPA 2008 APDCT Workshop

Outline

  • Introduction

  • Algorithms

  • Experiments

  • Conclusions and Future work


ISPA 2008 APDCT Workshop

  • Introduction

  • Algorithms

  • Experiments

  • Conclusions and Future work


Introduction

ISPA 2008 APDCT Workshop

Introduction

  • Relevance:

    • Available grid schedulers usually do not employ a strategy that may benefit a single or multiple users.

    • Some strategies employ performance information dependent algorithms (pida).

    • Most works are simulated.

  • Difficulty: monitoring information not reliable due to network latency.


ISPA 2008 APDCT Workshop

  • Introduction

  • Algorithms

  • Experiments

  • Conclusions and Future work


Study of 2 algorithms

ISPA 2008 APDCT Workshop

Study of 2 Algorithms

  • (AG) A. Galstyan, K. Czajkowski, and K. Lerman. Resource allocation in the grid using reinforcement learning. In AAMAS, pages 1314–1315. IEEE, 2004.

  • (MQD) Y. C. Lee and A. Y. Zomaya. A grid scheduling algorithm for bag-of-tasks applications using multiple queues with duplication. 5th IEEE/ACIS International Conference on Computer and Information Science and 1st IEEE/ACIS International Workshop on Component-Based Software Engineering, Software Architecture and Reuse. ICIS-COMSAR, pages 5–10, 2006.


What is reinforcement learning

ISPA 2008 APDCT Workshop

What is reinforcement learning?

  • Machine learning technique used to learn behaviours given a series of temporal events.

  • Non-supervised learning.

  • Based on the idea of rewards and punishments.


Algorithms

ISPA 2008 APDCT Workshop

Algorithms

  • AG and MQD use reinforcement learning to associate an efficiency rank to an RMS.

  • Reinforcement learning native to AG.

  • MQD was modified to use this technique to estimate computational power of an RMS.

  • AG allocates RMS in a greedy and probabilistic way.

  • MQD allocates RMS associatively and deterministically.


Algorithms1

ISPA 2008 APDCT Workshop

Algorithms

  • Calculating efficiency:

    • Reward is assigned to RMS that has performance better than average.

    • Reward can be negative (punishment).

    • RMS may not change its efficiency value.


Algorithms2

ISPA 2008 APDCT Workshop

Algorithms

  • Calculating efficiency:

    • parameters: a and l

    • a is the importance of the time spent executing a task

      • affects rewarding.

    • l is a learning parameter


Algorithms3

ISPA 2008 APDCT Workshop

Algorithms

  • AG:

    • With high prob, associates job to the best available RMS, otherwise, selects randomly.

  • MQD:

    • Groups of jobs sorted according execution time are associated to an RMS. Most efficient executes the heaviest jobs. Initial allocation to estimate RMS´ efficiency



ISPA 2008 APDCT Workshop

R1

E = 0

R2

E = 0

R3

E = 0

J2

J3

J4

J1

J5

J6

J8

J9

J7


ISPA 2008 APDCT Workshop

R1

E = 0

R2

E = 0,3

R3

E = -0,3

J4

J5

J6

J8

J7

J9


ISPA 2008 APDCT Workshop

R1

E = 0,3

R2

E = 0,057

R3

E = 0,51

J8

J7

J9



ISPA 2008 APDCT Workshop

R1

E = 0

R2

E = 0

R3

E = 0

J2

15

J3

50

J4

30

J1

40

J5

10

J6

70

J8

20

J7

20

J9

40


ISPA 2008 APDCT Workshop

R1

E = 0

R2

E = 0

R3

E = 0

J5

10

J2

15

J7

20

J8

20

J4

30

J9

40

J6

70

J3

50

J1

40


ISPA 2008 APDCT Workshop

R1

E = 0,3

R2

E = -0,3

R3

E = 0

J8

20

J4

30

J9

40

J6

70

J3

50

J1

40


ISPA 2008 APDCT Workshop

R1

E = 0,09

R2

E = -0,09

R3

E = -0,3

J8

20

J3

50

J1

40


ISPA 2008 APDCT Workshop

Avg per proc

Global Avg





ISPA 2008 APDCT Workshop

  • Introduction

  • Algorithms

  • Experiments

  • Conclusions and Future work


Experiments

ISPA 2008 APDCT Workshop

Experiments

  • GridbusBroker:

    • No need to install it in other grid sites

    • Only requirement: ssh access to a grid node

    • Round-robin scheduler (RR)

  • Limitations:

    • Does not support job duplication

    • Imposes a limit on the number of active jobs per RMS


Experiments1

ISPA 2008 APDCT Workshop

Experiments

  • Resources in 6 grid sites:

    • LabIA: 24 (Torque/Maui)

    • LCP: 28 (SGE)

    • Nacad: 16 (PBS PRO)

    • UERJ: 144 (Condor)

    • UFRGS: 4 (Torque)

    • LCC: 44 (Torque)


Experiments2

ISPA 2008 APDCT Workshop

Experiments

  • Objective: study performance of algorithms in a real grid environment.

  • Application: bag-of-tasks.

  • CPU intensive.

    • Duration between 3 and 8 minutes.


Experiments3

ISPA 2008 APDCT Workshop

Experiments

  • Evaluation criteria:

    • makespan.

  • Makespan was normalized with respect to RR


Experiments4

ISPA 2008 APDCT Workshop

Experiments

  • Phase I:

    • Tuning of parameters a and l

    • 500 jobs.

  • Phase II:

    • Performance of re-scheduling.

    • Later load increased to 1000 jobs.


Experiments5

ISPA 2008 APDCT Workshop

Experiments

  • One experiment is a run of consecutive executions of RR, AG and MQD.

  • A scenario is a set of experiments with fixed parameters.

  • For each scenario: 15 runs.

  • T-tests to verify statistical difference beteween AG/MQD e RR, with 95% confidence (the results have a normal distribution).



Experiments phase i

ISPA 2008 APDCT Workshop

Experiments (Phase I)


Experiments phase ii

ISPA 2008 APDCT Workshop

Experiments (Phase II)


ISPA 2008 APDCT Workshop

  • Introduction

  • Algorithms

  • Experiments

  • Conclusions and Future work


Conclusions and future work

ISPA 2008 APDCT Workshop

Conclusions and Future work

  • Results showed that was possible to achieve optimizations with both AG and MQD wrt RR

  • Experiments validate MQD simulation results found in the literature.

  • Reinforcement learning is a promising technique to classify resources in real grid environments.


Conclusions and future work1

ISPA 2008 APDCT Workshop

Conclusions and Future work

  • Study the behavior of AG and MQD with other kinds of applications, e.g., data intensive, with dependencies.




Defini es

ISPA 2008 APDCT Workshop

Definições

  • Gerenciador de recursos: sistema que gerencia a submissão e execução de jobs dentro de um domínio específico.

  • Resource Management System (RMS): sinônimo para gerenciador de recursos.

  • Batch job scheduler: escalonador típico de um RMS. Ex: SGE, PBS/Torque.


Defini es1

ISPA 2008 APDCT Workshop

Definições

  • Meta-escalonador: um escalonador que não tem acesso direto aos recursos, mas apenas aos RMS que os gerenciam.

  • Aprendizado por reforço: técnica que induz um agente a tomar decisões por meio de recompensas oferecidas.

  • Makespan: tempo total gasto por um meta-escalonador para finalizar a execução de um conjunto de jobs a ele designado.


Defini es2

ISPA 2008 APDCT Workshop

Definições

  • Job: aplicativo submetido ao grid por um usuário, executado em geral por um RMS. Exemplos de tipos de jobs:

    • Bag-of-Tasks: jobs que não possuem relação de dependência ou precedência explícita entre si.

    • Troca de parâmetros (APST): jobs de um mesmo executável que diferenciam-se por um valor de entrada que varia entre as execuções.




ad