slide1
Download
Skip this Video
Download Presentation
Reinforcement Learning applied to Meta -scheduling in grid environments Bernardo Costa Inês Dutra

Loading in 2 Seconds...

play fullscreen
1 / 44

Reinforcement Learning applied to Meta -scheduling in grid environments Bernardo Costa Inês Dutra - PowerPoint PPT Presentation


  • 76 Views
  • Uploaded on

Reinforcement Learning applied to Meta -scheduling in grid environments Bernardo Costa Inês Dutra Marta Mattoso. Outline. Introduction Algorithms Experiments Conclusions and Future work. Introduction Algorithms Experiments Conclusions and Future work. Introduction. Relevance:

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Reinforcement Learning applied to Meta -scheduling in grid environments Bernardo Costa Inês Dutra' - corin


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
slide1
ISPA 2008 APDCT Workshop

Reinforcement Learning applied to Meta-scheduling in grid environments

Bernardo Costa

Inês Dutra

Marta Mattoso

outline
ISPA 2008 APDCT WorkshopOutline
  • Introduction
  • Algorithms
  • Experiments
  • Conclusions and Future work
slide3
ISPA 2008 APDCT Workshop
  • Introduction
  • Algorithms
  • Experiments
  • Conclusions and Future work
introduction
ISPA 2008 APDCT WorkshopIntroduction
  • Relevance:
    • Available grid schedulers usually do not employ a strategy that may benefit a single or multiple users.
    • Some strategies employ performance information dependent algorithms (pida).
    • Most works are simulated.
  • Difficulty: monitoring information not reliable due to network latency.
slide5
ISPA 2008 APDCT Workshop
  • Introduction
  • Algorithms
  • Experiments
  • Conclusions and Future work
study of 2 algorithms
ISPA 2008 APDCT WorkshopStudy of 2 Algorithms
  • (AG) A. Galstyan, K. Czajkowski, and K. Lerman. Resource allocation in the grid using reinforcement learning. In AAMAS, pages 1314–1315. IEEE, 2004.
  • (MQD) Y. C. Lee and A. Y. Zomaya. A grid scheduling algorithm for bag-of-tasks applications using multiple queues with duplication. 5th IEEE/ACIS International Conference on Computer and Information Science and 1st IEEE/ACIS International Workshop on Component-Based Software Engineering, Software Architecture and Reuse. ICIS-COMSAR, pages 5–10, 2006.
what is reinforcement learning
ISPA 2008 APDCT WorkshopWhat is reinforcement learning?
  • Machine learning technique used to learn behaviours given a series of temporal events.
  • Non-supervised learning.
  • Based on the idea of rewards and punishments.
algorithms
ISPA 2008 APDCT WorkshopAlgorithms
  • AG and MQD use reinforcement learning to associate an efficiency rank to an RMS.
  • Reinforcement learning native to AG.
  • MQD was modified to use this technique to estimate computational power of an RMS.
  • AG allocates RMS in a greedy and probabilistic way.
  • MQD allocates RMS associatively and deterministically.
algorithms1
ISPA 2008 APDCT WorkshopAlgorithms
  • Calculating efficiency:
    • Reward is assigned to RMS that has performance better than average.
    • Reward can be negative (punishment).
    • RMS may not change its efficiency value.
algorithms2
ISPA 2008 APDCT WorkshopAlgorithms
  • Calculating efficiency:
    • parameters: a and l
    • a is the importance of the time spent executing a task
      • affects rewarding.
    • l is a learning parameter
algorithms3
ISPA 2008 APDCT WorkshopAlgorithms
  • AG:
    • With high prob, associates job to the best available RMS, otherwise, selects randomly.
  • MQD:
    • Groups of jobs sorted according execution time are associated to an RMS. Most efficient executes the heaviest jobs. Initial allocation to estimate RMS´ efficiency
slide13
ISPA 2008 APDCT Workshop

R1

E = 0

R2

E = 0

R3

E = 0

J2

J3

J4

J1

J5

J6

J8

J9

J7

slide14
ISPA 2008 APDCT Workshop

R1

E = 0

R2

E = 0,3

R3

E = -0,3

J4

J5

J6

J8

J7

J9

slide15
ISPA 2008 APDCT Workshop

R1

E = 0,3

R2

E = 0,057

R3

E = 0,51

J8

J7

J9

slide17
ISPA 2008 APDCT Workshop

R1

E = 0

R2

E = 0

R3

E = 0

J2

15

J3

50

J4

30

J1

40

J5

10

J6

70

J8

20

J7

20

J9

40

slide18
ISPA 2008 APDCT Workshop

R1

E = 0

R2

E = 0

R3

E = 0

J5

10

J2

15

J7

20

J8

20

J4

30

J9

40

J6

70

J3

50

J1

40

slide19
ISPA 2008 APDCT Workshop

R1

E = 0,3

R2

E = -0,3

R3

E = 0

J8

20

J4

30

J9

40

J6

70

J3

50

J1

40

slide20
ISPA 2008 APDCT Workshop

R1

E = 0,09

R2

E = -0,09

R3

E = -0,3

J8

20

J3

50

J1

40

slide21
ISPA 2008 APDCT Workshop

Avg per proc

Global Avg

slide25
ISPA 2008 APDCT Workshop
  • Introduction
  • Algorithms
  • Experiments
  • Conclusions and Future work
experiments
ISPA 2008 APDCT WorkshopExperiments
  • GridbusBroker:
    • No need to install it in other grid sites
    • Only requirement: ssh access to a grid node
    • Round-robin scheduler (RR)
  • Limitations:
    • Does not support job duplication
    • Imposes a limit on the number of active jobs per RMS
experiments1
ISPA 2008 APDCT WorkshopExperiments
  • Resources in 6 grid sites:
    • LabIA: 24 (Torque/Maui)
    • LCP: 28 (SGE)
    • Nacad: 16 (PBS PRO)
    • UERJ: 144 (Condor)
    • UFRGS: 4 (Torque)
    • LCC: 44 (Torque)
experiments2
ISPA 2008 APDCT WorkshopExperiments
  • Objective: study performance of algorithms in a real grid environment.
  • Application: bag-of-tasks.
  • CPU intensive.
    • Duration between 3 and 8 minutes.
experiments3
ISPA 2008 APDCT WorkshopExperiments
  • Evaluation criteria:
    • makespan.
  • Makespan was normalized with respect to RR
experiments4
ISPA 2008 APDCT WorkshopExperiments
  • Phase I:
    • Tuning of parameters a and l
    • 500 jobs.
  • Phase II:
    • Performance of re-scheduling.
    • Later load increased to 1000 jobs.
experiments5
ISPA 2008 APDCT WorkshopExperiments
  • One experiment is a run of consecutive executions of RR, AG and MQD.
  • A scenario is a set of experiments with fixed parameters.
  • For each scenario: 15 runs.
  • T-tests to verify statistical difference beteween AG/MQD e RR, with 95% confidence (the results have a normal distribution).
slide35
ISPA 2008 APDCT Workshop
  • Introduction
  • Algorithms
  • Experiments
  • Conclusions and Future work
conclusions and future work
ISPA 2008 APDCT WorkshopConclusions and Future work
  • Results showed that was possible to achieve optimizations with both AG and MQD wrt RR
  • Experiments validate MQD simulation results found in the literature.
  • Reinforcement learning is a promising technique to classify resources in real grid environments.
conclusions and future work1
ISPA 2008 APDCT WorkshopConclusions and Future work
  • Study the behavior of AG and MQD with other kinds of applications, e.g., data intensive, with dependencies.
defini es
ISPA 2008 APDCT WorkshopDefinições
  • Gerenciador de recursos: sistema que gerencia a submissão e execução de jobs dentro de um domínio específico.
  • Resource Management System (RMS): sinônimo para gerenciador de recursos.
  • Batch job scheduler: escalonador típico de um RMS. Ex: SGE, PBS/Torque.
defini es1
ISPA 2008 APDCT WorkshopDefinições
  • Meta-escalonador: um escalonador que não tem acesso direto aos recursos, mas apenas aos RMS que os gerenciam.
  • Aprendizado por reforço: técnica que induz um agente a tomar decisões por meio de recompensas oferecidas.
  • Makespan: tempo total gasto por um meta-escalonador para finalizar a execução de um conjunto de jobs a ele designado.
defini es2
ISPA 2008 APDCT WorkshopDefinições
  • Job: aplicativo submetido ao grid por um usuário, executado em geral por um RMS. Exemplos de tipos de jobs:
    • Bag-of-Tasks: jobs que não possuem relação de dependência ou precedência explícita entre si.
    • Troca de parâmetros (APST): jobs de um mesmo executável que diferenciam-se por um valor de entrada que varia entre as execuções.
ad