the grid observatory n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
The Grid Observatory PowerPoint Presentation
Download Presentation
The Grid Observatory

Loading in 2 Seconds...

play fullscreen
1 / 35

The Grid Observatory - PowerPoint PPT Presentation


  • 109 Views
  • Uploaded on

The Grid Observatory. Operated by L aboratoire de Recherche en Informatique Laboratoire de l’ Accélérateur Linéaire Imperial College London. With the support of France Grilles – French NGI of EGI EGI-Inspire Ile de France council (Software and Complex Systems programme )

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'The Grid Observatory' - gaia


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
slide2

Operated by

  • Laboratoire de Recherche en Informatique
  • Laboratoire de l’ AccélérateurLinéaire
  • Imperial College London
slide3

With the support of

  • France Grilles – French NGI of EGI
  • EGI-Inspire
  • Ile de France council

(Software and Complex Systems programme)

  • INRIA – Saclay (ADT programme)
  • CNRS (PEPS programme)
  • University Paris Sud (MRM programme)
slide11

Torque

CE

Logging& bookkepping

BDII

IC RTM

WMS

SQL

LDAP

HTTP

SFTP

Incoming

Anonymisation

Upload

Grid ObservatoryPortal

DPM via HTTPs

Storage Elements

On top of EGI monitoring - anonymized

lessons learned
Lessons learned

Sociology

  • Running a production system for usage by computer science and engineeringis nearly unchartered territory – we are a few explorators
  • Verified that 80% of the cost of Data Mining is in pre-processing
lessons learned1
Lessons learned

Technique

  • Build on existing monitoring tools
  • No fancy technology: the goal is usage, not the tool
slide16

The first barrier to improvedenergyefficiencyis the difficulty of collecting data on the energy usage of individualcomponents, and the lack of overall data collection

slide17

The GCO monitors energy usage ata large computing center, and publishesthemthroughthe GridObservatory.

slide18

A second barrierismaking the collecteddata usable, consistent and complete.

GCO adopts an ontologicalapproachin order to rigorouslydefinethe semantics of the data and the context of their production.

the grif lal computing room
The GRIF-LAL computing room

The LAL Computing Room

240 machines, 2200+ cores, 500TB of storage.

Mainly a Tier 2 in the EGI grid, but alsoincludes local services and the StratusLab Cloud testbed

Accessible approximation of a data center

sensors
Sensors

1 minute samplingperiod

slide21

Source: http://www.netways.de/uploads/media/Werner_Fischer_-The-Power-Of-IPMI.pdf

slide33

Dealing non-stationarity

  • Adaptive clustering with application to fault diagnosisToward Autonomic Grids: Analyzing the Job Flow with Affinity Streaming. SIGKDD'2009
  • MDL segmentation applied to workloadDiscovering Piecewise Linear Models of Grid Workload.CCGRID 2010
intelligibility
Intelligibility

How to build knowledge?

  • Supervised learning? No reference, too rare experts
  • Let’s build it on-line! Model-free policies e.g. Reinforcement Learning!
  • Unfortunately, tabula rasa policies and vanilla ML methods are too often defeated [Rish & Tesauro 2006).

Exploration/exploitation tradeoff

slide35

Intelligibility

  • FaultmodelsDistributed Monitoring with Collaborative Prediction. CCGRID 2012
  • Cloud managementCharacterizingE-Science File Access Behavior via Latent Dirichlet Allocation.UCC 2011