1 / 15

Scalarm: Scalable Platform for Data Farming

1. Scalarm: Scalable Platform for Data Farming. D. Król , Ł. Dutka, M. Wrzeszcz, B. Kryza, R. Słota and J. Kitowski. ACC Cyfronet AGH. KU KDM, Zakopane, 2013. Agenda. 2. About Us Problem statement Solution Contact. Who are we ?. 3. Computer Systems Group AGH

vala
Download Presentation

Scalarm: Scalable Platform for Data Farming

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. 1 Scalarm: Scalable Platform for Data Farming D. Król, Ł. Dutka, M. Wrzeszcz, B. Kryza, R. Słota and J. Kitowski ACC Cyfronet AGH KU KDM, Zakopane, 2013

  2. Agenda 2 About Us Problem statement Solution Contact

  3. Whoare we ? 3 Computer Systems Group AGH http://www.icsr.agh.edu.pl/ Knowledge in Grids Team http://www.icsr.agh.edu.pl/index.php/knowledge-in-grids-team Close collaboration with ACC Cyfronet AGH

  4. Whatare we doing ? 4 • Knowledge supported systems for Virtual Organizations • Framework for Intelligent Virtual Organizations • Grid Organizational Memory • X2R • Data monitoring and management • Service Level Agreement Monitoring • QStorMan • Self-scalable systems • Scalarm

  5. A short introduction to data farming 5 Processinitialization

  6. What problem do we address with Scalarm ? 6 Data farming experiments with an exploratory approach Parameter space generation with support of design of experiment methods Accessing heterogeneous computational infrastructure Self-scalability of the management part

  7. Typical work flow 7 2. A workstation or an institution server 3. National Grid environment 1. Local development 4. Cloud

  8. Whereis the problem ? 8 What about fault tolerence? What about collecting results ? 3. Shell + scheduler (gLite, QCG ...) What about parameter space ? 2. Shell Integrated DevelopmentEnvironment 4. Shell + proprietary software

  9. What is our solution ? 9 Specify application to run Execute simulation

  10. Scalarm overview 10 Self-scalable platform adapting to experiment size and simulation type Exploratory approach for conducting experiments Supporting online analysis of experiment partial results Integrates with clusters, Grids, Clouds

  11. Use case – Complete platform for data farming 11

  12. Use case – Simulation-as-a-Service 12 Nativeapp Execute simulation API

  13. But, doesitscale ? 13 Experiment evaluation: • Experiment size [#simulations]: 100 000, 200 000, 500 000, 1 000 000, 2 000 000 • Computational resources [#servers]: 2, 4, 8, 16 • #Clients: 240 * (#servers / 2)

  14. Results 14

  15. Do you want to now more ? 15 Talk to us during the conference Contact us -> dkrol@agh.edu.pl Visit a website -> http://www.scalarm.com

More Related