Deployment of e-Infrastructures for Scientific Communities in SEE region

1. SEE-GRID-SCI Ognjen Prnjat Towards SEE-GRID-SCI project proposal

2. Where and when and how and how much • Call INFRA-2007-1.2.2: Deployment of e-Infrastructures for scientific communities • Work Programme topics addressed: Integrating Activities – ICT-based eInfrastructures – Deployment of eInfrastructures for scientific communities • Deadline: 2nd May • Instrument: Combination of Collaborative Project and Coordination and Support Action: Integrated Infrastructure Initiative: NAs, SAs, JRAs • Budget: Around 3M Euro, same country funds + extra countries + JRA funds + apps funds + Moldova link SEE-GRID-SCIPreparatory Meeting – Kopaonik, Serbia, 13.03.2007 2

3. Who • As before + Armenia and Georgia SEE-GRID-SCIPreparatory Meeting – Kopaonik, Serbia, 13.03.2007 3

4. And why • Ognjen’s realistic assessment: • New dimension: regional applications • Greece coordination • Armenia, Georgia: new • Albania, Bosnia, Moldova, Montenegro, FYRO Macedonia: not on EGEE level yet • Serbia, Turkey: strong operational / applications support, could get involved in JRAs. Add value to regional operation support • Bulgaria, Romania: should get involved more with JRAs and applications • RBI, SZTAKI: must get involved with something, probably with applications. CE federation overlap SEE-GRID-SCIPreparatory Meeting – Kopaonik, Serbia, 13.03.2007 4

5. Objectives • O1: Engaging regional user communities and providing application-specific middleware extensions • O2: Providing the infrastructure for new communities • Enlarging • AM and GE • Operational tools extensions • O3: Finalising actions towards long-term sustainability and European Grid Organisation inclusion • O4: Strengthening the regional and national Human Network SEE-GRID-SCIPreparatory Meeting – Kopaonik, Serbia, 13.03.2007 5

6. Structure SEE-GRID-SCIPreparatory Meeting – Kopaonik, Serbia, 13.03.2007 6

7. NA2: NGIs • Non-EGEE country focus • Ensure the official establishment of the National Grid Initiatives in countries which have not one established yet. • Carry out the correct structuring of nascent and existing NGIs so as to achieve stable operational and organisational structure compatible with emerging European Grid Organisation. • Monitor NGI status in the region. • Provide guidelines for NGI organization and functioning and make them available beyond the consortium. • Liaise with related regional projects and European initiatives for exchanging know-how in terms of NGI developments, applications, operations, application services and operational tools development. • Organise a concertation event focused on NGI know-how exchange focused at the peripheral and outmost regions. • Main partners: GRNET + ICI + all SEE-GRID-SCIPreparatory Meeting – Kopaonik, Serbia, 13.03.2007 7

8. NA3: training • Design, develop, evaluate and maintain a content-rich portal for all of the SEE-GRID-SCI related coordination, research, dissemination processes, results and information. • Provide the tools for communication and dissemination of information about SEE-GRID-SCI • Promote widespread use of the SEE-GRID-SCI infrastructure by all members and all disciplines of the SEE community • Carry out training and dissemination events at regional as well as national level • Provide and maintain a portal for dissemination and training events • Main partners: SZTAKI SEE-GRID-SCIPreparatory Meeting – Kopaonik, Serbia, 13.03.2007 8

9. NA4: applications • Conduct a survey of regional Grid communities and applications and their requirements • Support regional applications’ gridification, deployment and usage • Ensure multi-disciplinary Grid environment • Main partners: depending on applications. TUBITAK lead. Alternative solutions possible. SEE-GRID-SCIPreparatory Meeting – Kopaonik, Serbia, 13.03.2007 9

10. NA4: MAIN POINTS • Applications should have REGIONAL impact i.e. have regional beneficiaries defined • Concept: one regional application per country • Grid partner main support for the application • Application group 3rd party • Other countries apps groups can be either: • Beneficiaries unfunded • Collaborators with application groups, funded, 3rd party to main country contractor • Applications bringing Grid resources have advantage [unless part of NGI with defined sharing policy] • Must understand BEFOREHAND the applications’ requirements on the infrastructure • Risk 1: shortage of resources • Risk 2: licensing SEE-GRID-SCIPreparatory Meeting – Kopaonik, Serbia, 13.03.2007 10

11. NA4 and ESFRI (AnetaK) • Idea: ESFRI projects support: LIFE WATCH • 1) ALTER net. Romania: Departments of Systems Ecology, University of Bucharest. Hungary: Institute of Ecology and Botany, Hungarian Acad.of Sci • 2) EUR-OCEANS. Greece - Hellenic Center for Marine Research Turkey - Middle-East Technical University, Institute of Marine Sciences • 3)ENBI, GR, HU, RO • CLARIN(Common Language Resources and Technology Infrastructure): most countries • Action: all to investigate if there is ESFRI project activity in their countries SEE-GRID-SCIPreparatory Meeting – Kopaonik, Serbia, 13.03.2007 11

12. SA1: operations • Develop the next-generation SEE-GRID infrastructure • Support the deployment and operations of the Resource Centres • Network resource provision and assurance • 200k for DF link to Moldova but can start only 2008 – Peter, is this good timing? • Separate proposals for SEEREN3 + Caucasus • Resource Centre SLA monitoring and enforcement • Issue: CRO and HU less infrastructure involvement • Main partners: All, UoB, TUBITAK SEE-GRID-SCIPreparatory Meeting – Kopaonik, Serbia, 13.03.2007 12

13. SA1: main points • Usual 2-layered approach • Application incubation in SEEGRID • Deployment *in production* over both SEEGRID and EGEE-SEE (computing and storage resources) • Operational support + core services by SEEGRID • ISSUE: which partners operate core services for new VOs – this must be clear beforehand SEE-GRID-SCIPreparatory Meeting – Kopaonik, Serbia, 13.03.2007 13

14. JRA1: apps add-ons • Capture commonalities across scientific fields in terms of application requirements on Grid middleware • Define development areas for middleware plug-ins to cater to application demands • Implement application-specific middleware extensions • Issue: EGEE link • Main partners: UoB, TUBITAK (?), CERN(?) SEE-GRID-SCIPreparatory Meeting – Kopaonik, Serbia, 13.03.2007 14

15. JRA2: operational tools development • Provide a benchmark of operational tools from the perspective of application and end-user friendliness and define development areas • Develop and deploy operational tools easing the application-specific monitoring and control • Develop installation and configuration scripts and tools which support application support and prioritization agreements. • SLA developments here instead of SA1? • Main partners: IPP, ICI, CERN, UoBL SEE-GRID-SCIPreparatory Meeting – Kopaonik, Serbia, 13.03.2007 15

16. Extra • SME involvement • Some initial equipment possible for GE and AM – do others really need it? SEE-GRID-SCIPreparatory Meeting – Kopaonik, Serbia, 13.03.2007 16

17. Financial issues: cost models • Full Cost model: You can declare all your actual direct costs + your actual indirect costs (= overheads). MUST BE CALCULATED. IF NOT: • Full Cost with Flat rate for overheads: actual direct costs + a flat rate for indirect costs (20% of direct costs minus subcontracting) • If Universities (secondary and higher education establishments), non-profit research institutions, non-profit public bodies or SMEs claim that they cannot calculate their indirect costs: 60% flat rate for indirect costs • SAs and NAs indirect costs 7%, others as according to cost model • MUST confirm cost model and o/h rate WITH AN AUDITOR SEE-GRID-SCIPreparatory Meeting – Kopaonik, Serbia, 13.03.2007 17

18. Financial issues: EC funding of direct costs • Not same for different activities • (*75% of their direct costs is reimbursed only for Universities (secondary and higher education establishments), non-profit research institutions, non-profit public bodies and SMEs) SEE-GRID-SCIPreparatory Meeting – Kopaonik, Serbia, 13.03.2007 18

19. More financial issues • Must provide us with PM cost (for those who haven't) • Start thinking what your 3rd parties will have budgeted for them, count more-less the same budget as SEEGRID2 SEE-GRID-SCIPreparatory Meeting – Kopaonik, Serbia, 13.03.2007 19

20. Admin issues • Letters of intent: partner, ministry • Must register with EPSS when Valia says so • A2 forms (?) • Your profiles 1 page • JRU: yes or no? • Action: Gabriel to send guidelines • Valia to check TP vs JRU mixed model clash SEE-GRID-SCIPreparatory Meeting – Kopaonik, Serbia, 13.03.2007 20

21. And finally • Time is short • Competition is fierce • Immediate replies are required SEE-GRID-SCIPreparatory Meeting – Kopaonik, Serbia, 13.03.2007 21

22. SA1 presentation • Antun • SA1 Team for writing and revision: Antun, Boro, Ioannis, Emanouil SEE-GRID-SCIPreparatory Meeting – Kopaonik, Serbia, 13.03.2007 22

23. SA1 objectives (1) • General objective: Providing the infrastructure for new communities • Sub-objective 1: Provision of the next-generation Grid infrastructure for the SEE region • to expand the current infrastructure, in order to serve the existing and new user communities, by utilising the model of the 2 layered infrastructures where the mature sites will be moved to EGEE production infrastructure while the fresh sites from new institutes and user communities will be incubated within the SEE-GRID infrastructure • this approach ensures that smaller sites, typical for the region, will have a chance to be a part of the regional SEE-GRID infrastructure acting as an incubator for their maturing into EGEE production • Sub-objective 2: Inclusion of the Caucasus region in the infrastructure: Armenia and Georgia SEE-GRID-SCIPreparatory Meeting – Kopaonik, Serbia, 13.03.2007 23

24. SA1 objectives (2) • Sub-objective 3: Operate the regional Grid infrastructure • Use of the state-of-the-art management tools developed in the previous 2 phases either by both SEE-GRID and EGEE • Deployment of new operational tools, developed within the JRA2 • The functionality and reliability of the tools such as monitoring, static databases, helpdesk, etc. will be further improved • Full tools integration and automation • SLA enforcement: Overall availability over 80% • Sub-objective 4: Provision of the network link to Moldova • To cater for immediate connection of this country to Romania and thus to the rest of the region SEE-GRID-SCIPreparatory Meeting – Kopaonik, Serbia, 13.03.2007 24

25. SA1 activities • SA1.1: Implementation of the advanced SEE-GRID infrastructure • Should deal with support for configuration, deployment and operations of the RCs within the SEE-GRID pilot infrastructure, as well as transition of mature RCs into EGEE • SA1.2: Network Resource Provision • Should supports liaison actions to ensure adequate network provision; it will also specifically deal with procurement of a link between Moldova and GEANT • SA1.3: Resource Centre Service Level Agreement monitoring and enforcement • Should ensure high reliability and performance of the infrastructure by constantly monitoring and enforcing the chosen level of service SEE-GRID-SCIPreparatory Meeting – Kopaonik, Serbia, 13.03.2007 25

26. SA1 deliverables • DSA1.1a: Infrastructure Deployment Plan • UoB-IPB, M04, R, PU • DSA1.2: SLA detailed specification and related monitoring tools • UoBL, M05, R, PU • DSA1.1b: Infrastructure Deployment Plan • UoB-IPB, M14, R, PU • DSA1.3: Infrastructure overview and assessment • UKIM, M23, R, PU SEE-GRID-SCIPreparatory Meeting – Kopaonik, Serbia, 13.03.2007 26

27. SA1 milestones • MSA1.1: Infrastructure Deployment Plan Defined • M04 • MSA1.2: SLA structure and enforcement plan defined • M05 • MSA1.3: Infrastructure performance and usage assessed • M05 SEE-GRID-SCIPreparatory Meeting – Kopaonik, Serbia, 13.03.2007 27

28. SA1 Open issues • Monitoring of various, just partially overlapping infrastructures • National • Regional • EGEE-wide • Operations • Helpdesks at the national level? • Interoperation of operations support systems • User support • VOs – local, national, regional, field-oriented • Running of application across various, just partially overlapping, infrastructures • BDII issues SEE-GRID-SCIPreparatory Meeting – Kopaonik, Serbia, 13.03.2007 28

29. JRA1 presentation • Presentation by Branko • Team: Branko, Zorko, Alex SEE-GRID-SCIPreparatory Meeting – Kopaonik, Serbia, 13.03.2007 29

30. JRA1: Development of application-level services • Description of work • JRA1.1 – Application requirements analysis • JRA1.2 – Implementation of application-specific plug-ins • JRA1.3 – Application extensions deployment and assessment • Deliverables • DJRA1.1 – Grid application requirements commonalities assessment - M06 • DJRA1.2 – Application plug-ins design report - M12 • DJRA1.3 – Application plug-ins implementation and deployment report – M22 • Milestones • MJRA1.1 Application commonalities assessed and areas of development identified - M04 • MJRA1.2 Selected application plug-ins implemented and deployed - M24 SEE-GRID-SCIPreparatory Meeting – Kopaonik, Serbia, 13.03.2007 30

31. JRA1: Crosscutting application concerns • Two in-advance identified key commonalities in meteorology, seismology, and health sciences: • Visualization of acquired or calculated spatial data • Data storage service facilitating collaboration and resource sharing • Activities • Assessment of target applications’ requirements and available components and solutions; • Definition of goals and scope; specification of key features, functional requirements and draft interfaces; • Plan for reuse, adaptation, and modification of existing solutions and development of necessary additions; detailed specification and assignment of related activities; • Building of solution components and development documentation; producing functional specifications and testing procedures; • Testing in a realistic environment with pilot applications; handling of detected problems; review of development and deployment documentation; • Deployment of production-level solutions, starting operational usage and support; employment with all interested communities and target applications; assessing user experiences and accomplished results; SEE-GRID-SCIPreparatory Meeting – Kopaonik, Serbia, 13.03.2007 31

32. JRA2 presentation • Emanouil/Todor • Team: Emanouil, Alex Stanciu, Antun SEE-GRID-SCIPreparatory Meeting – Kopaonik, Serbia, 13.03.2007 32

33. JRA2: operational tools development • Will focus on improving the existing SEE-GRID operational tools in order to enhance the infrastructure performance and automation • Objectives: • Provide a benchmark of operational tools from the perspective of application and end-user friendliness and define development areas • Develop and deploy operational tools easing the application-specific monitoring and control • Develop installation and configuration scripts and tools which enhance applications support and prioritization agreements Milestones: • MJRA2.1 – Benchmark of operational tools – M03 • MJRA2.2 – Operational tools implemented and running – M16 SEE-GRID-SCIPreparatory Meeting – Kopaonik, Serbia, 13.03.2007 33

34. JRA2: (continued) • DJRA2.1 Application specific installation and configuration scripts design - M06 IPP • DJRA2.2 Application specific monitoring and control tools design - M08 IPP • DJRA2.2 Assesment of operation tools developed M22 ICI SEE-GRID-SCIPreparatory Meeting – Kopaonik, Serbia, 13.03.2007 34

35. JRA2: operational tools development • Integrate the various kinds of sensors available, simplify deploymentof application specific sensors, and automate the alarms generation.Automate the short-circuiting of sites with bad SLA conformance and theability of applications to specify their own quality standards(integrated with HGSM). Integrate nagios with HGSM and enable more flexibility, especially with respect to application specific requirements and alarms. • Accounting. Emphasis onapplication-specific performance monitoring. Accounting should be enhanced with performancemetrics, that measure quality of the grid service, instead of thecurrent aggregated numbers. R-GMA is a distributed relationaldatabase and thus is well suited for collecting such information,although some changes to the middleware (or light-weight job-wrappers) may be required. Integration of info from gridice may be agood start. SEE-GRID-SCIPreparatory Meeting – Kopaonik, Serbia, 13.03.2007 35

36. JRA2: (continued) • (TO BE MOVED TO SA1) Development of installation / configuration scripts to supportsite-NGI-SEEGRID SLA/application support/prioritization agreements, taking into account the declared availability (full, night-only, day-only). The scripts should also support strengthened firewall configuration, which takes into account the requirements of the supported applications. • Web-based site-admin console with self-healing capability – A GUI that helps site admin to administrate/troubleshoot the site, and performs some simple self-healing operations (like restarting services that failed or got stuck), integrated with the monitoring/helpdesk tools. In case of failures it can create a ticket for the site administrator and point to the relevant documentation links or propose the right command to be issued. It should be a single point of reference for the site admin to verify the correctness of site setup. SEE-GRID-SCIPreparatory Meeting – Kopaonik, Serbia, 13.03.2007 36

37. NA4: detailed applications presentations • Applications presented: • Meteorological, Greece lead • Satellite image processing for ESR: Romania lead • Emergency response for toxic substances: Bulgaria lead • Seismology data acquisition with different applications: Turkey lead • ?Air pollution Hungary • ?BioMed Hungary • ?Ecology/geometeo: Moldova • Presentations FYROM, Moldova converging with one of the above SEE-GRID-SCIPreparatory Meeting – Kopaonik, Serbia, 13.03.2007 37

38. Regional Meteo application • Lead: National Observatory of Athens (3rd party to GRNET) • Other partners: SEWA (UoB), Montenegro meteorological institute (UoM). • Resources to be brought in: HG clusters with %, MMI (18 nodes?), SEWA (12 nodes) • Additional beneficiaries: other meteo communities • App: BOLAM model and MM5 model; ensemble prediction • Different requirements • Licensing issue: BOLAM • WHAT IS THE INPUT DATA? SEE-GRID-SCIPreparatory Meeting – Kopaonik, Serbia, 13.03.2007 38

39. Regional Meteo application cont • Numerical weather prediction • Traditional approach: crank the model of choice and get numbers • Problem: uncertainties related to the chaotic behavior of the atmosphere • Solution: • Instead of running a single model (deterministic forecast), run a multitude – an ensemble – of models starting from slightly different conditions (multi model ensemble forecasting) • Assess the probability of a particular weather event to occur • Requires large infrastructure, ideal for Grid • This approach has been applied worldwide, but not at the regional SEE level; will be innovative • Minimal requirements • ~15 clusters, 8 CPUs each, (MPI implementation) at about noon time for 2-3 hours • 2 GB disk space per cluster • Beneficiaries: • SEE academic meteorological communities • authorities • general public SEE-GRID-SCIPreparatory Meeting – Kopaonik, Serbia, 13.03.2007 39

40. Regional Meteo application cont • Mini-consortium/key personnel • Vassiliki Kotroni,Institute of Environmental Research, National Observatory of Athens, Athens • Momcilo Zivkovic,South European Weather Agency, Belgrade • Radivoje Vuckovic,Hydrological and Meteorological Service of Montenegro, Podgorica • New resources brought in by the application • NOA: % of HG clusters • SEWA: cluster 12 CPUs • HMSM: cluster 4 CPUs (14 more CPUs under procurement) SEE-GRID-SCIPreparatory Meeting – Kopaonik, Serbia, 13.03.2007 40

41. Application Actions • The following partners to send ½ page description of VOs/applications and consortia (including beneficiary F/UF, apps requirements) + regional dimension • Greece: Meteo with Montenegro and Serbia • Romania: Satellites with maybe Moldova and Hungary • Turkey: a seismo overview including a number of applications (FYROM, Moldova, Greece, Romania, Hungary, Albania, Bulgaria) • Bulgaria: emergency release response/pollution with Albania and maybe Turkey • Hungary: also air/pollution with Bulgaria + TR? • [Hungary: and Turkey and Croatia and maybe Macedonia: BioInformatics] needs regional justification • ?Moldova: ecology/geometeo with Bulgaria • ?Archeology Serbia? • ?Metheopaty Croatia? Concrete proposal needed + regional relevance • Croatia and Bosnia totally without proposals so far! SEE-GRID-SCIPreparatory Meeting – Kopaonik, Serbia, 13.03.2007 41