SA1 Planning

1. 2 SA1 Planning Alberto Aimar CERN – ETICS2 SA1

2. SA1 – Deliverables (Dates-20) • DSA1.1 – Execution plan for first 12 months of infrastructure operation M03 • This deliverable describes the execution plan for the first half of the ETICS 2 project, including the core service roadmap and the infrastructure deployment plan. • DSA1.2 – ETICS Core Services Design Specification M06 • This deliverable describes the overall architecture of the ETICS 2 core services. • DSA1.3 – ETICS Site Service Level Agreement M09 • This deliverable describes the Service Level Agreements upon which the ETICS service will be provided. The SLAs will define the service level the users can expect from the service in terms of availability, accessibility and support. • DSA1.4 – Execution plan for second 12 months of infrastructure operation M12 • This deliverable describes the execution plan for the second half of the ETICS 2 project, including the core service roadmap and the infrastructure deployment plan. • DSA1.5 – Infrastructure and core services certification and usage report M21 • This deliverable reports on the release management cycles and certification of the ETICS 2 infrastructure and core services, including lessons learned and corrective action to apply. • + QR (Every quarter – 20)

3. SA1 Project Setup

4. Organization • SA1 Internal Wiki - Repository of the project • https://cern.ch/twiki/bin/view/ETICS/SA1_internal • Nothing secret, but is not linked from outside • Could be made public • Internal documents, work in progress, meeting minutes, etc • Weekly Meetings • Prepared in advance with a short summary of progress and Issues • Minutes available on the SA1 Internal Wiki • Everybody should participate will be a phone meeting, not located to CERN • Action List • Maintained in Savannah • Specific ETICS 2 SA1Project

5. SA1 Releases • Have a production setup, we need to easily be able to bring it back up in case of problems (semi-automatic) • An identical test setup for the release candidate, as soon as we release (after thorough testing) we prepare for the next release • Have a realistic test bench that we control and can use for regression testing (a snapshot of gLite for instance) • Release about once a month, when we will have automated many current manual procedures • ETICS must “Walk the walk, not talk the talk”

6. Participants

7. Day-to-Day Activities User Support Bug tracking Testing Design and Development Release Testing (again and again) Releasing Automating • >>CLI Client (Python) • >>Config GUI (Web App) • >>Web Portal • Repository (DB and Web) • >>Web Services • ETICS Central Services/Infrastructure • >>Execution Infrastructure(s)

8. Short Term/Urgent Goals • Hiring (all partners, INFN and Eng) M2 • Test Infrastructure M1 • Infrastructure for Devmnt M1+15 • Automate Installations M2 • Current Release (to be tested) M1 • Next Release (speed improvements) M2 • Release (client improvements) M4

9. SA1 Services Release Testing M4 • Test infrastructure (hw) • Prepare a testbench (data) • Test Execution (manual or automatic)

10. To Discuss • User Support,… SA2 • Integrate Past Development SA2 • Model for Future Development/Integration SA2, JRA2 • Experience in Monitoring Infrastructure SA2

11. ETICS 2 SA1 Description • SA1 is responsible for maintaining the ETICS core software and computing infrastructures required to provide a high quality, high availability build and test service. SA1 will also integrate, test and validate components and contributions coming from other activities (e.g. SA2, JRA1 and JRA2) in the core functionality. (When needed, stating M4) • The focus of the Work Package is to maintain a sound foundation on which the reliability of the ETICS Service is built. When required, SA1 will extend the core services, when these extensions affect all users and cannot be performed by SA2 as a customisation of existing features or via an extensibility mechanism, such as the plugin framework. (On-Going, start M4)

12. The ETICS infrastructure, operated by SA1 will continue to provide a distributed high quality, high availability public infrastructure on which software can be built and tested. Procedures to join private resources in a controlled manner to be used in confidentiality sensitive applications will be developed and deployed. (M??) Building on the procedures defined and exercised by the first ETICS project WP2 work package, these will be further automated and integrated. • The self-monitoring of the infrastructures will also be extended to trigger alerts to operators upon detection of a malfunction of the infrastructure, including hardware and software failures. (M6) The infrastructure will also include the hosting of the ETICS service metadata store (a centralized database) and the integration of federated repositories in which build and test artefacts (e.g. installable packages, test results) can be stored.

13. In order to accommodate the growth in terms of number of official ETICS sites, SA1 will revise and extend existing SLAs (Service Level Agreement) (M12??) for all sites to comply with in order for build and test results to be trusted by the ETICS system. The SLAs will establish what users can expect from the service in terms of availability, accessibility and user support and will provide a base agreement against which the provision of outsourced services can be measured. • Security is also an aspect of the ETICS service that will require constant monitoring, (Initial M12, then M18??)especially when the system will start to be used by commercial companies of project with closed or sensitive IPR issues, both in terms of source code and test data. • CERN, INFN and UoW will remain the main providers of infrastructure resources, while new sites will be welcome, both in terms of only providing computing resources or running the entire ETICS software stack, interconnected to the public ETICS infrastructure. CERN will continue to provide the main ETICS configuration information database.

14. SA1 will also continue to improve the ETICS tools for providing users with the ability to dynamically deploy multi-node testbeds, potentially spanning multi-sites, in order to replicate real-life deployment scenarios on which to perform automated system tests. (M9??) In addition the existing tools will be expanded to include software release management tools (Basic Release M8, Workflow etc if needed M16) integrated with the build and test service and the repository interfaces and APIs to connect and federate the ETICS repository with other middleware repositories. • As more functionality is integrated in the system, overall performance will have to be improved, such that the system remains responsive under load and that the different tools (Basic M6 Multinode M15) involved in build and test sequences provide good response time. SA1 will also maintain and improve user documentation of the core services and administrator guides for the installation of ETICS services (Constant activitiy,Start M4)

15. The core software of ETICS that will be maintained by SA1 including the client and server side of the current ETICS software stack: • Build and test web service • Repository web service • Command-line clients foundations • Web clients: Build and test, administration and repository • The command-line client plugins wrapping tools for the calculation of metrics and specialisedplugins will also be developed and maintained by other activities in the project (SA2 and JRA2). The ETICS-proposed Certification Model will be applied to the ETICS core services,(When needed, start M4)in order to validate the approach and verify that the adequate tooling is available by ETICS to support the process.

16. SA1 To/From Other Activities

17. Internal Relationships

18. NA2 - Dissemination, Training and Certification • Coordinator: Engineering • Participants: CERN, INFN, 4DSOFT, VEGA • Objectives: • Increase awareness and knowledge about the ETICS system and its service by delivering dissemination and training events • Make the ETICS Grid Quality Certification Model known and accepted by wide audiences of users by promoting it and integrating it with other established quality assurance procedures • Speed up the creation on business activities by analysing business cases, charging models and service level agreements and possibly implementing one case study example in collaboration with commercial enterprises within or outside the consortium M6: Grid QCM Plugins M11: Grid QCM - Certify SA1

19. SA1 – Service Management • Coordinator: CERN • Participants: INFN, ENG, 4DSOFT, Wisconsin, SZTAKI • Objectives: • Maintain and extend existing ETICS core functionality, including documentation • Improve security of core services, such that sites autonomy and security is preserved, while the integrity of the overall eInfrastructure is ensured • Delivery of federated and secure software repositories and release management tools • Automation of the core service monitoring system (e.g. alerts should be triggered when monitored parameters exceed expected thresholds) • Provide second level user support including systematic tracking of support tickets • Regularly review and integrate extensions from SA2 and community contributions and results from JRA activities • Assessment and implementation of scalability strategies to ensure high availability of core services and infrastructure • Apply ETICS Certification Process to ETICS core services. The core services should pass the certification level 2 at the end of year 1 and level 3 at the end of year 2

20. SA2 – Infrastructures Support • Coordinator: INFN • Participants: 4DSOFT, SZTAKI, FZJ, VEGA • Objectives: • Speed up the extension of the ETICS build, test and certification services to new infrastructures, starting with the DEISA and the Aerospace Engineering community • Assist ETICS infrastructure providers in deploying and using the ETICS services efficiently to provide software development services to their users by providing first-level user support, escalation procedures and an on-line knowledge base of frequently asked questions and answers • Improve the functionality and efficiency of the ETICS system by collecting and organizing user requirements, implementing methods and tools to satisfy them, assisting in the integration of ETICS within existing services, analysing the ETICS performance and usability ‘on the field’ with respect to efficiency, scalability and fault-tolerance and defining possible improvement strategies and prototype implementations • Produce a set of guidelines for new infrastructures to facilitate the migration to ETICS and lower the initial learning curve

21. JRA1 – Testbed Management Technologies • Coordinator: WISCONSIN • Participants: ENG? • Objectives: • Develop technologies to allow efficient and secure federation of resource sites • Develop or integrate virtualization technologies in the job management middleware and in the system management tools to lower the local site maintenance costs, increase security and enable new operational models, such as the sharing or pre-configured virtual images or the off-line analysis of build and test failures • Modify the job submission middleware and the ETICS system management tools to run on IPv6 networks

22. JRA2 – Test Management Tools • Coordinator: 4D Soft • Participants: CERN, SZTAKI • Objectives: • Develop the necessary test interface standards for proper collaboration of testing and static analysis tools • Develop a tool by which the testing tools providers can register their tools as Web Services to be integrated into the ETICS testing infrastructure • Develop an interface for the users to construct their testing process in a convenient way • Develop a test execution system to validate the test sequence and generate a specification for the ETICS Execution Engine • Provide the means for users to visualise their test results

23. External Relationships • Infrastructures: • EGEE • DEISA • Projects • EGEE/gLite • D4Science • Aerospace Community • LCG/Physics experiments (CERN) • Collaboration and Standardization Bodies • OGF (US and Europe) • ETSI • ISO

24. SA1 - Objectives • Maintain and extend existing ETICS core functionality, including documentation • Improve security of core services, such that sites autonomy and security is preserved, while the integrity of the overall eInfrastructure is ensured • Delivery of federated and secure software repositories and release management tools • Automation of the core service monitoring system (e.g. alerts should be triggered when monitored parameters exceed expected thresholds) • Provide second level user support including systematic tracking of support tickets • Regularly review and integrate extensions from SA2 and community contributions and results from JRA activities • Assessment and implementation of scalability strategies to ensure high availability of core services and infrastructure • Apply ETICS Certification Process to ETICS core services. The core services should pass the certification level 2 at the end of year 1 and level 3 at the end of year 2

25. SA1 - Tasks • SA1.1 – Work Package coordination (Task leader: CERN) • This task deals with the regular coordination of the Work Package, reporting and review of milestones and deliverables. • SA1.2 – Core service maintenance and extensions (Task leader: CERN – Participants: INFN, ENG, 4DSOFT, UoW) • This task deals with the maintenance of existing ETICS core services, taking into account reported bugs and new requirements relevant to the core services. In particular this task includes the design and implementation of integrated release management tools, of the federated repository interfaces and APIs. This task also includes the integration into the core services of the results of the JRA activities, as well as contributions from SA2 which have sufficient scope to mandate their inclusion as core functionality. • SA1.3 – Core service documentation (Task leader: CERN – Participants: INFN, ENG, 4DSOFT, UoW, SZTAKI) • This task deals with the maintenance and updating of the core service documentation, including: User Guide and Administrator User Guide.

26. SA1.4 – Infrastructure Deployment, Maintenance and Upgrades (Task leader: CERN – Participants: INFN, UoW) • This task deals with extending the current deployment strategies of ETICS services and infrastructure management, such that the service maintains a high-level of quality and availability, under load, throughout the ETICS official public sites. This task will also include load-balancing deployment for solicited and/or critical services, as well as improving the coverage of the self-monitoring system of the deployed services and underlying infrastructures. • SA1.5 – Core service certification (Task leader: CERN – Participants: INFN, ENG) • This task deals with the applying the proposed ETICS Certification Process to the ETICS software itself, in order to demonstrate its applicability and that the ETICS service is able to provide the required information to determine the level of certification attained by the software stack.

27. Project Risks • Hardware availability (the project relies on the partner institutes to fund and provide the hardware). • This risk can be mitigated by ensuring the commitment of the partners in providing hardware resource by specific contractual Consortium Agreements. Resources coming from third-party organizations or projects are also estimated to become an important source of shared hardware • Risks incurred by SA2, JRA1 and JRA2 could affect their ability to deliver their results to SA1, thus limiting the availability of these features into production to the ETICS users. • This is a propagated risk due to the dependencies of this Work Package on the results of other activities. This risk is mitigated by the establishment of a detailed workplan at the start of the project and the constitution of the Technical Committee to supervise, coordinate and monitor the progress of all activities