1 / 8

Innovative Strategies for Post-EMI Software Lifecycle Management in WLCG

This document outlines initial concepts and strategies for an effective post-EMI software lifecycle management approach within the WLCG framework. It emphasizes utilizing widely accepted integration processes, including the use of the EPEL repository, to streamline software deployment. The importance of independent operation by product teams, along with structured testing, staging rollouts, and detailed requirement tracking through platforms like GGUS, is highlighted. It also discusses challenges in middleware configuration and deployment alongside the need for coordinated efforts in managing updates and decommissioning components.

signa
Download Presentation

Innovative Strategies for Post-EMI Software Lifecycle Management in WLCG

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. WLCG Software Lifecycle First ideas for a post EMI approach

  2. Basic Concepts • Use open, widely used process for integration • EPEL repository and process as integration point • https://fedoraproject.org/wiki/Bodhi_Guide • test repository for staged rollout • using test component against production repository • WLCG repo for non EPEL compliant packages • Alternative paths for middleware clients (tarball, CERNVMfs) • EPEL derived • Product Teams operate independently • Software repositories during development • Build tools ( most use Fedora tools ( mock etc.)) • Test tools for unit and system tests • Fine grained bug and requirement tracking • Production readiness on scale can be only verified in production • Pilots and Managed Staged Rollout • As little central coordination as possible

  3. What needs to be coordinated? • High level bug tracking (GGUS) • High level requirement tracking ( GGUS ) • requirement prioritization ( PreGDB twice a year, WLCG-MB ) - • Roadmaps • for requirement implementation • decommissioning of components, interfaces and APIs • move to new OS versions • Middleware Configuration ( TEG OPS WG5 ) • Middleware Deployment ( TEG OPS WG5 ) • this covers Pilots and Staged Rollout • Release announcement and documentation • who, what, when • Could be done via the WLCG-T1-Service Coordination Meeting • documented in an extended WLCGBaselineServicesTwiki page

  4. Roadmaps for: • Requirement implementation • Part of Pre/GDB and WLCG-MB • Needs to be prepared well in advance • Decommissioning of components, interfaces, APIs • same as now: Pre/GDB discussions, MB decisions • Move to new OS versions • same as now: Pre/GDB discussions, MB decisions • Currently this is done independently for ARC, gLiteand the OSG middleware stack • maybe this could be coordinated more closely

  5. Middleware Configuration • Complex problem • small sites profit from a simple tool • YAIM or RPM post install scripts • large sites use configuration management tools • Quattor, Puppet, cfengine .....  no generic support possible • need highly specific configuration • Less configuration would be great.... • but is not likely to happen overnight.... • Probably a mixture between a simple tool and improved generic configuration guide is needed • Working Group???

  6. Middleware Deployment • Based on EPEL + WLCG repository • YUM as a tool • Clients also provided in a form for distribution with the application software (Application Area, tarball,..) • this is only the case for gLite clients, ARC and OSG managed independently • Pilot services are negotiated between Product Teams, Sites and Experiments ( no central coordination(needed) ) • Staged Rollout needs to be managed  • coverage of experiment/service/major site config. • EMI WN is a good example • Site and experiment participation is crucial  • can’t rely on ad hoc agreements, needs formal commitment • needs follow-up and coordination • Could be part of the mandate of an WLCG Operations Team • Small WG needed to agree on the details

  7. Some Detailed Questions • Rollback with a roll forward repository? • For RPM based distributions via RPM Epoch • V1.2 is in production (epoch 0) • V1.3 released and generates an issue • V1.2 re-released with epoch set to 1 • For RPM V1.2 epoch 1 is then “newer” than 1.3 • Non-backward compatible changes? • Introduced in parallel along production versions • Like: SRM, GLUE • Support for legacy service/API is retired as any middleware component

  8. General Questions • Is there a need for a common approach for staged rollout for all middleware?

More Related