EGI Community Support

1. EGI Community Support CERN EGI Workshop Patricia Méndez Lorenzo(CERN, IT-GS/EIS) Monday, 30-06-08

2. Outlook • Introduction • Grid Support Infrastructure • Support Infrastructure in EGI/NGI • “Mind the Gap”: Transition Phase • The Specialized Support Groups: SSGs • The EGI/NGI Applications Support tasks • Applications Support in Numbers • Summary and Conclusions The following slides contain the contribution from EGEE members (Vangelis Floros, Frank Harris, Bob Jones, Cal Loomis), WLCG (Jamie Shiers) and EGI (Diana Cresti) Patricia Méndez Lorenzo CERN IT-GS/EIS

3. Introduction New Users How to attract them Existing Users How to maintain them GRID Environment Support • Applications Support in EGI/NGI • Day-by-day support is within the NGIs and many VOs operate over several NGIs • Different middleware supported Middleware developments Good operations procedures Good level of service Patricia Méndez Lorenzo CERN IT-GS/EIS

4. Grid Support Infrastructure • Any application entering the Grid will have to deal with (at least) two middleware stacks: • Supported middleware • Common to all applications - computing model agnostic • Application middleware • Requires a deep knowledge of the application models and their Grid use • The support team will have to become expert in both middleware stacks • Each support member will need to know the computational model of the supported application • This requires a big effort by the Grid support team and labor intensive interaction with the users Patricia Méndez Lorenzo CERN IT-GS/EIS

5. Grid Support Infrastructure • Fundamental tasks of any support team • Liaison between the community and services-operations and middleware teams • The application might not know many Grid related issues • Provides Grid expertise for the middleware implementation into the application computing model • Software development towards the final implementation • Creation of missing components which might be shared by other communities • Evaluation of the middleware functionality and scalability tests based on the community needs • Management of VO-specific services • User-support if needed (a la GGUS) • Expertise to solve Grid related issues for running applications • To the site and middleware level • Deep knowledge of both Grid infrastructure and VO computing model • Documentation and training (application oriented) • Liaison with the sites • Operations activities if needed • Creation and registration of the new VO, eventually VO manager, negotiation of resources and services, etc Patricia Méndez Lorenzo CERN IT-GS/EIS

6. Grid Support Infrastructure • The current project-oriented based infrastructures have succeeded (IMHO) due to the following reasons: • Full flexibility - zero bureaucracy • Previous slide: support can make operation tasks, VO management, can “create” a computing model if needed ... • The goal (moreover with new VOs) is to demonstrate that the level of infrastructure we can provide ensure the merge of any new VO in few days • Total trust of the communities on their support • High level of expertise achieved by working closely (& physically) together with the applications and Grid teams • This level is achieved after a LONG time and many productions Patricia Méndez Lorenzo CERN IT-GS/EIS

7. Applications Support in EGI/NGI • In EGI, the NGI is the most visible organization to the user • Mono-disciplinary and regional communities perfectly covered • But remember the fundamental goal:“foster international and interdisciplinary collaboration” • Interdisciplinary applications • Few countries have strong enough support structures to be capable of supporting many scientific disciplines • International applications • In many cases they have a central institution • The support locally placed close to them. This is what they know, this is what works • The support = the expertise, the expertise must be placed where the application developers are Patricia Méndez Lorenzo CERN IT-GS/EIS

8. Applications Support in EGI/NGI • The EGI project will support more than one middleware stack • In principle this should not be a problem • Production experience with some applications • Interoperability is a huge effort task • We do not mean simultaneous but independent use of different stacks • But... Usually different middleware stacks are regionally supported and developed • Few user groups have enough resources to adapt their applications to different middleware • This can create a barrier between users which historically used different middleware Patricia Méndez Lorenzo CERN IT-GS/EIS

9. Introduction • Grid Support Infrastructure • Support Infrastructure in EGI/NGI • “Mind the Gap”: Transition Phase • The Specialized Support Groups: SSGs • The EGI/NGI Applications Support tasks • Applications Support in Numbers • Summary and Conclusions Patricia Méndez Lorenzo CERN IT-GS/EIS

10. Avoiding the Gap • Based on the previous slides a smooth transition from the project-oriented infrastructure to EGI should be defined • Transition phase for those existing applications • We talk about applications which are already running smooth productions in a project-based scenario • They have a support and expertise infrastructure which they know and they thrust • The project is moving to a non co-located infrastructure and the stability cannot be ensured without an agreed transition proposal • Clear definition of the EGI scope during the whole project from the application support point of view Patricia Méndez Lorenzo CERN IT-GS/EIS

11. Transition: EGEE-> EGI • Necessary to: • Accommodate the already existing communities • Train NGIs and smoothly gain experience • Proposal: Specialized Support Groups (SSGs) • Configured among NGIs • Central structure with coordination roles during the whole project life • Play a key role during the transition phase • Formed by the Scientific Support Clusters already existing for many communities • Promote the creation of such structures for the nascent communities • The transition period will require the expertise of the current EGEE and additional projects experts which are already in production • EGI will support these current projects and their experts will have to be involved in the transition • The SSCs must be created based on the current communities requirements Patricia Méndez Lorenzo CERN IT-GS/EIS

12. Specialized Support Groups • Transition phase: • Ensure the same support infrastructure that the already existing communities enjoy through the Scientific Support Clusters • EGEE-EGI collaboration is fundamental • International applications are the most “threatened” groups • They have a located support team independent of any NGI • SSGs will have to ensure a fluent communication with the operations and middleware groups to ensure the same level of service they have now • Good (Service+Operations+Support) set ensures a production level • SSGs will have the responsibility to hide or at least smooth any negotiation with those NGIs taking the role of the support • SSGs will ensure the training of the new support members if (when?) they change from the current infrastructure Patricia Méndez Lorenzo CERN IT-GS/EIS

13. Specialized Support Group • After the transition: Stable regime • Moving towards a coordination role • Key role by identifying nascent scientific communities • Maintain a constant communication with the nascent Scientific Support Clusters • Promoting the creation of such clusters and involve them into the EGI infrastructure • Support for travel and meetings to ensure a self-supporting infrastructure • Avoid duplication of effort • Management of the new application requirements in terms of middleware and operations Patricia Méndez Lorenzo CERN IT-GS/EIS

14. Introduction • Grid Support Infrastructure • Support Infrastructure in EGI/NGI • “Mind the Gap”: Transition Phase • The Specialized Support Groups: SSGs • The EGI/NGI Applications Support tasks • Applications Support in Numbers • Summary and Conclusions Patricia Méndez Lorenzo CERN IT-GS/EIS

15. The EGI/NGI Application Support tasks • Event Organization • User Forums • International events (once per year) to bring together Grid users • Exchange of experiences and techniques • Scientific program coordinated by EGI which also need to ensure a high qualified technical presence to confront problems from different communities • Topical Workgroups • User facing similar problems while using the Grid infrastructure • Meeting prepared to devise solutions • Discipline Meetings • Associated to specific disciplines to discuss common solutions related to the use of middleware, techniques, data resources • Large and well-established communities will run them with their own actors • Small and recent communities will require a special effort by EGI Patricia Méndez Lorenzo CERN IT-GS/EIS

16. The EGI/NGI Application Support tasks • Application porting for new communities • Talks disseminating previous experiences using the new EGI infrastructure in the previous mentioned events • Involving new communities in a fast and reliable way • Flexible procedure to allow a fast immersion • EGI will have to ensure the required infrastructure in terms of services, middleware, VO, certificates, support • Infrastructure prepared to a full production level • The key of success of this demos is the (short) time • If too long, they fly away... Patricia Méndez Lorenzo CERN IT-GS/EIS

17. The EGI/NGI Application Support tasks • General coordination and change management • Liaison among the different support communities with management and technical committees • Passing VO requirements • Trigger actions to provide specific services and resources Patricia Méndez Lorenzo CERN IT-GS/EIS

18. The EGI/NGI Application Support tasks • VO interfacing with operations • Several specific areas where support will need to interact with operations • Liaison between operation teams and communities • VO operations in terms of registration, activation, certificates • Depending on the community the support can be engaged of some operations procedures which require a high level of expertise • Evolution and coordination of Help Desk • Central ticket system for the full user community (i.e., GGUS) • Representation of the communities in the operation meetings • Related activities coordinated by USAG group in EGEE meeting on a regular basis Patricia Méndez Lorenzo CERN IT-GS/EIS

19. The EGI/NGI Application Support tasks • VO interfacing with several middleware stacks • Without an efficient coordination the NGI-oriented project has the risk to duplicate effort and work • Based on the current experience this damages the vision of the project and the support to the communities • Standard Service Interfaces • EGI will have to play a strong role by coordinating between users, requirements and middleware providers • The user support team will be engaged of the testing of the middleware based in the requirements of their own supported communities • Separate incompatible interfaces • Risk for small communities with a lack of resources to test multiple middleware interfaces • Several solutions proposed, all of them with weak points (documentation and training, sites deploying several middleware (!)) • Common layer to hide middleware differences • Already experiences in EGEE, (for example Ganga) • Provides access to different stacks • Interoperation is still a fundamental task to work in Patricia Méndez Lorenzo CERN IT-GS/EIS

20. The EGI/NGI Application Support tasks • Central repositories • To ensure an efficient communication • Continuation of the RESPECT program in EGEE • Created in principle for generic packages and services • EGEE-III plans to expand it to “true” middleware services and to common application-level software • Centralized VO registry • Provides common configuration parameters to all Vos • Check the already existing Vos • Central registry of applications • Allow individual users to provide information about their applications, scientific results and contact persons to encourage collaboration around similar applications Patricia Méndez Lorenzo CERN IT-GS/EIS

21. The EGI/NGI Application Support tasks • Training and Documentation (Applications specific) • NGI dependence for end-users, system and site admin, etc • Coordination level is mandatory • Share resources and expertise • Documentation and training material • Identify possible scenarios and group persons requiring training • Through the already mentioned events as User Forums • Avoid duplication of effort with other groups (middleware experts) which will also provide training Patricia Méndez Lorenzo CERN IT-GS/EIS

22. Applications Support in Numbers All the mentioned FTE numbers are very preliminary and require further discussions Patricia Méndez Lorenzo CERN IT-GS/EIS

23. Summary and Conclusions • The Application Support infrastructure is the most exposed team to the end-users - they are literally “the front line” • Based on the current project-oriented environment, EGI will need to ensure a smooth transition to a NGI support infrastructure • Current users cannot suffer any gap • We have currently several communities in full production right now and these must be around the table • Experience gained in the past is key and involvement of the current experts is mandatory • Ensure a flexible Application Support infrastructure • Establish a full collaboration with operations, services and middleware • Create a robust infrastructure to admit any new community in few days • Still many points to clarify moreover in terms of Specialized Support Groups and Scientific Support Clusters tasks • This is the direction to go in • The goal is to ensure that ALL current communities will continue to be supported and that the infrastructure is ready to rapidly admit new communities Patricia Méndez Lorenzo CERN IT-GS/EIS

24. Final Remarks and Recommendations • Check-list for the Application Support infrastructure (a la EGEE) • VO support • Application porting support • User support • Scientific Support Clusters • Proposal • Establishment of a collaboration with 3 initial clusters (HEP, life sciences... To be established) • Creation of a support infrastructure based on the experiences and needs of these groups • This is the real life and based in these experiences the EGI/NGI support infrastructure can be created (also regarding the number of FTEs to define) • Expand the knowledge to the rest of communities Patricia Méndez Lorenzo CERN IT-GS/EIS