Towards a robust software environment for e-Infrastructure

1. Towards a robust software environment for e-Infrastructure Paul Messina Argonne National Laboratory Caltech (Ret.) USC-ISI December 9, 2003 Paul Messina

2. Infrastructure has long been important Ancient Rome was supplied by more than ten aqueducts, providing the city with some 143,845 m3 of water each day. Paul Messina

3. Infrastructure can last a long time Parts of several Roman aqueducts are still in use; they supply water to fountains in Rome. Paul Messina

4. Key Issue: Software is Infrastructure; we need to develop and maintain it accordingly • If we want e-Infrastructure to be an enabling technology for research and commerce, we have treat software as infrastructure just as we do hardware and networks. • This implies emphasis on robustness, functionality, evolution, and persistence Paul Messina

5. Research prototype middleware needs to become production quality • Research projects are not funded to do the regression testing, configuration, and QA required to create production quality middleware • Common rule of thumb is that it requires at least 10 times more effort to take proof-of-concept research software to production-quality • In addition -- for grid middleware -- interoperability is essential and must be verified Paul Messina

6. A potential mechanism:Open Middleware Infrastructure Institute • Concept: An international organization (an Institute) sponsored by international funding agencies and industry • Vision • Existence of a truly global, production middleware infrastructure • Mission: to produceand maintain standard-conforming and interoperable grid middleware • building on existing efforts Paul Messina

7. OMII Goal • To ensure that middleware becomes production-quality, part of the infrastructure, and acquires sufficient functionality quickly enough to meet the expectations of the emerging grid user communities Paul Messina

8. Why is an additional organization (e.g., OMII) needed? • Funding of grid middleware development typically falls far short of what is needed to produce production-quality software and its support • Prospective users of grids and governments that fund grid efforts expect mature software to emerge very soon • Lack of coordination of existing efforts may result in lack of interoperability across grids Paul Messina

9. OMII Implementation • The Institute would be a distributed/virtual organization, federating various efforts • Might have software development/production centers on each continent, for example • Its constituents would include/involve • developers, • producers, • and integrators of production-quality grid middleware Paul Messina

10. What Would OMII Do? • Produce open source software that • (a) user organizations could install by to provide grid functionality • (b) computer and software companies could adopt and give added value by supporting it, porting to new platforms, optimizing performance on particular platforms, etc., • such as was done with MPICH for MPI message-passing libraries Paul Messina

11. What Would OMII Do (cont’d)? • Maintain and support the software it produces • Follow Global Grid Forum and other relevant standards activities • E.g., W3C • Turn reference implementations into production-quality software Paul Messina

12. What Would OMII Do (cont’d)? • Offer a “GGF-standard compliance certification” function for producers of software who want to verify that their product complies with one or more GGF standards Paul Messina

13. Who would do the work? • The OMII would let subcontracts for specific work, as well as do work in-house • Software development could be done by • university groups, • research laboratory groups, • and industrial concerns. • There are examples of excellent software products from all three types of institutions • Berkeley Unix • MPICH, “PACKs,” MACH Paul Messina

14. Guided by applications • To ensure that the middleware produced by the OMII will meet user needs, the OMII would use as drivers selected international grid application projects, such as • LHC Computing Grid • “GVO” (“Global Virtual Observatory,” my invented name for the collection of virtual observatory projects in the US, EU, UK, Japan, …) • projects in bioinformatics • projects in the environment/earth observation and monitoring Paul Messina

15. UK OMII is about to emerge • The UK e-Science Programme has selected a proposal to create the UK component of OMII • Hopefully the US, the EU, and other world regions will also fund such activities • And hopefully technology roadmaps will be developed jointly and the activities will be coordinated Paul Messina

16. Concluding Remarks • Producing and maintaining infrastructure-quality software is a major, costly effort • Coordinating activities that produce infrastructure-quality middleware will lower costs and reduce risks • As will funding jointly some of the projects • Persistence is essential, hence an Institute, not just projects • Comments are most welcome Paul Messina