The astronomical Virtual Observatory : l essons learnt , looking forward

1. The astronomical Virtual Observatory :lessonslearnt,lookingforward Françoise Genova, ADASS, Paris, 8 November 2011

2. Examplestakenfrom the Europeanview, but otherprojects have followedsimilarpaths

3. The VO aim Enableseamlessaccess to the wealth of astronomicalresources An ambitious goal and no pre-existingorganisational model to follow • Wehad to invent a way of building the VO • Pragmaticapproachwith a few basic principles • A global VO • Keepin mind science usage and implementation by data centres • Fullfilastronomy’sneeds but when possible use generic building blocks to allowwiderinteroperability

4. A global VO • The VO has been thoughtfrom the verybeginning as a fully global endeavour • Neither a French (or Alsacian – Strasbourg region) nor a US nor a Japanese VO, but the astronomical Virtual Observatory • The basis of the VO isInteroperability • Global interoperabilityrequires international agreement

5. Interoperability: first steps • January 2002 Strasbourg OPTICON European WG but international participation First Interoperability meeting – > VOTable CDS/NVO >Pre-IVOA standard • June2002 Garching • Towardan International Virtual Observatory (ESO/ESA/NASA/NSF) • Creation of IVOA

6. Interoperability : IVOA standards Procedureadapted from W3C

7. Continuingto work on standards remainsmandatory Feedback fromimplementation and scientific usage Evolution of astronomy – new facilities, new science Evolution of the technologicalcontext Interoperability: currentstatus Passage to maintenance mode for many standards From C. Arviset

8. VO evolution • The VO has never been solely a technologydevelopment • Scientists and data providers participatedfrom the beginning in the VO development • Thingshad to be made in the properorder • The basic building blocks (standards and tools) had to be – and have been – built, with in mindtake-up by data centres and science users • Nowtowardsoperational phase • The focus ismovingtowards more support to take-up by scientists and data providers, plus outreachtowardseducation

9. VO Science requirements • Science requirements have been presentfrom the beginning • Scientists in VO projects • Science AdvisoryCommittees or equivalent • Science demos e.g. AVO RTD project (2001-2004) WA1 Science/WA2 Interoperability/WA3 Technology

10. 2MASS ESO-WFI Chandra Data available at selected point are highlighted in tree VLT-ISAAC HST-ACS Field of view outlines are plotted automatically DSS Image metadata My Data F. Genova, Interoperability meeting, 9 November 2009

11. Science feedback and priorities • IVOA has set up a Committee, then a Standing Committee for Science Priorities to identify in highpriority science cases, then gap analysis to identify the lacking standards • First example: help implementation by data providers > the ObsDMmetadatasubset • SED building, Search by objectclass/list • Work more closelywith the VO projects’ Science AdvisoryCommittees to gather ‘global’ requirements and feedback F. Genova, Interoperability meeting, 9 November 2009

12. TAP Library with documentation and tutorials

13. Take-up by data providers • A major action of IVOA during the last years has been to define a ‘simple’ subset of metadata to beprovided by data providers to facilitateimplementation, good enough for data discovery and accesstools • Hereat ADASS wesee the archives of the major facilities but not only • Hugediversity of possible data providers – the VO aimsatgivingaccess to the wealth of astronomicalknowledge

14. The Euro-VO census of data providers • Census of European Data Centres (EuroVO-DCA, EuroVO-AIDA, 2009, 2010) • Inclusive definition : Data Centres populate the VO with data and services, service to the community, added-value, sustainability, quality • 69 ‘data centres’ answered • Data archives, services, theory data and services • Someof these services are arewidelyused by scientists to access to bibliography, data and tools • The provision of data and services has clearly been stronglyencouraged by the develoment of the VO

15. Data centres in Europe (and elsewhere!) • A hugediversity in aims • large services provided by international agencies, with archives of the large ground-based and space instruments • large systematicsurveys of the sky, results of large simulations • generalist data bases and services • smaller contributions of scientific teams whichsharetheirexpertise • Hugediversity in size and organisations • An ecosystem of data and service providers willing to share data and knowledge - a distributed, heterogeneous system with no a central point norhierarchical organisation

16. Strands of workduringoperational phase • Support to take-up by data providers • Support to take-up by the scientificcommunity • Continuoustechnicaldevelopment • Standards (update of existing standards and new standards because of feedback/evolutions) – VO teams + IVOA • Tools • Outreachtowardseducation and the general public (appeared in IVOA meeting in May 2011)

17. Support to take-up • Scientists • Topical ‘Community feedback’ workshop • Calls for proposals for advanced usage • Schools • Tutorial • Data providers • Implementationtools • Tutorials • Data Centre Schools • Data Centre Forum to discussrequirements and feedback?

18. IVOA evolution • BetterconnectionwithSACs to get science requirements • Implementation feedback • Development of the information sharing role: on take-upactivities, implementationtools, outreachactivities, etc, although all theseactivities are not under IVOA responsibility

19. VO status • The strands of worknecessaryduringoperational phase are wellunderstood • The basic building blocks are here • Major challenge: sustainability • Interdisciplinary usage canappear as a must in many « political » contexts

20. Interdisciplinary aspects • IVOA had in mind to use generic components when possible. e.g. for twocritical components for « wide » interoperability • Registry of Resources: OAI-PMH, Dublin Core • Vocabulary: RDF + SKOS (semantic web) • Re-use/adaptation by other disciplines: pragmaticapproachthroughdissemination of knowledgethrough staff (HELIO et al., VAMDC)

21. European VO specific challenge • A richlandscapeincluding the twoEuropeanAgencies, ESA and ESO, and national programmes • Several of the founding parents of the astronomical VO • Challenge: coordinate/federate VO projects • Differentresearch/fundingsystems • Differentprojects • Euro-VO: the European ‘glue’ • coordinateactivities (e.g. regularTechnology Forums) • reachall EU countries includingthosewith no organised VO project • critical mass for Science AdvisoryCommittee, support to take-up and outreach (templatesre-used in the national context)

22. Euro-VO Results • A verysignificantincrease in collaboration • Technical collaboration, e.g. on the definition of standards and tools but also on R&D • Differentkinds of meetings which have shaped the collaborations and relations with data centres and users • Attention given to non-partnerEuropean countries to support theircommunities and to help themshapetheirownpoliticies F. Genova, Interoperability meeting, 9 November 2009

23. The Europeancontext • Strategy for astronomydiscussed and set-up by AstroNet ERA-NET, whichincludes ~all fundingagenciesfrom ~all EU countries • Science Vision (2008) and Infrastructure Roadmap (2009) • The VO isrecognized as an important infrastructure of astronomy • But the recommendations are not easy to implement

25. F. Genova, Interoperability meeting, 9 November 2009

26. F. Genova, Interoperability meeting, 9 November 2009

27. Europeanfunding system • Europeanfunding: a complex system whichevolvescontinuously • Organizedinto successive Framework Programmes • Calls and « instruments » • Euro-VO: a series of projectswhichprogressivelybuilt the landscape • Structured in phases in three successive Framework Programmes • Phase A (FP5): AVO, OPTICON Interoperability WG • Development (FP6): VO-TECH, EuroVO-DCA • Transition to operations (FP7): EuroVO-AIDA, EuroVO-ICE (on-going, ‘bridging’)

28. The future for Euro-VO • How to implementAstronetrecommendation? • Definearticulation/balance between national/Agency level and Europeanlevel • Sustainability of national/Agency projects • Sustainability of the European layer • Stronglydependent on Europeanfundingopportunities • ContinuingEuropean/international coordination ismandatory • Projects on-goingin ‘neighboring’ disciplines (HELIO, Europlanet, VAMDC)

29. High Level Expert Group on Scientific Data AstronetRoadmap

30. The VO in the generalcontext of scientific data policies • The generalcontext in whichweworkisrapidlyevolving • High Level Expert Group: Collaborative Data Infrastructure • Requirementthat data obtained on public fundsare made publiclyavailable • Astronomyat the forefront: a global, heterogeneous, interoperable, OPEN, widelyused, data infrastructure • WE HAVE USERS: on-line services are everydaytools for the astronomicalcommunity • Interdisciplinary usage isseen as the basis, but disciplinarypillars are necessary in a Collaborative Data Infrastructure • Astronomycanbeseen as an interesting use case! Let’sconvinceourfundingAgencies…