1 / 18

F.Pasian, R.Smareglia – INAF/O.A.Trieste L.Benacchio – INAF/O.A.Padova

The prototype TNG Long-Term Archive –– Interoperability issues and interactions with the Italian GRID project. Towards an Italian Virtual Observatory?. Joining the International Virtual Observatory Alliance.

yale
Download Presentation

F.Pasian, R.Smareglia – INAF/O.A.Trieste L.Benacchio – INAF/O.A.Padova

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The prototype TNG Long-Term Archive –– Interoperability issues and interactions with the Italian GRID project Towards an Italian Virtual Observatory? Joining the International Virtual Observatory Alliance Struggling to convince your funding agencies that it is worth to devote manpower and funds to coordinate efforts within an international VO activity, while they are still not necessarily convinced it is useful to develop archives F.Pasian, R.Smareglia – INAF/O.A.Trieste L.Benacchio – INAF/O.A.Padova “TOWARD AN INTERNATIONAL VIRTUAL OBSERVATORY” Garching, 10-14 June 2002

  2. “Telescopio Nazionale Galileo” (TNG) • 3.5 m Nasmyth telescope - NTT-based - La Palma • First light in 1998 - AaT since the beginning • Long integration / testing / commissioning phase • First generation instruments completed in 2000: • optical imager (OIG) • low-resolution imager/spectrograph (DOLORES) • high resolution spectrograph (SARG) • near-IR camera/spectrograph (NICS) • ADOPT@TNG, DIMM, Meteo • Assisted observations “TOWARD AN INTERNATIONAL VIRTUAL OBSERVATORY”

  3. The TNG LTA pilot project • A pilot project is currently being finalized, aiming at demonstrating the feasibility of technical solutions for the construction of the Long-Term Archive (LTA) for theTNG  building a prototype LTA • Interfaces to TNG “archiving at the telescope” (AaT) • End of pilot phase foreseen to be end of June 2002 • Financed by CNAA, results to be submitted to INAF • INAF board on computing, archives and networks to produce a report in early July “TOWARD AN INTERNATIONAL VIRTUAL OBSERVATORY”

  4. Scope of the pilot project • Provide suggestions / plans / prototypes to: • allow the long-term storage of scientific and technical data from the TNG • guarantee accessibility by the TNG Observatory staff and by the scientific community to original and derived data • test practically the feasibility of deriving value-added information from the stored data • provide tools supporting life cycleof observing proposals • provide interoperability with a number of existing international data centers and archives • guarantee harmonization with other Italian projects related to archiving of data of astrophysical interest “TOWARD AN INTERNATIONAL VIRTUAL OBSERVATORY”

  5. Product Tree “TOWARD AN INTERNATIONAL VIRTUAL OBSERVATORY”

  6. Interoperability • Direct participation in “Interoperability WG” of OPTICON • Prototype LTA uses standards (e.g. VOTable) defined within “Interoperability WG”  aims at being compatible with AVO development • Catalogue of observations available at CDS for basic-level interoperability with other archives • More in future LTA “TOWARD AN INTERNATIONAL VIRTUAL OBSERVATORY”

  7. Lessons (being) learned • Data quality is an issue: • inconsistencies in data (esp. scientific exposures) • some information missing (documentation, observation schedule, etc.) • improved observing procedures are possible • AI analysis of technical / engineering data identifies some possible problems in T&I systems  data quality (see G.Longo’s talk on Friday) • An archive must be developed in synch with the observatory: • late development of LTA (even if prototype)  late discovery of inconsistencies “TOWARD AN INTERNATIONAL VIRTUAL OBSERVATORY”

  8. Coordination with Italian research Grid • The Italian astronomical community will not build its own AstroGrid • It will rather coordinate with nation-wide initiatives to foster a Grid infrastructure: • eScience Grid • IG-BIGEST “TOWARD AN INTERNATIONAL VIRTUAL OBSERVATORY”

  9. The eScience Grid in Italy (1/2) • Proposal submitted in October 2001 to Ministry of Education University & Research (MIUR) • Bid for ~15 M€ of available ~30 M€ funds; answer soon Title: “Enabling Platforms for High-Performance Computational Grids Oriented to Scalable Virtual Organizations”Strategic Programme on Enabling IC Technologies for Knowledge Society “TOWARD AN INTERNATIONAL VIRTUAL OBSERVATORY”

  10. The eScience Grid in Italy (2/2) • Objectives: • R&D Grid technological development project (CNR&INFN) • Research on optical networks (CNIT) • Deployment of an Italian eScience Grid infrastructure (INFN) for: • Astrophysics • Biology • Computational Chemistry • Geophysics • Earth Observation • International integration by INFN (following requirements) “TOWARD AN INTERNATIONAL VIRTUAL OBSERVATORY”

  11. Astrophysics in the ItalianeScience Grid • Four demonstrators: • distributed computing • accessing and processing VST data • accessing archives and databases (inter alia the prototype TNG LTA) • accessing observational facilities as “data nodes” • About 1 M€ (1/15 of the total MIUR request) “TOWARD AN INTERNATIONAL VIRTUAL OBSERVATORY”

  12. The Italian Grid for Business, Industry, Government, EScience&TechnologyIG-BIGEST • New initiative leveraging from MIUR proposal and aiming at the common development of the IG-BIGEST • Italian Participation to the Expression of Interest for anIntegrated Project "ENABLING GRIDS AND E-SCIENCE IN EUROPE" (EGEE) in the context of 6th EU Framework • All major Italian Scientific National Research Institutions, Universities and Computing Centres reached general consensus: • to take part to the large European Initiative EGEE having CERN as leading partner and INFN as the organization representing the Italian Grid eScience comunity • to have only one large Integrated Project aiming at the development and deployment of the Grid Infrastructure in Europe as Geant does for the Networking “TOWARD AN INTERNATIONAL VIRTUAL OBSERVATORY”

  13. The participants and the Italian Grid eScience infrastructure • High bandwidth network provided by Garr-G Geant • High performance computing infrastructures provided by National Research Institutions, Universities and Super- Computing Centers. Participants : • INFN, CNR, INAF, INFM, INGV, ENEA…… • Computing Centers: CINECA, CILEA, CASPUR…. • Universities • Università di Lecce, Università di Napoli, Università di Pisa, Università di Roma Tor Vergata, Centro Fermi Roma1, Università di Cosenza, Università di Genova, Università di Padova, Università di Politecnico Torino, Università di Salerno, Università di Messina • Formal participation has been finalized “TOWARD AN INTERNATIONAL VIRTUAL OBSERVATORY”

  14. Conclusions (and roadmap?) • TNG LTA Pilot Project has coordination duties (of national archives and with international activities) • TNG prototype LTA is VO-compliant; so will be future archives (TNG LTA, VST, LBC, etc.) • INAF board on computing, archives and networks will provide recommendations • The goal is to have the Italian community joining AVO Phase-B “TOWARD AN INTERNATIONAL VIRTUAL OBSERVATORY”

  15. database Datastore Message Exchange Network WSS-std instr. non-std instr. IWS VERIFY TDB TAG Staging Disk ExternalDisk Data Network DTIN DTOUT A a T U I F UserDisk on-line archive userapplication DTFLUSH AS observer’s CDs off-line archives CDs WR DTWRITE AaT process commands/messages on-line storage Storage telemetry scientific data Externalprocesses data descriptors off-linestorage “TOWARD AN INTERNATIONAL VIRTUAL OBSERVATORY”

  16. SA_INS SA_DET_ID SA_DET SA_EXP DET_ID DET_ID CRDELT1 INSTRUME INS_MODE BITPIX CRDELT2 OBSERVER OBS_MODE PIXSIZE DETOFF1 DATE_OBS OBS_TYPE NAXIS DETOFF2 EXPSTART EXP_TYPE DETSIZ1 EXP_ID EXPTIME FLT_ID DETSIIZ2 RA … DEC … AIRMASS EXP_ID EXP_ID GRM_ID LMP_ID SLT_ID SA_FILE SA_CD MDBT … EXP_ID CD_DATE wss_sess_id … $_filename NCOPIES DATE_CR … DATE_CR CD_ID TIME_CR … NAXIS1 DATE_MD … NAXIS2 TIME_MD … $_path_fts $_path CD_ID CD_ID FITS database shared with Technical database “TOWARD AN INTERNATIONAL VIRTUAL OBSERVATORY”

  17. MDBT TAG AAA database TIME SESS_ID HASHTAB DATE_CR VALUE TIME_CR ACRON STR CD_TAB DATE_END TAG TAG TIME_END CD_DATE $path NCOPIES … CD_ID CD_ID TAG XYZ database TIME HASHTAB VALUE shared with FITS database STR ACRON Technical database TAG TAG … TAG CUB database TIME HASHTAB VALUE ACRON STR TAG TAG “TOWARD AN INTERNATIONAL VIRTUAL OBSERVATORY”

  18. Cyclic operations scheme from: di Serego et al., 26 Jan 1995 “TOWARD AN INTERNATIONAL VIRTUAL OBSERVATORY”

More Related