1 / 25

Astronomical Archives: It’s rules for small and big telescope. IA2 vision and services

This article discusses the importance of archiving astronomical data and highlights the main goals and services of the Centro Italiano Archivi Astronomici (IA2). It covers data handling, data services, data curation and preservation, and future perspectives. The article also mentions the TNG Archive and IA2's standards for data handling and management.

loydv
Download Presentation

Astronomical Archives: It’s rules for small and big telescope. IA2 vision and services

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. R. SmaregliaC. Knapic, M. Molinaro and the IA2 Team Astronomical Archives: It’s rules for small and big telescope. IA2 vision and services 12-05-2016 – Use of the small Telescope in the GIANT ERA – Chianti Topics

  2. Why archiving is important? MAST A. Aloisi - ADASS XXV 60% of MAST papers based, in whole or in part, on archival (HST) data!

  3. Biblioteca Apostolica Vaticana • Public Open: • 1.500.000 printed book • 8.300 incunaboli ( 65 in pergamena) • 80.000 manuscripts • and more … Last 20-30 years: Every day 400 visitors Only 15.000 manuscripts were viewed by expert

  4. Open Science - OpenAccess

  5. IA2 MainGoals IA2 is an INAF data center e-infrastructure for astronomical data Centro Italiano Archivi Astronomici (IA2) main goals are : • archiving systems • safety • data curation and preservation • distribution over several geographical distributed sites • providing services and tools (TWiki, work-flow, etc..) • data publication, also in the VO of Astronomical Data • IA2 manages data of several PROJECTS. • Mainly they come from: • - TELESCOPES (raw; INAF ground based) • - SURVEYS (raw and/or calibrated) • - SIMULATIONS (ITVO) • - EU Projects (GENIUS, VIALACTEA, INDIGO) • IA2 is part of INAF-ICT http://www.ia2.inaf.it

  6. IA2 – Data Center • IA2 • MainGoals & Servicies • Data handling: NADIR • Properties and configurations • Data Services: • GUI interface • VO services • VAPE project • ODA-Oracle • Data Sharing • Future perspectives • Single Sign-On • Data Curation & Preservation • Data Valorization (DOI) • Data Cloud http://www.ia2.inaf.it

  7. IA2 Services

  8. Updated at 22/02/2016

  9. Updated at 22/02/2016

  10. TNG Archive 2015 Map of Accesses ITA: 2879 ESP: 1367 GBR: 517 USA: 488 FRA: 288 NLD: 161 CHE: 148 ARG: 94 DEU: 92 CHL: 76 DNK: 71 PRT: 70 POL: 68 UKR: 61 BRA: 28 RUS: 26 SWE: 24 CHN: 24 CAN: 17 JPN: 14 LTU: 1 IND: 1 ARM: 1 URY: 8 MEX: 6 BEL: 6 ISR: 5 HUN: 5 FIN: 3 ROU: 2 AUS: 2 TWN: 1 MUS: 1

  11. IA2 Standards: Data Handling/Management IA2 manages Astronomical Data mainly in FITS format (Flexible Image Transport System) for images and spectra and GADGET2 for simulations. Current projects implies also the management of Hierarchical FITS and MBFITS formats. IA2 archives follows the directives of Open Archival Information System (OAIS)

  12. Archive data handlingrequirements • Modular software, optimized to be asmuchaspossibleflexible; • Scalability in both serial and parallel data distributionparadigms; • Handling of calibrated data; • Policy and versionsrevisedeasily, in a flexiblemanner; • Consistency on geographicallydistributedarchives and securedifferentiation in archivescontent; • Strong logging and errorhandling; • Possibility to correct meta-data content, maintainingconsistencesacrossdistributedarchives;

  13. Data Management: NADIR • New Archiving Distributed InfrastructuRe(NADIR): • Archival software toolaimed to guarantee high scalability, modularity, robustness, errortracking, tools to monitor the status of the archivingsystem. • The archivingsystemiscomposed of severalblocks (devices) in a TANGO Controls System • TANGO Controls: • an open source device-orientedcontrolstoolkit for controlling hardware or software devices. • operatingsystemindependent and supports C++, Java and Python for all of the components. • A deviceislogicallyseenas an entity to be controlled with public attributes and private properties • Attributes and commands are defined to control and monitor the device, its status and configuration • TANGO devices for NADIR are archivingsoftwares

  14. NADIR • NADIR mandatoryrequirements: • INSTRUMENT; • OBS-DATE; • NADIR functionalrequirements: • PARTNER; • PINAME; • PROGRAM; • NADIR notfunctionalrequirements: • Coherentfilling of fits keyword values in terms of types and valuesconsistencies to allowqueryefficiency • Data propagationdepends on ingestion date and policy. • Manage any data format ( FITS, HDF5, MBFITS, … )

  15. NADIR – LBT Configuration • Source point of information: • LBT telescope site – temporaryarchive/repository • Ingestion site for standard FITS files of raw, technical or guiding data • Limited storagecapacity • Full metadata DB since the first operations. • Master archive in Tucson: • Full archive (metadata and data) for allpartners and all data • In the sameplace, first data reduction for some data, data qualityevaluationforeseen in the near future (interaction with operation team) • Data distribution to partnersboth to other LBT distributed Archive sites and to externals (data delivery, notarchive) • Distributed sites(MPIA – Heidelberg; OATs – Trieste) • Partialarchives with proprietary data • Data reduction with proprietarypipelines • Data delivery to otherarchives(Ohio State University – Columbus; INAF – IASF – Lambrate (Mi) ; INAF – OARm - Roma)

  16. Archive Inteface • Data publisher • Mapping back end database to front end database • Setting data policy and proprietaryperiod • Front End database • Generictable with allobservations (butonly last version for each file) • Oneinstrumenttable for eachinstrument with specificsearchfileds • Dedicated Web Interface • Under development a portal generator with a new publicationinterfacebased on VO compliant front end database

  17. IVOA Standards • Virtual Observatory (VO) protocols and standards are getting mature and the astronomical community asks for astrophysical data to be easily reachable VO-Dance Framework: • VO – Dance: an application that from existing DB and archive structures can translate their content to VO compliant resources • VO-Dance(JavaEE web app) • SCS, SIA, SSA • IA2TAP • TAP • TAPSchemaManager

  18. Easy access to your data - SSO Additional tool for TNG Observatory will be a Grouping software. It will allow a contributed management and distribution of programs credentials and privileges. It is also the first step to the SSO (Single Sign On) technology adoption (under development). Grouper is an enterprise access management system designed for the highly distributed management environment and heterogeneous information technology environment common to Universities. It could work on top of Identity managers like IDEM (for the Italian counterpart) and is compatible with SHIBBOLET.

  19. Data Valorization - DOI DOI (Digital Object Identifier): DOI is a character string used to uniquely identify an object such as an electronic document. The DOI for a document is permanent, whereas its location and other meta-data may change. Referring to an on-line document by its DOI provides more stable linking than simply referring to it by its URL, because if its URL changes, the publisher need only update the meta-data for the DOI to link to the new URL.

  20. Example 1: GAPS@IA2 • GAPS requirements • Strong interaction with HARPS-N@TNG private data; • Customizablere-process of GAPS data with appropriate spectral line mask and options; • Performqueries on additional meta-data content; • A flexible and collaborative tool to manageadditional infoabout the project and the observations; • A repositorywhereto access, synchronize, share and search for interesting data;

  21. Example 1: GAPS@IA2 UserAccess Time Series Upload Summed rawreduction UserAccess F-stars reduction results GAPS Database GAPS Master Catalog Time SeriesPubblication DRS reduction ObservationReport UserAccess User Access

  22. Example 2: VAPE Virtual observatory Aided Publishing for Education • Web application for easily preparing VO services and publishing educational data in the VO • Targeted at people involved in education • Built with Java, Wicket, Bootstrap, JPA (EclipseLink), nom.tam.fits, Log4j • Service publishing is done through the VO-Dance IA2 application • Everything is transparent to the user • Data ingestion is handled automatically by NADIR • Users have to sign up and log in, provide service information and metadata, wait for service approval and registration, load their data • FITS files must include TELESCOP, INSTRUME, OBJECT, PURPOSE (must be ’edu’ as we only handle educational data), OBJ_TYPE, OBSERVER, INSTITUT keys • TELESCOP and INSTRUME must match the names assigned to the • respectiveentities in VAPE

  23. http://ia2-edu.oats.inaf.it:8080/vape

  24. Example 2: VAPE • Succesfully used internally for publishing data of the educational telescope “Le StelleVanno a Scuola” (SVAS, @OATS) • The SVAS service has been registered (EURO-VO registry) and is accessible through the standard VO tools

  25. Conclusions:

More Related