1 / 6

Archive Access Tool

Archive Access Tool. Review of SSS Readiness for EVLA Shared Risk Observing, June 5, 2009. John Benson. Scientist. AAT – Archive Components. Data ingest and storage Archive storage uses ESO NGAS system (with ALMA) Ingest data types Raw telescope data in real time Science data products

sanaa
Download Presentation

Archive Access Tool

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Archive Access Tool Review of SSS Readiness for EVLA Shared Risk Observing, June 5, 2009 • John Benson • Scientist

  2. AAT – Archive Components • Data ingest and storage • Archive storage uses ESO NGAS system (with ALMA) • Ingest data types • Raw telescope data in real time • Science data products • Calibration data • Logs, scripts, proposals • Meta data retrieval and storage • Meta data miner programs based on data file formats • Meta data storage – Oracle 11g databases

  3. AAT – Archive User Access • User Interface - queries • Web-based query tools • Python scripts • User interface – query response • Observational meta data – project level, scan level, summaries • User Interface – data delivery • User specifies delivery location • Proprietary data protection • File format conversion

  4. ATT – SRO Readiness • Data Ingest and Storage – June 2009 • Six NGAS hosts : 1 – EVLA site, 5 – AOC. • NGAS cluster hierarchy with master nodes. • Create duplicate NGAS nodes • Ingest SDM and BDF files real-time into evlangas-1 • Mirror into AOC NGAS cluster • (135 SDM/BDF file sets in the archive) • Data Ingest and Storage – March 2010 • Two NGAS clusters, primary storage, copy storage • Improved data transfer between EVLA site – AOC (ESO) • 1 Gbit fiber EVLA – AOC • (expect < 20MB/s)

  5. AAT – SRO Readiness • Archive Access Tool – Meta data – June 2009 • Cataloging SDM (WIDAR) • Minimal db table schema changes (CASA-like) • File set id’s for archive meta data and NGAS • (automated, moderate failure rate) • Archive Access Tool – Meta data – March 2009 • Possible db schema modifications (WIDAR modes) • Support connections to archived log files, scripts, calibration files.

  6. AAT – SRO Readiness • Archive Query Tool – June 2009 • Web tools include telescope = EVLA • Supports queries and creates responses for EVLA SDM/BDF file sets. • Displays scan level query responses (SDM) • Delivers SDM/BDF file sets : • SDM/BDF file set in directory • Convert to CASA MS • Convert to AIPS uvfits file • Archive Query Tool – March 2010 • Provide ability to deliver data for selected observing scans.

More Related