Swift archive

1. Swift archive Lorella Angelini/HEASARC

2. Swift MissionCharacteristics Collaboration USA/Italy/UK • Multi-wavelength observatory • Burst Alert Telescope (BAT): 10-150 keV • detect ~ 150 GRB per year • onboard computation of positions • arc-minute positional accuracy • Dedicated telescopes for X-rays, UV, and optical afterglow follow up: • 0.3-10 keV X-ray Telescope (XRT) • 170-650 nm UV/Optical Telescope (UVOT) • 0.3-2.5 arc-second locations • existing hardware from JET-X and XMM • determine redshifts from X-ray absorption, lines, and Lyman-a cutoff • Rapid response satellite • 20 - 70 sec to slew within FOV of BAT • autonomous operations • factor 100 improved response time • continue monitoring of fading afterglow • rapid dissemination of the data Launch 5 Dec 2003

3. Latest on the Mission • Spacecraft delivered to GSFC (August 2002) • XRT & UVOT completed & calibrated (XRT: Sep 02;UVOT: Nov 02) • Integrated with the Spacecraft • BAT completed, under calibration and final delivery in May 2003 • Approved of a SWIFT GO program : • Cycle 1 AO via the ROSS-2003 NRA (Sep 1 2003; due Dec 1 2003) • Open only to scientists at US institutions (similar programs may be possible in UK & Italy): • Cycle 1 begin ~4.5 months after launch for about 12 months • Proposals are for the following areas : • New GRB projects not duplicative of team key projects & not requiring GI-specific observatory pointings • Correlative observations of GRB’s with other telescopes • Theoretical studies to advance mission science return on GRBs

4. Swift rapid data dissemination plan • Data are downloaded via : • TDRSS. Series of messages after a trigger is detected on board. • broadcast via the GCN (minutes) • later archived as part of the final Swift archive (< a week) • Malindi. All the telemetry. About 6 -7 download per day • Two processing types of the Malindi data (same pipeline software) • Quick look (output data is not part of the final archive): • Data processed every Malindi dump, e.g. the same observation is re-processed when new data arrive. • Data public in 3-6 hours after each dump (from FTP SDC). • Permanent archive: • Final processing after an observation (~ 1 day of data ) • Archived after all telemetry for observation is available ( < a week)

5. Overview Swift Data Archive • Archive includes all data levels, calibration, software and documents • Data are in FITS format OGIP standards • GIF, JPEG and HTML used for products preview or to record processing • No proprietary period for data • But check-out (~45 days) & verification mission phase (3 months) where data are available only to the Swift team • Archival data at HEASARC, ISAC (Italy) & LEDAS (UK) • Data transfers are via DTS protocol (adapted from XMM)

6. SwiftDataFlow MOC PSU SDC GSFC Swift Staging area Italy archive ISAC HEASARC archive UK archive LEDAS QL World

7. HEASARC responsibilities • Populate and maintain the Swift science and calibration data archive • Provide the following services • Public access to the Swift archive • Science data analysis software environment • Web server for the Swift Science Support Center • Database system for the Swift tables • Long-term archive • Generate GRB summary products • These responsibilities are documented in the Swift-HEASARC MOU (16 Oct 2001) • Technical agreement via ICD in place with the SDC, ISAC, BAT Team (completed and Signed) & MSSL (draft)

8. Archival Data • Production data populate the archive from the start of the mission start & delivered in less than a week : • TDRSS messages (SDC) - by trigger number • Level 1,2,3 production data (SDC) - by observation • Monitoring & Calibration data (SDC/SSC) - by type & CALDB • GRB summary data products (HEASARC) associated with the GRB catalog - by GRB • Populate the archive on routine basis 5 months after launch • Surveys : XRT (ISAC), BAT (BAT team), UVOT (MSSL) - by time • Populate the archive not earlier than 1 yr into the mission with updates every 3 months Data volume estimate is 1 Gygabyte/day (does not include the surveys or GRB products)

9. Database tables • Each data set has at least one associated table • Allow user to select and retrieve data • Record high level information related to the data set (position, time and other parameters) • Additional tables associated with the production data: • As-flown timeline (log times of specific events) • Instrument configuration tables (log modes used by each of the instruments during an observation) • All tables associated with the production data contain keys to relate: • table entries with data files (e.g. sequence number) • different tables (e.g. target ID) • Tables delivered daily for the production data • For the surveys and GRB products tables update with data delivery

10. Recent Activities • Modify and/or develop, package & distribute automatic protocols for the archive (http://legacy.gsfc.nasa.gov/dts/dts.html) : • To transfer/deliver data Data Transfer System (DTS vn. 6) • To ingest Swift databases (DBSI vn. 1) • To ingest data in the archive (Data Archive System, DAS vn. 1) • Test plans: for DTS & end2end archive procedure • Tests of the protocols (May-Jun, Sep-Oct 02) SDC, ISAC, HEASARC • Prototype new interface to access table information & data archive • Web prototype to ingest data from the follow-up Team • Ingest XRT & UVOT calibration data in their flight format • Ingest the Swift software in the HEAsoft package

11. Automatic ingest protocols • HEASARC modified DTS (originally written & operational for XMM) : • To cope with multiple sites data transfers • To facilitate transfers of large files • Add extra the security capabilities • HEASARC developed DAS to move data & tables from the staging area in the final archive location • DAS consists of perl scripts initiated by DTS • Includes capability to move & replace data • Makes a copy of the data set on a back-up storage • Sends e-mail to operator to ingest databases • Keeps log of the data in the archive • HEASARC developed scripts to ingest the tables (DBSI) • In the EXOSAT DBMS (used by the other DC sites) • Specific ingest procedures for the Swift tables

12. DTS test results

13. Swift archive interface • The Swift archive will be access via : • HEASARC Browse • Dedicated Swift archive page • The Swift archive page is ‘just’ a front end page to Browse: • To post news on the Swift archive • To provide up-front common Swift parameters for searches (besides position & time) • To link the other data archive centers related to Swift & the GCN • Query outputs from Browse and the Swift archive page will be identical (e.g. no changes in the underline software) • We are implementing Swift databases by relating in the DBMS the Swift tables to facilitate user query & the retrieve of information .

14. Follow-up Team Web site • Follow-up team collect data in response to a GRB alert • But there is no organized way to keep a record of these data • Record this information is important : • To log which observatories (and/or energy range) have performed /will perform observations for a given GRB • To help plan future Swift observations • To assemble the results (and/or data) together with the Swift results in the catalog (and in the archive) • We develop a WEB prototype to demonstrate a mechanism to submit information from the follow up team • The prototype was presented at the last Swift SWG • Currently gathering inputs & suggestions before final implementation

15. General Features • Input form based on the Remote Proposal System (e.g. RPS) : • Cover page with the team member information (automatically filled). • Observation form to input results and data file • clone or add or delete previous input results • Submit : verify the form and email back all input information • Observation form allows to : • Write results in pre-allocated boxes (makes easier to parse numbers) • Upload files (format, content, and naming convention TBD) • Provide a link (http &/or URL ) to a site or to the data • Write a note to be shown with the data • The input information is ingested in a database system : • It is tagged with GRB, team member and observatory name • Search capability via GRB name or time (uses Browse HEASARC)

16. Ingest : Calibration & Software • Calibration data • Delivery of the Panther XRT calibration data (25 Nov 2002) • First delivery of the UVOT calibration data (29 Nov 2002) These data are in telemetry format are used to test the reformatting to FITS and science software. • SSC delivered the Build 2 Swift software to the HEASARC : • It has been currently integrated within HEAsoft • Test distribution tar file (Build3 Feb, Build 4 April) to ensure • no-crosstalk between different software packages • Properly run on different platforms

17. This year’s milestones • DTS and end2end tests with UKDC (March 2003) • Final hardware configuration for the archive (May 2003) • Populate with the calibration data out the pipeline • Implementation of the Swift data archive access (June 2003) • Script framework to derive the GRB high level data products • Software distribution : First release (Sep 2003) Launch 5 Dec 2003

18. BeppoSAX Archive at HEASARC • HEASARC hosts currently 90% of the BeppoSAX public data: • Data are from the NFI mainly LECS & MECS, 90% of the PDS and 10% of HPGSPC • Data are delivered to the HEASARC via CDs by the BeppoSAX-SDC now ASDC (ASI Science Data center) • The ASDC is now ready to re-process the NFI with the latest software & calibration & it s currently processing the WFC data • Future delivery to the HEASARC : • WFC data FOT (not FITS, Starting from March 03) • WFC processed data (FITS, Starting from May 03) • NFI re-process data (FITS all NFIs, Starting in April 03) • Delivery via DTS (already tested)