1 / 17

HEASARC and Related Data Center Activities

HEASARC and Related Data Center Activities. Frank Marshall Acting HEASARC Director. HEASARC Overview. NASA’s archive for X-ray and Gamma ray data Established in Nov 1990 First wavelength specific “active” archive Partnership between GSFC and SAO (since 1999)

tarmon
Download Presentation

HEASARC and Related Data Center Activities

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. HEASARC and Related Data Center Activities Frank Marshall Acting HEASARC Director

  2. HEASARC Overview NASA’s archive for X-ray and Gamma ray data • Established in Nov 1990 • First wavelength specific “active” archive • Partnership between GSFC and SAO (since 1999) • Contains data from 26 missions • All data in FITS format, along with associated software and calibrations • Provides the necessary scientific and technical expertise for the use and interpretation of the data • Develop/maintain multi-mission analysis/support tools such as XSPEC, PIMMS, and Proposal Submission • Online access to all data, catalogs of observations and sources and browse data products • Defines and coordinates data, software, and media standards

  3. Laboratory for High Energy Astrophysics Organization LHEA Frank Jones (acting) Lois Workman 660 Instrument Development Office Lois Workman (acting) 660.2 Office of Guest Investigator Programs & Data Management Frank Marshall 660.1 Gamma Ray, Cosmic Ray & Gravitational Wave Astrophysics Branch Neil Gehrels 661 X-ray Astrophysics Branch Robert Petre 662

  4. HEASARC Overall Organization HEASARC provides the multi-mission infrastructure that is used by the GOFs and science support centers: Archive, database, web services, FITS standards,multi-mission software, & expertise Science staff are co-located with LHEA science branches

  5. Swift Astro-E2 Rossi XTE INTEGRAL Chandra XMM-Newton GLAST HEASARC Mission Support HEASARC provides the multi-mission infrastructure that is used by dedicated mission data centers (RXTE, Chandra, XMM-Newton, INTEGRAL, Swift, Astro-E2, and GLAST) Archive infrastructure, database support, web services, proposal software, FITS standards,multi-mission software, & science expertise Prevents duplication of effort and promotes reuse of software which result in cost savings to missions At the end of mission life the HEASARC maintains the archive, science expertise and software (e.g. CGRO, ASCA, ROSAT)

  6. HEASARC Budget & Staffing • Budget of $1.5M that mostly pays for people • LHEA Science Staff (4 USRA, 1 UMD, 3 GSFC): • Angelini (Restoration, BeppoSAX, Swift, XIMAGE, XRONOS) • Arnaud (XSPEC, XSELECT) • Corcoran (ROSAT, Caldb, HETE-2) • Drake (User support, EUVE, Archive population) • Lochner (E/PO) • McGlynn (Archive Scientist, VO, SkyView, ClassX) • Pence (FTOOLS, FITSIO, FITS standards, HERA, FV) • Frank Marshall (Acting Director while N. White is at HQ) • Two addition FTE at SAO (separately funded) • Science staff have 30% time for research • Successfully compete for observing time e.g. XMM-Newton and Chandra • Programming Staff (5 Software contractor, 1 GSFC) • Database, archive access, data ingest, data restoration, web services, XANADU analysis software, education & outreach, system support, and user support. • Additional funding competed for via e.g. AISRP

  7. HEASARC Co-located Data Centers • Provide dedicated support to specific missions utilizing HEASARC infrastructure • Develop mission specific software • Interface between community, instrument teams and/or foreign data center • Data processing and archive creation • Proposal support and grants administration • Science expertise in the specific mission • Currently there are 7 active GOFs/SSCs: • XMM-Newton, RXTE, BeppoSAX (ASI), INTEGRAL, Swift, Astro-E2, GLAST • 1 to 5 scientists per mission (typically 3), plus comparable number of programmers and support staff • Co-located staff at foreign data centers (XMM-Newton, INTEGRAL, Astro-E2) • Current total cost ~ $3M (depending on mission phase)

  8. SAO Connection • Coordinates SAO Chandra and HEASARC activities • Chandra calibration data & archive interface • Transparent access to Chandra archive from HEASARC Browse • DS9 Software • Remote Proposal Submission Software • Two FTE staff at SAO • Steve Murray directs the activity

  9. Actions from Previous HUG • Offsite archive backup. Backup in second building is planned. • Consistent statistics for outreach web-sites. Done. • Browse suggestions/concerns: plotting, cross-correlation radii, data products selection, exposure times, results formatting. Some implemented, others in process. • Need HETE-2 Data in FITS. Limited FITS support, see HETE-2 talk. • nH issues, limits. Link to MAST-supported alternative • Integral simulator and calibration concerns. See Integral Talk. • NVO impact on HEASARC. ADEC White Paper.

  10. Highlights • INTEGRAL launch • New load-balancing Web and database servers • 2500 Chandra and 1100 XMM datasets in archive by end of 2002. • Browse full integration of remote (Chandra) data products. • Bibliography/data links to and from ADS for XMM, RXTE, ASCA, ROSAT • XTE table and data product simplification. • GRBCAT and “Is it an X-ray source?” tools. • Outreach CD rated “outstanding” in NASA review • SkyView featured in Scientific American and Wired. • GRB follow-up VO prototype shown at AAS.

  11. Highlights: HEADCC Meeting • Coordinate activities of all high-energy astrophysics missions and centers. • Held Nov. 13, 2002 at CfA • Agreements • Work to ensure interoperability of Chandra and XMM data sets. • Coordinate development of POW, FV and DS9 viewing tools • Look at upgrading CALDB to meet requirements of Chandra and other newer missions. • Continue to work towards a single parameter interface layer. • Explore building standard model for coded aperture mask instruments.

  12. The Physical Archive Active Missions RXTE (1995- ) Chandra (1999- ) [data at CXC] HETE-2 (2000- ) Integral (2001-) XMM-Newton (1999- ) Past Missions Ariel 5 EXOSAT ASCA Ginga BBXRT BeppoSAX CGRO HEAO 1 Copernicus HEAO 3 COS B OSO 8 DXS ROSAT Einstein SAS 2 EUVE SAS 3 Vela 5B Upcoming Missions Swift (late 2003) Astro-E2 (2005) GLAST (2006) • Data from 26 missions currently in the archive • 344 astronomical catalogs & mission tables (up from 307 at last HUG meeting) • The archive volume was 2700 Gigabytes as of the end of 2002

  13. Usage & Data Statistics Gigabytes transferred per year 1998 1999 2000 2001 2002 FTP 842 1391 1846 2251 2481 WWW (http) 424 603 1083 1428 2931 Gigabytes retrieved per quarter Browse tar file directory

  14. Data Transfer Rates (ftp & http) The ftp transfer rate in 2002 averaged 6.8 GB per day, up from 6.2 GB per day in 2001. Data transferred was predominantly RXTE (56%) with other missions of note, ASCA and ROSAT (3.7% each), CGRO (3.2%) and XMM-Newton (2.7%). Significant data transfers were also made from non-mission directories such as Software (17%) and Retrieve (13%). Data Attractiveness (DA, defined for each mission as data ftped in a year divided by the archive size): RXTE DA=129% ROSAT DA=63% CGRO DA=37% XMM DA=29% The http transfer rate of Web pages, images, etc., in 2002 was 8 GB per day (244 GB per month), compared to 3.9 GB per day (119 GB per month) in 2001. The http transfer rate in 2002 was 118% of the ftp rate - the first time the http transfers exceeded the ftp transfers. Monthly transfers from the HEASARC’s EPO areas “Imagine the Universe” and “StarChild” averaged ~500,000 requests and ~1,100,000 requests during the school years, respectively. They get fewer hits during the summer.

  15. HEASARC Data Archive As of November 2002, the HEASARC’s public data holdings (the anonymous ftp area) were about 2688 GB, including some still-proprietary datasets. (Duplicate ROSAT and ASCA datasets totaling ~160 GB have been removed from these statistics.) The biggest mission archives are RXTE (40% of the total) and ASCA (20%), with XMM-Newton and CGRO each ~8%, ROSAT and BeppoSAX each ~5%, and EXOSAT and HETE-2 just under 4% each. In the last 12 months, the HEASARC archive growth was about 1.4 GB per day (~500 GB over the year), to which the rapid growth of the XMM-Newton archive and the opening of the HETE-2 archive were major contributors. The growth rate in 2003 is expected to be similar.

  16. Future Archive Growth In the next 3 years, there are several new missions that plan to archive data at the HEASARC: Swift (late 2003 launch) Astro-E2 (early 2005 launch) GLAST (fall 2006 launch) These new missions are expected to each generate 400-1000 GB of data per year so the HEASARC archive doubling time is predicted to be about 2.5 years. Very rough HEASARC archive size projections: Date Archive size 2004.0 4120 2005.0 6055 2006.0 7990

  17. Summary and Prospects • The HEASARC is a mature system with diverse and growing holdings. • Archive and software systems continue to evolve to meet the needs of the community • Over the next several years, many of the newer missions will operate in the hard X-ray and gamma-ray regimes. • Science goals, operations and other requirements somewhat different from many earlier missions. • The diversity of the HEASARC holdings makes it a smaller scale ‘Virtual Observatory’ • Links to other NASA and ground data enhances value of HEASARC holdings • HEASARC experience can be valuable in guiding VO efforts • Conversely, more generic VO approaches may be useful within the HEASARC.

More Related