Hubble Legacy Archive Lee Quick - TIPS meeting July 19, 2012

1. Hubble Legacy ArchiveLee Quick - TIPS meeting July 19, 2012 • Goals • Data • History • Current Work • Demo

2. Goals of the HLA Project • Provide intuitive searching, browsing and download of Hubble data • Create online, enhanced data products for all Hubble data • Develop enhanced user interface capabilities • hla.stsci.edu

3. Hierarchy of HLA Image Products • Level 1 data are the individual exposures, projected ("drizzled") onto a common frame for each visit. • Level 2 data are the combined images (exposures with the same filter, same camera, and within the same visit). • Level 3 images are deep combined images or mosaics creating by combining data taken in different visits to the target. • Level 4 data are false color images (a combination of two or three colors, depending on what is available). • Level 5 data are the community-contributed high-level science products (HLSP). • Level 0 (calibrated, unprocessed) are not available for direct searches but may often by found using the More... link.

4. HLA - Data Holdings

5. HLA (a brief) History

6. HLA Early Data Release - August 2007 • Data online for immediate access • Footprint service to make it easier to browse and download • Interactive image display • HLA combined images for ~25% of ACS data • Prototype ACS source lists • NICMOS grism extractions for ~10% of G141 data (produced at ST-ECF)

7. HLA Data Release 1 – February 2008 • NEW DATA PRODUCTS • HLA combined images for ~90% of ACS data • NICMOS grism extractions for ~80% of available data (produced at ST-ECF) • DAOphot source lists for ~50% of ACS images • Prototype SExtractor source lists for ~10% of ACS images • A few prototype combined WFPC2 images (produced at CADC • USER INTERFACE ENHANCEMENTS • Footprints for ACS, WFPC2, STIS, NICMOS, FOS, and GHRS • STIS images accessible in HLA interactive display • Non-HLA data products included in inventory and images, accessible through DADS • Ability to filter inventory table to select observations of interest • Readout of mouse RA/Dec position in interactive display • Plot cuts of rows or columns through images (or 2-D spectra) in interactive display • Introductory training movies on how to use the HLA

8. HLA Data Release 2- September 2008 • NEW DATA PRODUCTS • Combined HLA images for ~95% of WFPC2 data (produced at CADC) • Combined HLA images for ~95% of ACS observations • Both DAOphot and SExtractor source lists for 70% of ACS observations • USER INTERFACE ENHANCEMENTS • (Shopping) Cart to allow users to download multiple files at once • Enhanced plotting capabilities • Additional catalog overlays in interactive display (SDSS, 2MASS, GSC2, FIRST) • User defined search lists • Much faster footprint interface • Additional advanced search options (Proposal ID, Moving Targets, Filter)

9. HLA Data Release 3 – May 2009 • NEW DATA PRODUCTS • HLA drizzled combined images for ~30% of NICMOS data • Sample of 1235 ACS extracted grism spectra (provided by ST-ECF) • Preliminary DAOphot and SExtractor source lists for ~80% of WFPC2 images • Sample prototypes of ACS multi-visit mosaics • USER INTERFACE ENHANCEMENTS • User search list positions are accepted in a variety of input formats • Selected datasets (e.g., from clicking on footprints) can be added to cart • Selection of datasets can be used for sorting and filtering (e.g., to show selected data at the top of the table) • Cart can be used to retrieve both DADS and HLA data products • Cart includes estimated download data volume • GALEX catalog is available for overlay in interactive display • Advanced contrast controls in interactive display can be used to change color balance

10. HLA Data Release 4 – March 2010 • NEW DATA PRODUCTS • Fully reprocessed NICMOS data • Fully reprocessed WFPC2 data (produced at CADC) with new source lists • Access to WFC3 Early Science release data • New high-level science products • Additional ACS multi-visit mosaics • Updated ACS combined images • USER INTERFACE ENHANCEMENTS • Plotting tool allows for more dynamic user interaction, including zooming and panning • Spectra from FOS and GHRS are displayed in the plotting tool using the "Interactive Display" links in the inventory and images views. • COS and WFC3 observations are included in the search results and footprints • Proprietary data are included in the search results plus footprints and can be retrieved from DADS (by privileged users) via the HLA cart • Search by IPPPSOOT exposures and HST visit names (propid_visit) • Inventory table updates • Consolidated Help Center holds HLA resources such as FAQ, Help, Movies etc., and includes a new HLA documentation search tool. • Source lists are dynamically generated from our databases, which improves consistency between the various catalog sources and makes it possible to download them from the inventory • Search results get automatically gzip-compressed when the browser permits it (which is usually the case). • The interactive display shows the original image pixel value in the coordinate box when the mouse cursor stops moving.

11. HLA Data Release 5 – March 2011 • USER INTERFACE ENHANCEMENTS • A completely new footprint interface, featuring: • Zooming in/out capability • Panning • Fully integrated table, so that selections in one screen remain valid in the others • Selection across instruments possible • Option to search through proprietary and/or non-proprietary data • Improved download flow from shopping cart • Single or sequential download • Partial downloads can be restarted • Sources clickable in interactive display • When filtering RA or Dec columns, the values may be entered either in decimal degrees or in "sexagesimal" format (hh:mm:ss, dd:mm:ss) • Multiband source list (ACS and WFPC2) • NEW DATA PRODUCTS • Completed WFPC2 processing (over 300 new visits with source lists) • New High Level Science Products • ECF-generated ACS and NICMOS extracted GRISM data hosted at STScI

12. HLA Data Release 6 – January 2012 • NEW DATA PRODUCTS • WFC3 image data: includes ~62% of data from observations that have a release date up to 11-30-2011 • New High Level Science Products • Additional ACS (updated to release date 2-16-2011), WPFC2 and NICMOS data • USER INTERFACE ENHANCEMENTS • Interactive Display enhancements. • The inventory and images interfaces now include a "Select All" button that can be used to select all the datasets in the current list. • The HLA Simple Image Access Protocol server is now fully compliant with the Virtual Observatory standards. The HLA SIAP server will be added to the VO registry for easy access to HLA data from scripts and other VO tools. • Searches using lists of positions now show the number of lines in the list.

13. Current work – WFC3 Source lists • Based on existing HLA pipelines (ACS, NICMOS) with structural changes • Some algorithmic changes • New alignment module performs very well on crowded fields • Sky equalization included • More rigorous handling of header information • Source lists are undergoing quality review, will be released later

14. Current work – Spectral Data • Strong desire to better integrate spectral data in the HLA • General purpose spectral pipeline not yet available - focus on High Level Science Products • First release will include collections of processed spectra (e.g., STARCAT) • Interface modifications to handle spectral products • Preview code replaces image previews and cutouts • New Spectral Plotting tool with better functionality to view and interact with spectra • Difficulties: • Spectral data products inhomogeneous in properties, combination process • Naming conventions in development • Large number of data points in some spectra can challenge some browsers • User preferences not yet established - may need to iterate on specifics • Expect ~200 products at first release