1 / 22

Data Access Layer Working Group Image and Spectral Access

National V IRTUAL O BSERVATORY. Data Access Layer Working Group Image and Spectral Access. Doug Tody National Radio Astronomy Observatory. Source Catalog. Table. Event List. Dataset. Visibility Data. Image. NDImage. 1D Spectrum. SED. Time Series. DAL Services.

monet
Download Presentation

Data Access Layer Working Group Image and Spectral Access

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. National VIRTUAL OBSERVATORY Data Access Layer Working Group Image and Spectral Access Doug Tody National Radio Astronomy Observatory NVO Summer School, Aspen Sept 14 2004

  2. Source Catalog Table Event List Dataset Visibility Data Image NDImage 1D Spectrum SED Time Series DAL Services NVO Summer School, Aspen Sept 14 2004

  3. Simple Image Access (SIA) • Provides access to "image" data • (instead of spectrum, catalog, etc.) • regularly sampled (pixelated) data • generally an image of the sky, with a WCS • Service-oriented data discovery • query service to discover data • Access to image metadata • can get image metadata without retrieving the actual image • uniform description based on standard data models • Access to image datasets • data may be virtual or computed on demand • uniform interface to any type of image data NVO Summer School, Aspen Sept 14 2004

  4. SIA - Basic Usage • Simplest possible query • HTTP GET, e.g., http://myvo.nvo.org/SIAPServlet?POS=12.0,10.0&SIZE=0.2 • similar to cone search but ROI is a rectangle defining the ideal image coverage on the sky, not merely a search region • Query response • VOTable describing images, one image per row • for each image there is an access reference, a URL • Get Data • simply fetch the image at the given URL • SIA is deceptively simple! It can do a lot more than is apparent, but simple usage should be kept simple. NVO Summer School, Aspen Sept 14 2004

  5. SIA - Interface Concepts • Service protocol independent • URL (REST), WS, ADQL all implement the same interface model • document-oriented, pass through • Relational query model • relational: flat table, relationships inferred from metadata • generally it is up to the client to refine the query • Uniform access to data • atlas, cutout, etc., images all treated the same • mediation to a standard data model (partial currently) • Image service types • cutout, mosaic, atlas, pointed NVO Summer School, Aspen Sept 14 2004

  6. SIA - Interface Concepts • Virtual data • most VO data analysis uses virtual data • virtual data is generated on the fly by the service • may involve subsetting, filtering, transformation, analysis • Data Model-based data access • addresses problem of heterogeneous data • allows disparate software to share the same data • same physical data can be viewed via different models, e.g., image or spectrum NVO Summer School, Aspen Sept 14 2004

  7. SIA - Interface Summary • Query • Simple positional query • POS, SIZE • Image FORMAT • FITS, graphic, HTML, metadata • Image generation parameters • fully specify projection on the sky but simplify FITS WCS • naxis, cframe, equinox, crpix, crval, cdelt, rotang, proj • defaults are derived from POS, SIZE • Others • Intersect (covers, enclosed, center, overlaps) • service defined, e.g., filter or bandpass name NVO Summer School, Aspen Sept 14 2004

  8. SIA - Interface Summary • Query response • Response is a VOTable • One candidate image per table row • Includes standard metadata, including WCS • title, date, pos, naxes, naxis, scale, format, etc. • FITS WCS parameters • subset but includes CD matrix • spectral bandpass metadata • service processing metadata • did service interpolate pixels? NVO Summer School, Aspen Sept 14 2004

  9. SIA - Interface Summary • Get Data • The image "access reference", a URL, is used to fetch the dataset • URL often points to a service which generates data on-the-fly (OTF) • e.g., image cutout or mosaic • A separate get is required for each image • Note the query and get may be performed by different clients • multiple get operations may proceed concurrently • Use of URL permits caching of images NVO Summer School, Aspen Sept 14 2004

  10. SIA - Interface Summary • Staging Data • Included in SIA interface design, but not yet implemented • Asynchronous staging of data is required for large computations • e.g., a large image mosaic, or generation of 10000 cutouts • Interface • same as for synchronous image access (same query, getData) • adds accessImage method, messaging, polling, multiple clients • third party delivery possible, including MySpace NVO Summer School, Aspen Sept 14 2004

  11. SIA - Future Work • Advanced queries • query on additional image metadata, e.g., collection, bandpass, time • syntactical queries (ADQL), queries on virtual tables • Extended data model • metadata standardization (UCD normalization) • dataset characterization, identification, provenance • image subtypes, e.g., image cube, synoptic imagery • Query response • intelligent ranking of query response (like Google) • logical grouping of related images, e.g., multi-band survey data • metadata extension mechanism, e.g., as for AVO demo • Data Access • Staging of data, authentication NVO Summer School, Aspen Sept 14 2004

  12. Simple Spectral Access • Provides access to "spectral" data • similar to SIA but deals with tabular spectrophotometric data • Service-oriented data discovery • query service to discover data • Access to dataset metadata • can get dataset metadata without retrieving actual dataset • uniform interface based on standard data models • Access to actual dataset • data may be virtual, i.e., computed on demand • uniform interface to any type of spectral data • hides details of how data is stored or represented externally NVO Summer School, Aspen Sept 14 2004

  13. SSA - Basic Usage • Simplest possible query • HTTP GET, e.g., http://myvo.nvo.org/SSAPServlet?POS=12.0,10.0&SIZE=0.2 • other query types, e.g., WS, or ADQL in the future, also possible • Query response • VOTable describing spectral datasets, one per row • for each dataset there is an *access reference*, a URL • Get Data • simply fetch the dataset at the given URL • returned data adheres (normally) to a standard data model and data format • Data Format • A returned 1D spectrum may be a simple VOTable (or text file, or FITS binary table, etc.) with some general metadata followed by a simple spectrum table with wavelength, flux, and uncertainty columns. • Once again, although basic usage is simple, the interface can do more than is immediately apparent. NVO Summer School, Aspen Sept 14 2004

  14. SSA - Interface Scope • SSA deals with several types of data • Spectral Energy Distributions (SEDs) • 1D spectra • time series • Why this grouping? • common spectrophotometric data model • all are sampled, spectrophotometric, tabular data NVO Summer School, Aspen Sept 14 2004

  15. SSA Data Model • Sampled spectrophotometric sequence • projected at constant time results in 1D spectrum • projected at constant spectral value results in time series • projecting both results in a photometry point • A SED is: • a collection of these three types of objects • at a constant point on the sky (usually!) • typically spanning a wide range of spectral values • Summary • a SED attempts to describe the full spectral energy distribution of an object, encompassing as much of the emitted energy as possible NVO Summer School, Aspen Sept 14 2004

  16. SED Composition Photometry point spectrum segment NVO Summer School, Aspen Sept 14 2004

  17. NVO Summer School, Aspen Sept 14 2004

  18. OTF Generation of Spectral Data • Spectral Archives • spectral data resembles catalog data as much as image data • most spectral data access will probably be to pre-computed data • Virtual Data examples • spectrum from an image cube • SED from multi-band image data (plus catalog data etc.) • time series from synoptic imagery or catalog data • spectrum or time series from radio Pulsar data NVO Summer School, Aspen Sept 14 2004

  19. SSA Data Formats • Concepts • science data model (SDM) • semantic model for the data - what IS this data • export data format (EDF) • expresses the SDM in a specific data representation • identically the same SDM regardless of representation • Formats • native XML • VOTable • FITS binary table • text table, e.g., CSV • plus graphics, HTML • spectral data can also be viewed as an image, with restrictions NVO Summer School, Aspen Sept 14 2004

  20. SSA Service Interface • Each class of data gets a separate interface • SED, spectrum, time series • similar but separate access interfaces preferred • Similar to SIA • query, query response, getData • additional query parameters NVO Summer School, Aspen Sept 14 2004

  21. SSA Query • Required parameters • POS, SIZE, FORMAT • region is circular, as for cone search, unlike SIA • FORMAT provides more options than just FITS for the EDF, including XML (native and VOTable), and text • Optional parameters • time, bandpass, collection, ID, rank • aperture, verbosity NVO Summer School, Aspen Sept 14 2004

  22. SSA Query Response • VOTable • one table row per candidate dataset • access reference (URL) used to fetch data • component data models included directly as objects • uses GROUP, UTYPE from VOTable 1.1 NVO Summer School, Aspen Sept 14 2004

More Related