Science Data Products

1. Science Data Products Daryl Swade DMS Systems Engineer S&OC System Design Review #1

2. Intro/Overview • Science Data Product design shows what the DMS software should produce. • Software design then shows how to generate data products. • JWST science data products generated by DMS are dynamic due to • updates to calibration reference data and algorithms, • increased knowledge of the science instruments learned in the early stages of the mission, and • DMS software error corrections. • Intent of DMS Science Data Product Generation is to provide data users with basic reduced data products they need to begin detailed analysis. • DMS data products conform to Flexible Image Transport System (FITS) convention. • standard archival format for astronomical data S&OC System Design Review #1

3. DMS Data Flow – Data Product Generation S&OC System Design Review #1

4. JWST Up-the-Ramp Readout S&OC System Design Review #1

5. Data Level Definition S&OC System Design Review #1

6. Science Data Product Inputs S&OC System Design Review #1

7. Generation of Original Exposure Data Files S&OC System Design Review #1

8. Level 1 FITS File Structure S&OC System Design Review #1

9. Science Data Cube single exposure, single detector S&OC System Design Review #1

10. Image Orientation Detector and science image frame orientations as seen looking through the optics toward the sky NIRCam MIRI NIRSpec red – detector frame blue – science frame green – apertures NIRISS FGS1 FGS2 Transformation requires rotation, parity flip, and/or shift. S&OC System Design Review #1

11. Information Flow to Level 1 FITS Headers S&OC System Design Review #1

12. Level 1a-Level 1b Header Keyword Population S&OC System Design Review #1

13. Files in an Exposure Data Set S&OC System Design Review #1

14. Exposure File Naming Convention • At data levels 1 and 2, the science data files are FITS format files that contain the pixel values • for a single exposure • from a single detector. • Goals for file name: • DMS data user able to map the data file to the observation input through APT. • Operations staff can map every exposure to a construct in the JWST commanding scripts. • The file name is based on the following exposure hierarchy: • Program (PPPPP) • Observation number (OOO) • Visit (VVV) • Visit Group (GG) • Parallel Sequence id (S, 1 prime, 2-5 parallel) • Activity (AA, base 36) • Exposure number (EEEEE) • Example: • jw<PPPPP><OOO><VVV>_<GGSAA>_<EEEEE>_<detector >_<suffix> .fits • jw12345012009_01101_00012_nrcalong_uncal.fits • Visit ID - <PPPPP><OOO><VVV> - explicitly embedded in exposure file name • Exposure number resets with each new activity • The exposure number within the file name does not necessarily monotonically increase for each exposure within a single visit. S&OC System Design Review #1

15. Guide Star Pipeline • Guide Star Pipeline detailed design to be addressed in DMS SDR4 • Jitter data file generated by Guide Star Pipeline • Separate process from science data pipelines • Data product created at visit level • Guide Star (GS) acquisition images • About GS - Full frame image, Acq1: 128x128, Acq2: 32x32 • At science attitude: Acq1: 128x128, Acq2: 32x32 • Fine guide box: 8x8 @ 16Hz (64 msec.) • May not want to save all 8x8 fine guide data • Centroids in engineering data for jitter • Include additional pointing information in engineering data S&OC System Design Review #1

16. NIRSpec MSA Data Flow S&OC System Design Review #1

17. NIRSpec MSA Metadata File Structure Data array for each shutter planned open or closed Slitlet information, background shutter, fraction of source in shutter Source id, source name, alias, RA, Dec S&OC System Design Review #1

18. Generation of Calibrated Exposure Data Files S&OC System Design Review #1

19. Level 2 FITS File Format S&OC System Design Review #1

20. Level 3 Data Products - Associations • Relationships between multiple exposures are captured in an association. • An association is a means of identifying a set of exposures that belong together and may be dependent upon one another. • The association concept permits exposures to be calibrated, archived, retrieved, and reprocessed as a set rather than as individual objects. • For initial processing, an association and its level 3 data products will not extend beyond a single observation. • Design will include the capability to support multi-observation and multi-program associations. • For each association, DMS will generate the most combined and least combined data products. • Intermediate data products will not be saved. • For imaging associations, the most combined will be either a mosaic or a dither product. S&OC System Design Review #1

21. Association Components S&OC System Design Review #1

22. Association Data Products • Association Pools • Created by Science Data Processing • Contains a list of all potential exposures that may form an association • Association Table • Created in calibration pipeline environment from Association Pool information • Contains list of exposures to be used to create a single Level 3 Data Product • Level 3 Data Products • Formed from exposures listed in Association Table S&OC System Design Review #1

23. Association Processing • All exposures – even single, unassociated exposures – will have a level 3 data product. • Well-established image rectification software (Drizzle) will be run at level 3 on all exposures. • All data will get a source based, level 3 file name. S&OC System Design Review #1

24. Level 3 Image FITS File Format Most combined image FITS table of selected keywords from input exposures Additional extensions to be defined by calibration working group S&OC System Design Review #1

25. Level 3 Data Product File Name • Shift in naming convention for level 3 data product file names • The association data product name will provide an indication of the file content. • Based on data discovery lessons learned from MAST High Level Science Product (HLSP) and Hubble Legacy Archive (HLA) experience, • A proposed name for the JWST level 3 data products was patterned after the HLSP data file names. (http://archive.stsci.edu/hlsp/hlsp_guidelines.html#Filenames). • The format for the level 3 data product file name would be: jw<ppppp>_<target(epoch)>_<science_instrument>_ <optical_element>_ <product_name>.fits • <ppppp> is the program identifier • <target> is the target name, which may contain proposer defined non-alphanumeric characters and an epoch • <science_instrument> is nircam, nirspec, miri, niriss, or fgs • < optical_element > is the optical element or optical element combination used • <product_name> indicates the type of data product, e.g., image • Examples from SODRM: • jw94010_m31_nircam_f356w_image.fits • jw95020_cdfs-07-e1_nircam_f115w_image.fits • jw93060_w51a+g49.5-0.4_miri_f1500w_image.fits S&OC System Design Review #1

26. Data Level Format Summary S&OC System Design Review #1