1 / 26

Science Data Products

Science Data Products. Daryl Swade DMS S ystems Engineer. Intro/Overview. Science Data Product design shows what the DMS software should produce . Software design then shows how to generate data products. JWST science data products generated by DMS are dynamic due to

adelio
Download Presentation

Science Data Products

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Science Data Products Daryl Swade DMS Systems Engineer S&OC System Design Review #1

  2. Intro/Overview • Science Data Product design shows what the DMS software should produce. • Software design then shows how to generate data products. • JWST science data products generated by DMS are dynamic due to • updates to calibration reference data and algorithms, • increased knowledge of the science instruments learned in the early stages of the mission, and • DMS software error corrections. • Intent of DMS Science Data Product Generation is to provide data users with basic reduced data products they need to begin detailed analysis. • DMS data products conform to Flexible Image Transport System (FITS) convention. • standard archival format for astronomical data S&OC System Design Review #1

  3. DMS Data Flow – Data Product Generation S&OC System Design Review #1

  4. JWST Up-the-Ramp Readout S&OC System Design Review #1

  5. Data Level Definition S&OC System Design Review #1

  6. Science Data Product Inputs S&OC System Design Review #1

  7. Generation of Original Exposure Data Files S&OC System Design Review #1

  8. Level 1 FITS File Structure S&OC System Design Review #1

  9. Science Data Cube single exposure, single detector S&OC System Design Review #1

  10. Image Orientation Detector and science image frame orientations as seen looking through the optics toward the sky NIRCam MIRI NIRSpec red – detector frame blue – science frame green – apertures NIRISS FGS1 FGS2 Transformation requires rotation, parity flip, and/or shift. S&OC System Design Review #1

  11. Information Flow to Level 1 FITS Headers S&OC System Design Review #1

  12. Level 1a-Level 1b Header Keyword Population S&OC System Design Review #1

  13. Files in an Exposure Data Set S&OC System Design Review #1

  14. Exposure File Naming Convention • At data levels 1 and 2, the science data files are FITS format files that contain the pixel values • for a single exposure • from a single detector. • Goals for file name: • DMS data user able to map the data file to the observation input through APT. • Operations staff can map every exposure to a construct in the JWST commanding scripts. • The file name is based on the following exposure hierarchy: • Program (PPPPP) • Observation number (OOO) • Visit (VVV) • Visit Group (GG) • Parallel Sequence id (S, 1 prime, 2-5 parallel) • Activity (AA, base 36) • Exposure number (EEEEE) • Example: • jw<PPPPP><OOO><VVV>_<GGSAA>_<EEEEE>_<detector >_<suffix> .fits • jw12345012009_01101_00012_nrcalong_uncal.fits • Visit ID - <PPPPP><OOO><VVV> - explicitly embedded in exposure file name • Exposure number resets with each new activity • The exposure number within the file name does not necessarily monotonically increase for each exposure within a single visit. S&OC System Design Review #1

  15. Guide Star Pipeline • Guide Star Pipeline detailed design to be addressed in DMS SDR4 • Jitter data file generated by Guide Star Pipeline • Separate process from science data pipelines • Data product created at visit level • Guide Star (GS) acquisition images • About GS - Full frame image, Acq1: 128x128, Acq2: 32x32 • At science attitude: Acq1: 128x128, Acq2: 32x32 • Fine guide box: 8x8 @ 16Hz (64 msec.) • May not want to save all 8x8 fine guide data • Centroids in engineering data for jitter • Include additional pointing information in engineering data S&OC System Design Review #1

  16. NIRSpec MSA Data Flow S&OC System Design Review #1

  17. NIRSpec MSA Metadata File Structure Data array for each shutter planned open or closed Slitlet information, background shutter, fraction of source in shutter Source id, source name, alias, RA, Dec S&OC System Design Review #1

  18. Generation of Calibrated Exposure Data Files S&OC System Design Review #1

  19. Level 2 FITS File Format S&OC System Design Review #1

  20. Level 3 Data Products - Associations • Relationships between multiple exposures are captured in an association. • An association is a means of identifying a set of exposures that belong together and may be dependent upon one another. • The association concept permits exposures to be calibrated, archived, retrieved, and reprocessed as a set rather than as individual objects. • For initial processing, an association and its level 3 data products will not extend beyond a single observation. • Design will include the capability to support multi-observation and multi-program associations. • For each association, DMS will generate the most combined and least combined data products. • Intermediate data products will not be saved. • For imaging associations, the most combined will be either a mosaic or a dither product. S&OC System Design Review #1

  21. Association Components S&OC System Design Review #1

  22. Association Data Products • Association Pools • Created by Science Data Processing • Contains a list of all potential exposures that may form an association • Association Table • Created in calibration pipeline environment from Association Pool information • Contains list of exposures to be used to create a single Level 3 Data Product • Level 3 Data Products • Formed from exposures listed in Association Table S&OC System Design Review #1

  23. Association Processing • All exposures – even single, unassociated exposures – will have a level 3 data product. • Well-established image rectification software (Drizzle) will be run at level 3 on all exposures. • All data will get a source based, level 3 file name. S&OC System Design Review #1

  24. Level 3 Image FITS File Format Most combined image FITS table of selected keywords from input exposures Additional extensions to be defined by calibration working group S&OC System Design Review #1

  25. Level 3 Data Product File Name • Shift in naming convention for level 3 data product file names • The association data product name will provide an indication of the file content. • Based on data discovery lessons learned from MAST High Level Science Product (HLSP) and Hubble Legacy Archive (HLA) experience, • A proposed name for the JWST level 3 data products was patterned after the HLSP data file names. (http://archive.stsci.edu/hlsp/hlsp_guidelines.html#Filenames). • The format for the level 3 data product file name would be: jw<ppppp>_<target(epoch)>_<science_instrument>_ <optical_element>_ <product_name>.fits • <ppppp> is the program identifier • <target> is the target name, which may contain proposer defined non-alphanumeric characters and an epoch • <science_instrument> is nircam, nirspec, miri, niriss, or fgs • < optical_element > is the optical element or optical element combination used • <product_name> indicates the type of data product, e.g., image • Examples from SODRM: • jw94010_m31_nircam_f356w_image.fits • jw95020_cdfs-07-e1_nircam_f115w_image.fits • jw93060_w51a+g49.5-0.4_miri_f1500w_image.fits S&OC System Design Review #1

  26. Data Level Format Summary S&OC System Design Review #1

More Related