1 / 20

VIRCAM CPL: Lessons Learned

18 April 2007. Second Generation VLT Instruments. 2. Introduction to CASU. Small group within the Institute of Astronomy, specialising in survey astronomy.Pipeline reduction of imaging dataAPM (Schmidt Plates 1 x 40k x 40k)INT Wide Field Camera (4 x 2k x 4k)ESO WFI on 2.2m at La Silla (8 x 2k x

bayle
Download Presentation

VIRCAM CPL: Lessons Learned

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

    1. 18 April 2007 Second Generation VLT Instruments 1 VIRCAM & CPL: Lessons Learned Jim Lewis and Peter Bunclark Cambridge Astronomy Survey Unit

    2. 18 April 2007 Second Generation VLT Instruments 2 Introduction to CASU Small group within the Institute of Astronomy, specialising in survey astronomy. Pipeline reduction of imaging data APM (Schmidt Plates 1 x 40k x 40k) INT Wide Field Camera (4 x 2k x 4k) ESO WFI on 2.2m at La Silla (8 x 2k x 4k) MOSAIC-1 on KPNO 4m (8 x 2k x 4k) MOSAIC-2 on Blanco 4m at CTIO (8 x 2k x 4k) AAO WFI on AAT (8 x 2k x 4k) CIRSI on INT (4 x 1k x 1k) INGRID on WHT (1k x 1k) UFTI on UKIRT (1k x 1k) WFCAM on UKIRT (4 x 2k x 2k)

    3. 18 April 2007 Second Generation VLT Instruments 3 Introduction to VISTA and VDFS VISTA: 4m survey telescope on Paranal VIRCAM: The IR camera for VISTA 16 Non-buttable 2k x 2k detectors VDFS: PPARC funded facility to provide an end-to-end data-flow system for VISTA and WFCAM. Quality control and calibration pipelines Paranal and Garching Science pipeline for full calibration of science data. Cambridge Science archive acts as the point of access of the reduced data. Plus some further processing. WFAU, Edinburgh

    4. 18 April 2007 Second Generation VLT Instruments 4 Data Flow Raw telescope data is assessed by the summit pipeline (QC1) Shipped to Garching (discs) Shipped to CASU for science reduction and calibration. (discs) Calibrated data shipped to Edinburgh for archiving (ftp)

    5. 18 April 2007 Second Generation VLT Instruments 5 IR Data Reduction Worries IR detectors are currently inherently more unstable than optical CCDs. Some odd electronic effects Much poorer cosmetic quality than CCDs Sky emission > 100x brighter than most objects And it’s variable both spatially and temporally! Exposure times are short, so data rates are very high. 200-500 Gb/night expected for VISTA public surveys Rice tile compression can save factors of 3-4 in 32 bit integer data

    6. 18 April 2007 Second Generation VLT Instruments 6 VDFS Pipeline Recipes Create master calibration frames (dark, twilight flats, confidence maps, etc) Linearity analysis Detector noise & dark current properties Persistence and crosstalk analysis Illumination correction analysis Full reduction recipes for standard star and programme fields

    7. 18 April 2007 Second Generation VLT Instruments 7 Summit & Garching Pipelines QC1 parameters e.g. photometric zeropoints, astrometric fit quality Written using ESO qfits/CPL infrastructure Both use the same software modules Amount of processing can be scaled down Calibration images (flats etc) from a master library.

    8. 18 April 2007 Second Generation VLT Instruments 8 Cambridge Pipeline Full reduction e.g. better sky correction Results shipped to WFAU Some of the same CPL based modules reused. Some additional modules required.

    9. 18 April 2007 Second Generation VLT Instruments 9 Current Status of VISTA Pipelines Version 0.5 released to ESO in March Full compliment of recipes (almost!) PAE

    10. 18 April 2007 Second Generation VLT Instruments 10 The Decision To Use CPL (1) We were given the choice. We already had advanced pipeline software in our own environment for WFCAM and other IR instruments. First view of CPL and qfits was not favourable.

    11. 18 April 2007 Second Generation VLT Instruments 11 The Decision To Use CPL (2) CPL v2.0 was a great improvement CASU has an ‘ESO workstation’ to generate ‘data’ using the VIRCAM acquisition software Integration into Paranal and Garching systems greatly simplified by using CPL

    12. 18 April 2007 Second Generation VLT Instruments 12 Things I wish I had known…

    13. 18 April 2007 Second Generation VLT Instruments 13 My First Attempt At A Recipe INPUT: FITS image of WFCAM data Binary FITS table of astronomical objects Binary FITS tables containing the 2MASS catalogue TASK: Work out RA, Dec coverage of image using header FITS WCS keys Extract 2MASS standards Cross match standards to object catalogue Fit a WCS Write resulting WCS to FITS image

    14. 18 April 2007 Second Generation VLT Instruments 14 World Coordinate Systems CPL has no WCS facility But it soon will

    15. 18 April 2007 Second Generation VLT Instruments 15 Input File Access CPL doesn’t give write access to input files. cpl_frame, cpl_image Data abstraction is not perfect cpl recipes can’t be written to update a file

    16. 18 April 2007 Second Generation VLT Instruments 16 CPL vs FITS tables CPL does not allow for direct manipulation of FITS tables cpl_table structures can be created from a FITS table. But one must read the WHOLE table even though you might only be interested in a few rows or columns. Tables with many rows take too long to read Break them up Tables with many character type entries cause the system to run out of memory pointers Remove unnecessary character columns

    17. 18 April 2007 Second Generation VLT Instruments 17 User Documentation Reference manual, users guide, software developers kit User’s guide and SDK are helpful but limited in scope Very few examples Get a copy of a working CPL pipeline

    18. 18 April 2007 Second Generation VLT Instruments 18 QC Garching and Summit pipelines generate QC parameters Provides data for QC trending to be done both on site and in Garching The ‘ESO System’ is really two systems QC passed through FITS headers (Garching) QC passed through PAF files (Chile) Neither system appears to be documented. PAF files contents are not documented in CPL users or reference manuals. CPL has no standard class for passing QC parameters

    19. 18 April 2007 Second Generation VLT Instruments 19 File Tags DPR TYPE OBSTYPE PRO CATG Product type No two recipes can produce the same value DO CATG File type for files used as input to recipes Raw frame DO CATG values must be unique to each recipe. There are some rules on the values here.

    20. 18 April 2007 Second Generation VLT Instruments 20 Output File Names Recipe creates and names output files. File names must be hard coded! Several files of the same PRO CATG must be indexed in a predictable way. esorex can change the file name

    21. 18 April 2007 Second Generation VLT Instruments 21 Headers CPL cannot directly manipulate FITS headers These must be read into a cpl_propertylist structure.

More Related