Steve Crawford crawford@saao.ac.za Collaborators: Martin Still, the SALT astronomers, and David Buckley

1. PySALT: SALT’s PyRAF Pipeline Steve Crawford crawford@saao.ac.za Collaborators: Martin Still, the SALT astronomers, and David Buckley

2. Outline • Introduction to PySALT: the SALT data reduction and analysis software package • PySALT and the SALT Pipeline • Reducing SALT data • Slotmode analysis tools • The Future of PySALT development SALT Goettingen Board Meeting

3. PySALT: Optimizing Performance Purpose of the SALT Science Software Development: Science quality data reductions of SALT observations Data analysis tools for the unique modes of SALT Algorythms to handle the data archiving and timely distribution to principle investigators Advantages: Maximize scientific returns for the telescope. Minimize redundant work 3. Establish a framework for archiving and distribution 4. Provide for a method for fast response science How to interact with the virtual observatory Provide software for unique modes of SALT SALT Goettingen Board Meeting

4. Maximizing Scientific Return Steps to Publishing a Paper: Observe an Object ??? Publish!!! Minimizing the steps in ??? will greatly increase the scientific productivity of SALT  Minimize time spent reducing data (we’re all doing the same things) Spend more time on data analysis  = Different Publications More Publications Plus having standard reductions open up new observations to Scientist that may never have considered attempting. SALT Goettingen Board Meeting

5. What are the needs of the SALT pipeline? • Flexibility to handle a large range of data including unique modes such as high speed imaging and Fabry-Perot Spectroscopy (not to mention high-speed Fabry-Perot polarimatry) • Ability to process that data in real time • To deliver high quality science data to the PI’s • An automated system included the ability to analyze data quality and reduction status SALT Goettingen Board Meeting

6. Development Langauge: Python/PyRAF • PyRAF provides a similar environment and access to IRAF • Python is a well developed, well-supported programming language with run-time compilation and scripting ability. • Free, open source software • Large and growing library of numerical and image processing software. • Ability to wrap and interact with a large number of other code bases SALT Goettingen Board Meeting

7. PySALT PySALT is the main software package for SALT data reduction and analysis. Along with library containing tasks for error handling, I/O, interaction with the database, and other shared tasks, the PySALT package currently contains three main software packages: First Public Software Release: 28 April 2008 PIPETOOLS Tasks to automate the data reduction And data handling SALTRED Basic CCD data Reductions SLOTTOOLS Slotmode photometry And analysis tools SALT Goettingen Board Meeting

8. PIPETOOLS Prepare Data The Pipetools package is designed to handle the daily reduction, archiving, and distribution of the data. Each day at 10:30 am, the pipeline automatically begins processing the data from the previous night. An SA will verify the data quality before beginning a separate task that will archive the data, place the data on an ftp server, and notifying the PI that their data is ready for retrieval. Clean Images Create documentation Archive FTP Email SALT Goettingen Board Meeting

9. SALTRED Steps in SALT CLEAN Prepare Data General preparation of the data Before running other corrections. Correct for the different gain levels From each of the amplifiers According to the database file Gain Correction SALTslot.py Apply the cross-talk correction Xtalk correction Overscan and trim the data. Can Also apply the master bias but Currently does not. Bias correction CR Clean Cosmic Ray clean the data Mosaic the multi-extension data Into one file. Mosaic images SALT Goettingen Board Meeting

10. Performance • Fast reduction of SALT slot mode: • Cleans the data at a rate of >50, 000 images per hour • Reduce data as fast as the data are taken • Due to the purchase of a dedicated pipeline computer with optimized I/O performance, a typical night’s worth of data can be fully reduced in under 20 minutes (~2.5 GB) with a heavy night’s worth of data in 40 minutes (6 GB) • The Bottleneck: Internet Bandwidth! SALT Goettingen Board Meeting

11. SLOTTOOLS SLOTPHOT Main task that measures photometry for slotmode observations. The taks will: Subtract the global sky background Follow the drift of one of the two sources Measure the photometry for two sources via four different methods Slotview Task that provides an interactive environment for viewing the lightcurve and data produced by Slotphot. In addition to repeating the photometry the user can also save the data, change the targets, delete sources, and assess the quality of the light curve. SALT Goettingen Board Meeting

12. SALT Goettingen Board Meeting

13. Flaring on Pre-Main Sequence Stars H. Worters (UK/SAAO) SALT Goettingen Board Meeting

14. Documentation Documentation will be supplied in five parts: • IRAF-like help files for each task • Development documents for each task • Reduction recipes for each mode • Planning documents for each package. • Manuals for each package. Slottools manual distributed with the most recent software release. SALT Goettingen Board Meeting

15. The Future of PySALT Basic Reductions Task for doing the basic reductions which are common for most modes. Still need tools for flat fielding and fringe correction. Extended Reductions: Tasks for reducing each of the individual modes. It will be important to produce reduced data for long-slit, multi-slit, slot mode, drift-scanning, and many more Analysis Tools: It will be important to develop tools for analyzing the wide range of modes from SALT data. For example: saltslot.py SALT Goettingen Board Meeting

16. Development Goals for PySALT A primary goal of the pipeline is to supply science-quality reduced data to the principle investigator. At the current time, our planned development include the following modes: Reduced images with astrometric solutions ready for relative photometry Fullframe Slotmode Fullframe Slotmode Reduced images ready for relative photometry Wavelength calibrated spectra ready for further analysis Long slit Long slit MOS MOS Wavelength and polarized calibrated spectra ready for further analysis Polarimetry Slot Spectra Slot Spectra Polarimetry Wavelength calibrated data cube ready for further analysis Fabry-Perot Fabry-Perot SALT Goettingen Board Meeting

17. SALT Community The SALT Science webpages are currently a static format containing the data reduction and analysis software Interactive community that allows for a clearinghouse for SALT Science Method for accessing Data through the web manager Interaction with the VO and online Analysis tools Fourm for discussing SALT Science SALT Goettingen Board Meeting

18. Ask not what the SALT pipeline can do for you… • Feedback--if you get data, give us feedback. Is the data good enough? Is there anything you would change? Is there something you need or would be great to add? • Adopt python/PyRAF as software environment • Share algorithms, software, • Contribute support Triton and Neptune from SALT 20080517--N. Loaring SALT Goettingen Board Meeting

19. Summary • The SALT PyRAF pipeline has been processing data successfully since Oct ‘07 • Software currently include basic CCD reductions plus analysis tools for slotmode and was released April 28. • Future plans include reductions and analysis tools to support most of the major modes of SALT • Next major released planned for Dec ‘08 and will include tasks for long slit reductions. SALT Goettingen Board Meeting

20. WET CampaignD. Buckley (SALT) SALT Goettingen Board Meeting

21. Resources • SAAO Pipeline Development: • http://www.salt.ac.za/science-support/pyrafiraf-package/developers-page/ • PyRAF • http://www.stsci.edu/resources/software_hardware/pyraf • Python • http://www.python.org/doc/ • http://www.diveintopython.org/ SALT Goettingen Board Meeting