1 / 23

PRIMA DRL Design Jeroen de Jong CDR, 30 Sep 2004

PRIMA DRL Design Jeroen de Jong CDR, 30 Sep 2004. Implications of the requirements Design Implementation issues. Requirement implications. Software needs to be both robust and flexible Maintainable over many years -> well designed and organized code

ronaldcourtney
Download Presentation

PRIMA DRL Design Jeroen de Jong CDR, 30 Sep 2004

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. PRIMA DRL DesignJeroen de JongCDR, 30 Sep 2004 • Implications of the requirements • Design • Implementation issues

  2. Requirement implications • Software needs to be both robust and flexible • Maintainable over many years -> well designed and organized code • Clear distinction between interactive and non-interactive modules • All pipeline algorithms must be available in standard ESO environment

  3. Pipeline and DAF • The ESO pipeline will do batch data reduction of nightly data. • The Data Analysis Facility (DAF) will be used for error analysis and calibration of the entire PRIMA dataset. • The pipeline will receive and parse calibration files from the DAF.

  4. DAF requirements • For proper scientific verification the DAF will have a database which contains all information about whatever has been done with the data. • The database may only be modified through well defined procedures. • A user interface must provide comprehensive access to all data. • An interactive experimental command lineenvironment will be provided.

  5. Modules • Common modules • Mathematical and Astronomical Library • Data Reduction Algorithms • FITS I/O Library • Configuration Files • Common Tools Library (e.g. logfile handling)

  6. Modules • Pipeline Modules • CPL Plugins to Data Reduction Algorithms • Pipeline Recipe Executor • Pipeline Configurator (to parse configuration files) • Calibration Matrix Interface

  7. Modules • Non-interactive DAF modules: • Database • Analysis Chain Manager:Sets up analysis chains in the database, manages branches to try different algorithms. • Sequencer: Executes algorithms and stores results in database. • Calibration Engine: Generates the Calibration Matrix.

  8. Modules • Interactive DAF modules: • Trend Analysis: Set of tools and algorithms for identifying and fitting trends and/or systematic errors. • WWW User Interface: Web application server which provides the standard user interfaces. • Visualisation Tool: Library for drawing standard scientific diagrams and displaying images. • Interactive Console: Provides an interactive command prompt to experiment with the data and

  9. Data Reduction Algorithms • Get all relevant data for an observation • Check quality of data • Convert laser metrology to observed wavelength • Compute scalar products of baseline and separation vector as function of time. • Correct scalar products for earth's motion • Compute weighted mean for each star/ref pair • Get calibrated baseline solution • Derive the projected separation • Correct for differential chromatic refraction

  10. DAF Database • Keeps relations between data products • XML files? FITS headers? • File for each analysis step • Relations defined with special tags

  11. DAF Trend Analysis • Most complex DAF module • Identifies and fits systematic errors in the data • First phase: Interactive plotting and fitting • Second phase: Automatic selection and optimization routines will be introduced. • Probably needs to disentangle several simultaneously occuring trends. • Uses inside DAF same infrastructure as DRA

  12. DAF Analysis Chain Manager • Scans and classifies FITS files • Fills the raw data tables in the database • Propagates each classified file throughthe analysis step files • Maintains the consistency of the database after: • (De)selection of data • Changes in the recipe

  13. DAF Sequencer • Executes the algorithms through plugin interface • Stores all results in the database • Recursively traverses through analysis chain • Only recomputes steps when data is newer, or the parameters or algorithm have changed.

  14. WWW User Interface • Why? Easier than standard GUI: • Many actions just forms and tables. • Only event driven interaction in Applets • Use Web Application Server: • Handles accounts and roles • Keeps state of user in memory • Webpages generated based on user ID and accompanying memory contents

  15. Web Server MainServlet VisualizationPlugin AbstractApplication ObsBlockEditor CatalogEditor AnalysisTools CalibrationTool DataSelector ServerContainer Sequencer AnalysisChainManager CalibrationEngine

  16. Interactive Console • Interactive scripting and command line environment • All algorithms and utilities from pipeline and DAF are available • Readonly access to database • Scripts can easily be plugged into DAF by extending an API

  17. Visualization Tool • Provides all 2D scientific diagrams • Displays images • Must be suitable for Applets • Easy interface from interactive console • Possible candidates: • Scientific Graphics Toolkit (SGT) from NOAA • VO tools???

  18. External Libraries • Mathematical and Astronomical: • Is there a sufficient complete ESO numerical library?? • Maybe GNU Scientific Library? • FFTW (Fastest Fourier Transform of the West) • Common Pipeline Library • FITS libraries: CPL FITS I/O with Java wrappers? • GANDALF as base for many DAF modules

  19. Programming Languages • ANSI C for Common and Pipeline Modules • All computations and ESO standard infrastructure • Java for most DAF infrastructure modules • Very robust object oriented language (derived from C++) • Standard libraries available for almost every software protocol or task • Jython for the Interactive Consoleand plugin scripts • Fully transparant Java interface (no extra coding required)

  20. Language Interfaces Jython Java JNI IDL ANSI C Python FORTRAN

  21. Comparison with other language combinations

  22. Conclusion • Modules are identified • Investigation of external tools is ongoing • No investigation yet on ESO standards • Reasonable DAF design based on present expertise • Some implementation issues addressed

More Related