1 / 13

dfos: tutorial

dfos: tutorial. www.eso.org/~qc/dfos/. Lesson 1: dfos - vision and architecture Directory structure Installation Tips and tricks. Lesson 2: Association. Lesson 3: Operations. The calibration memory. Three memory files under $DFO_MON_DIR:

mairi
Download Presentation

dfos: tutorial

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. dfos: tutorial www.eso.org/~qc/dfos/ • Lesson 1: • dfos - vision and architecture • Directory structure • Installation • Tips and tricks • Lesson 2: • Association • Lesson 3: • Operations

  2. The calibration memory Three memory files under $DFO_MON_DIR: • raw_list: all raw files per <date>; created from scratch by createAB • mcalib_list: all *real* mcalib files (final names, final place) • ‘calib memory’ part I • day range: • starting with <date> • N days backwards (N_MCAL_LIST: ‘depth’ of calib memory) • created by listMcalib (usually as part of moveProducts) • vcalib_list: all *virtual* mcalib files (predicted names) • ‘calib memory’ part II • depth: N(VCAL_LIST); dynamically shifted by createAB/updateAB all filled by fitsreport -c fitsreport_assoclists.cfg

  3. The calibration memory Equivalents: • your $DFO_CAL_DIR branch (content controlled by mcalib_list) <=> master calibration database • raw_list + mcalib_list <=> OST • vcalib_list: no equivalent! Maintenance: • Self-maintained (createAB/updateAB) • need management only if you jump in time • user-control possible through listMcalib

  4. Finding calibrations • Sequential creation: only after day 3 all vcalibs known • Duplicate creation: all vcalibs known to all days!

  5. Missing calibrations • Still missing mcalibs? • increase N_MCAL_LIST, run listMcalib -> scan older dates • Still missing? • well, wait or call Paranal ... • Headers: will tell you if missing calibs have been measured recently, but generally don’t solve the problem

  6. Daily operations: simple scheme 1. Pre-processing: daily, cronjob • download headers • create Report • … 7-10 days later: • Download fits files • check completeness • incomplete: wait • complete: createAB [dataclient -t h -d 2005-01-02] [createReport -d 2005-01-02] [dataclient -t r -d 2005-01-02] [checkdownloads -d 2005-01-02] [createAB -m CALIB -d 2005-01-02]

  7. Daily operations: simple scheme 2. Processing • createJob • execute jobs file • measureQuality • certifyProducts 3. Post-processing • moveProducts 4. Pre-processing, processing (SCI) 5. Post-processing SCIENCE / ALL • moveProducts -m SCIENCE • updateDP • finishNight [createJob -m CALIB -d 2005-01-02] [JOBS_NIGHT] [measureQ. -m CALIB -d 2005-01-02] [certifyP. -m CALIB -d 2005-01-02] [moveProd. -m CALIB -d 2005-01-02] [createAB-> certifyP. -m SCIENCE …] [moveP. -m SCIENCE -d 2005-01-02] [updateDP -d 2005-01-02] [finishNight -d 2005-01-02]

  8. Daily operations: simple scheme CALIB SCIENCE Simplified:

  9. Daily operations: simple scheme • Process one Paranal day: • low load … takes one DFO day • heavy load … takes three DFO days

  10. Daily operations: optimized scheme • Parts to be automated (cronjob: dfosCron): • header download • createReport • Data downloads: • dvdMonitor • fits download (bulk) • checkCompleteness • results in typically three complete plus one incomplete day (for continuous data acquisition)

  11. Daily operations: optimized scheme Twice a week: burst of new data; process three days in a bulk:

  12. Sequences • Commands come in sequences (command groups which can be executed sequentially, interrupted by decisions or actions) • [check “sequences” for details] • seq1: dataclient (headers) + createReport • seq2: dataclient (fits) • seq3: checkCompleteness • seq4: createAB + createJob • seq5: processAB + measureQuality • seq6: certifyProducts + moveProducts • seq7: updateDP + finishNight

  13. dfoMonitor • Follows sequences • knows about status of sequences • monitor plus active GUI • links to related information

More Related