1 / 8

Job Options for Algorithms and Tools JOE v2.0

Job Options for Algorithms and Tools JOE v2.0. GANGA Software Workshop - 20-22 April 2004 - CERN Chun Lik Tan (Alvin) - Birmingham clat@hep.ph.bham.ac.uk. Agenda. Format of extracted information Persistency of job options. Format of extracted information. Background:

maura
Download Presentation

Job Options for Algorithms and Tools JOE v2.0

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Job Options forAlgorithms and ToolsJOE v2.0 • GANGA Software Workshop - 20-22 April 2004 - CERN • Chun Lik Tan (Alvin) - Birmingham • clat@hep.ph.bham.ac.uk

  2. Agenda • Format of extracted information • Persistency of job options

  3. Format of extracted information • Background: • Plex - This is the Python lexical analysis engine behind JOE’s job options parser.http://www.cosc.canterbury.ac.nz/~greg/python/Plex/ • JOE is capable of parsing both text-based and Python script-based job options files.

  4. Format of extracted information • Parser produces tokens which are recognised as algorithms, tools, services, properties, values, comments or included files. • The overall structure is essentially a multi-depth Python dictionary. The keys used are the objects of the tokens extracted.

  5. Format of extracted information • A job options file object (i.e. internal JOE representation) has three constituent member data structures: • the job option ‘tree’ dictionary, the included file objects list, and the comment objects list.

  6. Format of extracted information • Structure of job options ‘tree’ dictionary: • { Tool1:{Property1:{Property2:{’_value’:Value}}}, Tool2:{Property1:{’_value’:Value}}, Algorithm:{Property:{’_value’:Value}}, Service:{Property:{’_value’:Value}},} • The latter two member data structures are straight forward lists.

  7. Persistency of job options • JOE adopts a very pragmatic approach to job option persistency.... Save it in the same format it is capable of reading i.e. in text or Python. • Other formats e.g. Python pickle objects, XML representation can and will be supported when the need arises.

  8. Persistency of job options • To diff or not to diff... • Saving only changes imply having the original job options file at hand. • A particular job options file from different LHCb/ATLAS releases may/will differ slightly. Changes may be meaningless when applied to newer files. • Position sensitive changes?

More Related