1 / 16

Kay Graf KM3NeT Computing and Software WG Meeting 2014/ 09/11

Overview. Kay Graf KM3NeT Computing and Software WG Meeting 2014/ 09/11. Topics for today. activity reports DB status (CB) QA/QC forms (GA) Jpp ( MdJ ) aanet (AH) Restless Oyster & Introducing Rainbow Alga ( TG) topics w/out reports: software strategy and its implementation

addison-skinner
Download Presentation

Kay Graf KM3NeT Computing and Software WG Meeting 2014/ 09/11

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Overview Kay Graf KM3NeT Computing and SoftwareWG Meeting2014/09/11

  2. Topics for today • activity reports • DB status (CB) • QA/QC forms (GA) • Jpp (MdJ) • aanet (AH) • Restless Oyster & Introducing Rainbow Alga (TG) • topics w/out reports: • software strategy and its implementation • discussion, feedback • open action points • software overview • progress towards GRID • offline data format • Jpp status • data processing for PPM-DU – asked for a meeting Comp-Soft-WG Meeting - 2014/09/11 - K. Graf

  3. Software Strategy • workshop on 31stof July at Erlangen – indico page: http://indico.cern.ch/event/332843/ • attendees: working group coordinators of the physics and software group and the main developers of the main KM3NeT software packages • minutes: http://indico.cern.ch/event/332843/material/minutes/0.rtf • aim: define a common ground, from where a dependable, fast data processing chain can be set up and the necessary, missing software can be developed. Comp-Soft-WG Meeting - 2014/09/11 - K. Graf

  4. Software Strategy – Proposal (to be ratified by the PSC) decisions of the meeting are the following — and should act as strong proposal for the further line of software development: • Accept and implement the dataformat-driven strategy (see KM3NeT_SOFT_WD_2014_003) for the KM3NeT offline software (both simulation and experiment data processing): KM3NeT will develop a data processing chain with well-defined interfaces (mainly i/o and configuration data formats) at each processing step. One official software package per step will be maintained (with alternatives for crosschecks), which can read/write the official data formats directly. Comp-Soft-WG Meeting - 2014/09/11 - K. Graf

  5. Software Strategy – Proposal (to be ratified by the PSC) • Accept the following data formats as working solution — documented also in the wiki: • global detector description format as proposed by AH • evtdata format for simulation (based on ANTARES-SOFT-1998-007and ANTARES-SOFT-1999-003) • raw- and online data format as defined by the DAQ group (see wiki page) • offline data format as proposed by AH (see this document) global_det_idndoms \n dom_idline_idfloor_idnpmts \n pmt_id_global x y z dx dydz t0 \n pmt_id_global x y z dx dydz t0 \n ... pmt_id_global x y z dx dydz t0 \n repeat for each dom Comp-Soft-WG Meeting - 2014/09/11 - K. Graf

  6. Software Strategy – Proposal (to be ratified by the PSC) • Definitions: • no software may change standard definitions (e.g. the standard coordinates that are to be defined (see action point))  • the time of KM3NeT is UTC for all software (including DB) • simulation data only contains pure photon/photoelectron information with possibly some merging (<1ns) - no smearing • particle ID from PDG is used in evt file format • no MC truth data is removed within the processing chain • Documentation of all software packages to include doxygen (annotated sources) and a wiki page (small tutorial, programme parameters) Comp-Soft-WG Meeting - 2014/09/11 - K. Graf

  7. Software Strategy – Discussion • asked for feedback on my mail from 4th of August – none received (other than some modifications of the detector description format). • Comments are highly welcome, we should discuss and prepare implementing these points in the next months - the final decision will be taken at the next steering committee meeting and on the next collaboration meeting. Comp-Soft-WG Meeting - 2014/09/11 - K. Graf

  8. Software Strategy – Open Action Points 1 • Present offline data format to collaboration via mail beginning of September, for final approval during Paris collaboration meeting -> AH, KG • ⇒ first a “running” version of aanethas to be finished • Revise usage of antcc classes in Jpp (use offline classes directly) -> MdJ, RB • ⇒ at a later stage • Reiterate compatibility between online and offline data format -> MdJ, RB, AH • ⇒ to be done (especially data types) • Define proposal for the KM3NeT standard coordinate system -> TE • ⇒ will be based on ANTARES, TE puts together ANTARES conventions currently • Assemble information for all software used in KM3NeT, create matrix (used in which processing step, responsible person, status) -> KG, AT • ⇒ see later • thereafter: decide on standard packages and formalise processing chain Comp-Soft-WG Meeting - 2014/09/11 - K. Graf

  9. Software Strategy – Open Action Points 2 • Create a set of standard events (cascades and muon bundles) at CC-Lyon (in iRODs /in2p3/km3net/mc/benchmark) (for benchmarking) -> AT • ⇒ no feedback • Change particle ID and detector coordinate change in km3 -> CJ • ⇒ will be implemented in the next two weeks • Implement detector description format in Jpp -> RB • ⇒ done Comp-Soft-WG Meeting - 2014/09/11 - K. Graf

  10. Software Overview • KM3NeT_SOFT_WD_2014_005 • one sheet: software description, one software matrix – see next slide • missing: • ORCA chain • anything in the HE chain? Comp-Soft-WG Meeting - 2014/09/11 - K. Graf

  11. Comp-Soft-WG Meeting - 2014/09/11 - K. Graf

  12. GRID • thanks to Christos and Vania, there has been progress towards data processing in the GRID • unfortunately our proposal to EGI has been rejected – however with a new (EGI-encouraged) possibility next year • EGI has started support nevertheless: • access to the RAL cvmfs repo (/cvmfs/km3net.gridpp.ac.uk) ⇒ every cluster, data center or Grid Site that replicates that repo can use it • Christos has install Jpp on that repository and distributed it as “VO_KM3NET_ORG_SW_DIR" at the GridSites supporting the KM3NeT.org VO (tested successfully JSirene on HellasGrid) • we are invited to the EGI / Géantevent Wednesday the 24th Sept. afternoon- a dedicated session on storage will happen there: • http://indico.egi.eu/indico/conferenceDisplay.py?confId=2160 • https://indico.egi.eu/indico/sessionDisplay.py?sessionId=23&confId=2160#20140924 • are you interested to go there? Comp-Soft-WG Meeting - 2014/09/11 - K. Graf

  13. Offline data format • Aart is developing aanet as KM3NeT offline data format (with manipulation classes), see thisdocument): • following "aa" classes from Antares; break compatibility with ANTARES • principles: KISS, YAGNI, less is more, zen of python, etc. • very simple class hierarchy: you can look at a header file and immediately know what's in and Evt. ­ goal is that anyone (new students, seniors) can do real work as easily as possible. • Use as much a possible general tools (c++, root, python). ­ learning new stuff is always hard, but at least these are good things to learn for all students. • code is written with 3 uses in mind • TTree::Draw • pyroot/Cintscripting • compiled c++ • doxygen at http://antares.in2p3.fr/users/heijboer/aanetdox Comp-Soft-WG Meeting - 2014/09/11 - K. Graf

  14. Jppstatus (MdJ, RB) • implemented .detx detector format and conversion tools between formats (.txt is now incompatible – use r879 if you need that format!) • implemented interfaces for adding alternative algorithms, e.g. for ToT-response simulation • added example and data directory, to be run “as is” • a selection will be used as test scripts during make • new release tagged: • see http://elog.km3net.de/Computing+and+Software/13 Comp-Soft-WG Meeting - 2014/09/11 - K. Graf

  15. PPM-DU Data Processing • all runs < run 448 (GPS-tagged) are available at iRODS:data/raw/PPM-DU/filter/2014 (note: no monthly subdir!) • processed in Lyon using JPP in /afs/in2p3.fr/throng/km3net/src/Jpp/Jpp_install_tag_r879 • converterv1Tov2 -d 1 -t GPS1 –u GPS2 –v GPS3 • JTriggerProcessor-d 1 -T 1 -@ "writeL1=1" -@ "trigger3DShower.numberOfHits=1" -a /afs/in2p3.fr/home/g/graf/km3net_ppm_du_timeoffsets_corrected.txt -c 3 • (note “1” instead of “true” is necessary for writeL1 even if it is a bool) • now move to nightly processing at CP • scripts are prepared but not tested yet Comp-Soft-WG Meeting - 2014/09/11 - K. Graf

  16. PPM-DU Data Processing • missing: • check .txt → .detx detector conversion is correct (results are identical): http://elog.km3net.de/Computing+and+Software/12 • determination of effect of inter-DOM offsets • define the final quasi-online processing parameters • change the retrieval of GPS times from runlog.txt to runlog.xml (database xml upload format) Comp-Soft-WG Meeting - 2014/09/11 - K. Graf

More Related