1 / 9

REcent Developments

REcent Developments. GaudiPython / WP8. GaudiPython parallel. lcg developments. multiprocessing module added to LCG: Available in LCG Externals part of pytools package version 1.1 requirements line : use pytools v1.1 LCG_Interfaces

jens
Download Presentation

REcent Developments

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. REcent Developments • GaudiPython / WP8

  2. GaudiPython parallel

  3. lcg developments • multiprocessing module added to LCG: • Available in LCG Externals • part of pytools package version 1.1 • requirements line : use pytools v1.1 LCG_Interfaces • LCG Config may be off... replace 1.0_python2.5 with 1.1_python2.5 in PYTHONPATH

  4. SMAPS • smaps added to Multicore R&D twiki page • tar.gz with smaps.py script • and a simple number-crunching script • documentation pdf • also available in • /afs/cern.ch/sw/lcg/external/smaps/1.0 • with documentation

  5. parallel workshop • Results/decisions : • syntax change : • --ncpus=N for multicore parallel, instead of --parallel • --ncpus=-1 for all available cpus, instead of =0

  6. parallel workshop • Outcomes : • Modify OutputStream/PersistencySvc in Gaudi specially for Event IPC • IPCSvc.h/cpp added to PersistencySvc • modelled on existing OutputStream • objects collected and dumped to a member TBufferFile • IIPCSvc.h added to GaudiKernel • dumpObject() • loadBuffer() • syntax change : --ncpus=N, --ncpus=-1 • Combine 2 approaches : • Athena MP fork-all-workers / GaudiPython single output stream

  7. parallel SCHEMA

  8. GP parallel : cluster • Cluster-version of GaudiPython • Using pp-1.5.7 and pyssh modules (Not LCG) • possibly update from pyssh to pssh (maintained) • Open SSH session to a (given) list of hosts • Event Parallelism and message passing, instead of single Task with return value

  9. Gp parallel : cluster • Test mechanism implemented : • “GaudiServer” running on remote node • modified ppserver... • added Worker Classes/process, modifed processing of incoming data • environment prepared as part of SSH Session setup • GaudiServer forks one/many Worker Processes, but continues to listen on a single socket • Events can be streamed to socket as TBufferFiles, and then queued to Workers • Concept model... just working in scripts!

More Related