1 / 23

Calibration streams in the Event Filter.

Calibration streams in the Event Filter. Status report Mainz, Thursday 13 October 2005 Sander Klous – NIKHEF On behalf of the EF calibration team: Martine Bosman, Andrea Negri, Serge Sushkov and Sarah Wheeler. Definition and scope. Physics streams. Calibration streams. 40 MHz x 1.5 MByte.

lucus
Download Presentation

Calibration streams in the Event Filter.

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Calibration streams in the Event Filter. Status report Mainz, Thursday 13 October 2005 Sander Klous – NIKHEF On behalf of the EF calibration team: Martine Bosman, Andrea Negri, Serge Sushkov and Sarah Wheeler.

  2. Definition and scope. Physics streams Calibration streams 40 MHz x 1.5 MByte Level 1 Level 2 Level 1 Level 2 Other people 75 kHz x 1.5 MByte ? Event Filter Processing time: 1 Sec/Evt Number of nodes: 1500 Output: 200 Hz 320 MB/s Calibration issues in the EF. • At the moment only EF output. • Identify calibration types. • Size, rate, contents. • Requirements for the EF. • Data flow. • Processing issues. • Implementation scenarios. • Design and modifications. • Processing. • Memory management. • Networking/Timing. • Plan of work. 2 kHz x 1.5 MByte TDAQ workshop - Mainz

  3. Use cases(identification of calibration streams). Based on the Hawkings/Gianotti document. • All listed calibration types are identified at HLT level (after PESA). • Known calibration types in the HLT. • Various duplicates of physics streams: e.g. inclusive high pT electrons and muons (tracking), Z to di-lepton (energy), minimum bias (background). • Total: 35 MB/s (10% of physics data). • Liquid Argon Calorimeter. Pulse shape analysis, timing calibration and tuning of filter coefficients. • High pT electron sample • Electro Magnetic data only. ROI only. • Raw data, 5 consecutive samples (i.e. special event type). • Calorimeters and TRT. Hadronic response studies, comparison to test beam data. TRT: e/ separation. • High pT isolated hadrons. • All subdetectors. ROI only. • RAW data. TDAQ workshop - Mainz

  4. Use cases – continued…(identification of calibration streams). • MDT small chambers. Hourly realignment. • Small muon sample. • MDT information only. Overlap regions only. • Reprocessing of raw data. • Inner Detector subdetectors (Pixel, SCT and TRT). ROD monitoring (TRT only) and alignment. • Generic high pT events. • All subdetectors. ROI only. • Post-processing of track fit information on HLT level. • Other foreseen calibration types. • Liquid Argon Calorimeter might need Z  ee calibration at HLT level. • High statistics (1 kHz) ROI muon sample, containing MDT, CSC and RPC/TGC information. • Your favorite missing calibration stream… TDAQ workshop - Mainz

  5. Data flow characterization. SFI SFI? Subdetector Fragment 1 Lvl 1/2 info SFO SFO SFO SFO SFO SFO SFO SFO Subdetector Fragment N Lvl 1/2 info Special events Physics High pT Z di-lepton Min. bias Subdetector Fragment 1 ROI info 32 MB/s 1.6 MB/s 1.6 MB/s 320 MB/s Partial event Transport time: LVL2Cal 0.01 ms EF Node n Stripping/Collecting LVL2Cal Transport times/rates: LAr OverlapMu IsoHad GenPT LAr OverlapMu IsoHad GenPT 0.5 ms 0.05 ms 5 ms 0.5 ms 50 Hz 5 Hz 5 Hz 100 Hz Output 1 MB/s 2.5 MB/s 5 kB/s 2 MB/s 4 MB/s Transport times: 19 ms EF Node n EF processing: 1 second/event Full calibration events Sorting EF processing: 1 second/event EF Node n Transport time: 19 ms Partial calibration events 1 kHz! Additional Processing

  6. EF processing issues. SFO SFO SFO SFI Node n EFD SharedHeap Event Input PT #1 PTIO Result ExtPTs PTIO PT cal ExtPTs PT #2 PTIO Trash Output Output Output Calibration Stream Diagnostic Main output stream • Definitions: • Sorting for calibration: CalID. • Stripping/Collecting for calibration: CalCollect. • Detector calibration: CalDetect or Calibration. • Full event streams. • Very similar to physics streams. • Output to multiple SFOs/streams. • Sorting of events for calibration (CalID). • Partial event streams. • Processing similar to physics stream. • Stripping and collecting (CalCollect). • Handling of different output event size. • Sometimes requires additional processing (CalDetect/Calibration). • Special event streams. • Processing times completely different from physics stream. • Handling of different input and output event sizes. • Central issue: Robustness of the EFD TDAQ workshop - Mainz

  7. EF processing issues. SFO SFI • Definitions: • Sorting for calibration: CalID. • Stripping/Collecting for calibration: CalCollect. • Detector calibration: CalDetect or Calibration. • Full event streams. • Very similar to physics streams. • Output to multiple SFOs/streams. • Sorting of events for calibration (CalID). • Partial event streams. • Processing similar to physics stream. • Stripping and collecting (CalCollect). • Handling of different output event size. • Sometimes requires additional processing (CalDetect/Calibration). • Special event streams. • Processing times completely different from physics stream. • Handling of different input and output event sizes. • Central issue: Robustness of the EFD Node n EFD SharedHeap Event Input ExtPTs Output Dataflow application Main output stream TDAQ workshop - Mainz

  8. Calibration stream scenarios (1). SFO SFI SFI SFO SFO Node n EFD Event Result Input PT #1 PTIO PT #1 PTIO ExtPTs ExtPTs PESA Trash Trash Output Output Output Calibration Stream Main output stream Main output stream Additional functionality: • CalID algorithm. • Parallel output streams. Full calibration events. e.g. Z di-lepton Physics only events Node n EFD Event Result Input CalID PESA TDAQ workshop - Mainz

  9. Calibration stream scenarios (2). SFI SFO SFI? SFO SFO Node n EFD Node n Event EFD Event Input Input Result PT #1 PTIO Sorting PT #1 PTIO ExtPTs ExtPTs PTIO PT cal CalID PESA ExtPTs CalID Stripping Collecting Output Output Output Output EF output Calibration Stream Calibration Stream 1 Calibration Stream 2 Main output stream Special streams. e.g. LVL2Cal Additional functionality: • PT for calibration. • Information handling • Stripping/collecting. • Memory management. Partial calibration events. e.g. GenPT CalResult Calibration • Networking/Timing issues.

  10. Design and modifications (1). • CalID algorithm. • Lightweight algorithm. • Runs after PESA in physics PT – Stability issues. Athena configuration: multiple top algorithms. • Workload: Low – implementation only, thorough testing required. • Impact: High – required for (almost) all calibration streams. • Coordination: Sorting – New PT answers should be discussed. • Parallel output streams. • Slight modification of existing algorithm. • Runs in EFD, probably required for PESA as well. • Workload: Low – Modification of standard EFD task. • Impact: High – Required for most (calibration) streams. TDAQ workshop - Mainz

  11. Design and modifications (2). • PT for calibration. • Create stripping/collection algorithm. • Requires new eformat (see next slide). • Requires modifications in output task. • Allow multiple PTs to run consecutively (works already). • Transfer information between these PTs. • Should be possible with new eformat for EFResult / CalResult. • It might be interesting to transfer “intermediate results”. • Would avoid to • Run calibration algorithms in the same PT as PESA. • Reanalyze complete event in second PT. • Since EF is a dataflow application, this should be accomplished by writing an extra “Intermediate Result” object in the Shared Heap. • This requires ByteStream conversion for complex classes. • Workload: Medium – With exception of item 4 (no use cases yet). • Impact: High – Maybe with exception of item 4. TDAQ workshop - Mainz

  12. Memory management and information handling. SFO SFI Virtual event Event fragments EFResult PT #1 PTIO Stripping PTIO PT cal PESA CalID CalResult Stripping Collecting Output New eformat. Node n 1 0 1 0 0 1 EFD SharedHeap 1 0 1 0 0 1 1 Event EFResult CalResult 1 0 1 0 0 1 1 Input - x - 1 - 0 x - - - - - - - 1 ExtPTs 1 1 1 0 0 1 ExtPTs • Integrate with: • Virtual event. • SharedHeap. • Event handling in the EF. • Event modification in EF, i.e. stripping/collecting. TDAQ workshop - Mainz

  13. Open issues Many new developments on a very tight schedule. • Memory and performance. • Management: move from open/close backpressure mechanism (barrier) to analog (Nano sleeps). • Timing: revise SFI – EFD – SFO protocol. • Coordination… • Investigate common/similar design issues with monitoring • 128 bit header word. First discussion yesterday. • 32 bits to register appropriate output streams. • Investigate usage of this header. Streaming only, since EFResult fragment contains much more detail and is “just around the corner”. • PT answer to EFD. Composite structure. • EFD – SFO sorting, i.e. how is an output stream defined? • Load balancing between calibration and physics. • Distribution of calibration constants to EF software. • Communication between Athena algorithms and configuration and calibration databases. • Could this be done via e.g. the information service? TDAQ workshop - Mainz

  14. Plan of work. • Short time scale. • Run calibration algorithm in PT. • Implement parallel output streams. • Medium time scale. • Implement CalID algorithm and sorting. • Eliminate dead time in input and output tasks. • Medium – Long timescale. • Change memory management. • Implement new eformat. • Implement stripping/collecting. • Long timescale. • Transfer of intermediate results between first and second PT TDAQ workshop - Mainz

  15. Conclusions. • Our understanding of calibration streams in the Event Filter improved a lot. • We think we have a realistic overview of the workload involved in modifications of the Event Filter. • Implementation has started and this will lead to even better understanding of the topic (and of the work involved). • There are still some (many) open issues. Coordination is important, especially because of the tight schedule. • More information: https://uimon.cern.ch/twiki/bin/view/Atlas/EventFilterCalibration TDAQ workshop - Mainz

  16. Conclusions. • Our understanding of calibration streams in the Event Filter improved a lot. • We think we have a realistic overview of the workload involved in modifications of the Event Filter. • Implementation has started and this will lead to even better understanding of the topic (and of the work involved). • There are still some (many) open issues. Coordination is important, especially because of the tight schedule. • More information: https://uimon.cern.ch/twiki/bin/view/Atlas/EventFilterCalibration TDAQ workshop - Mainz

  17. Appendix TDAQ workshop - Mainz

  18. Appears to fit with solutions shown in LVL2Mu presentations. Ultralight project (Manuela Cirilli). LVL2Mu calibration stream (Speranza Falciano). Etc. (Enrico Pasqualucci, Alessandro de Salvo). Additional functionality: HLT output A distributed trigger for calibration? SFI? PT #1 PTIO Event Distributor HLT output Calibration Stream Moore’s law for networking Gary Stix, Scientific American, January 2001 Node n EFD Input Event ExtPTs Output The Event Filter is CPU dominated. You would like it to be bandwidth dominated… TDAQ workshop - Mainz

  19. ByteStream conversion • Write converters? • No, a lot of work. • Robustness issues. • Generic ByteStream conversion? • No support for complicated classes (e.g. multiple inheritance, polymorphism). • Something else? • Not on the priority list. • Under discussion… TDAQ workshop - Mainz

  20. Memory management and networking/timing 19 milliseconds transport time 0.25 ms dead time SFI SFO Barrier PT #1 PTIO PTIO PT cal PESA CalID 1+ second processing time Stripping Collecting 19 milliseconds transport time 0.25 ms dead time Node n EFD SharedHeap Virtual event Event Event Event EFResult EFResult EFResult CalResult Intermediate Results Input ExtPTs ExtPTs Output TDAQ workshop - Mainz

  21. Memory management and networking/timing SFI SFO PT #1 PTIO PTIO PT cal PESA CalID Stripping Collecting 0.01 ms transport time 25 ms dead time • Eliminate dead time. • Redesign of SFI – EFD – SFO communication protocol. • Coordination with networking group. • Barrier is insufficient. • Oscillations. • Other memory requests. Node n EFD SharedHeap Virtual event Event Event Event EFResult Barrier EFResult EFResult CalResult CalResult CalResult Intermediate Results Intermediate Results Input Intermediate Results ExtPTs ExtPTs 0 sec. processing time Output 0.01 ms transport time 25 ms dead time TDAQ workshop - Mainz

  22. Memory management and networking/timing SFI SFO PT #1 PTIO PTIO PT cal PESA CalID Stripping Collecting 0.01 ms transport time 25 ms dead time • Eliminate dead time. • Barrier is insufficient. • Oscillations. • Other memory requests. Node n EFD SharedHeap Virtual event Event Event Event EFResult Barrier EFResult EFResult CalResult CalResult CalResult Intermediate Results Intermediate Results Input Intermediate Results ExtPTs ExtPTs 0 sec. processing time Output 0.01 ms transport time 25 ms dead time TDAQ workshop - Mainz

  23. Memory management and networking/timing SFO SFI PT #1 PTIO PTIO PT cal PESA CalID Stripping Collecting 0.01 ms transport time 25 ms dead time • Eliminate dead time. • Barrier is insufficient. • Oscillations. • Other memory requests. Node n EFD SharedHeap Virtual event Event Event Nano sleeps Event EFResult EFResult EFResult CalResult CalResult CalResult • Solution: Nano sleeps, • However… • multiple control loops. • Risk of oscillations. • Additional complexity. • Workload: • High – Especially testing. • Impact: • ??? Intermediate Results Intermediate Results Input Intermediate Results ExtPTs Nano sleeps ExtPTs 0 sec. processing time Output 0.01 ms transport time 25 ms dead time TDAQ workshop - Mainz

More Related