1 / 6

CWG4 – The data model

CWG4 – The data model. The group proposes a time frame - based data model to: Formalize the access to data types produced by both detector FEE and data processing stages by prepending a generic M ultiple D ata H eader

Download Presentation

CWG4 – The data model

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. CWG4 – The data model • The group proposes a time frame - based data model to: • Formalize the access to data types produced by both detector FEE and data processing stages by prepending a generic Multiple Data Header • Provide strict memory management while minimizing the need for copying data for processing purposes (data service instead of “copy around”) • direct access to the data in memory without any additional processing • Use efficient data layouts allowing for fast navigation among data types and sources and usage of data from vectorizedalgorithms • Ongoing investigation and prototyping of efficient AOD formats • Flat vs. hierarchical object structures and the impact on processing speed and data compression • Investigation on I/O and compression and the output of synchronous reconstruction to be discussed with CWG7 (reconstruction) • Future work: integration simulation and benchmark • Realistic raw time frame simulation (CWG8) + time frame aggregation (CWG4) + FLP to EPN flow (CWG3) + concurrency model and platforms (CWG5) down to EPN reconstruction -> To be done in CWG13

  2. Multiple Data Header • FLP would add a common header type for all data blocks: raw and produced @ FLP (MDH) • Common part • Unique HW ID (FLP/EPN)+ version ID • Summary info for what follows (partly extracted from the single data header (SDH)) • Data type, number of blocks, block length, status • Used for navigation in the time frame & unique identification • Specific part • Relevant SDH info for fast navigation (error bits, fired trigger,see CDH now) • Transient block address table (for DDL data coming in sync) • Make data blocks look the same CWG4 - Data model

  3. The new generic data block: extension of the current schema • All data blocks produced by both FEE cards or arbitrary processing tasks on FLP (e.g. cluster finding) to be described as generic MDB blocks. A MDH is foreseen to point to several correlated “events” coming asynchronously on different links on the same FLP. Events will have a sub-frame structure (like today) • Processing of MDB blocks is transparent to the node type (FLP, EPN) • EPN’s will process MDB blocks but not required to produce MDB at their turn but rather the persistent event format.

  4. Data block types Type=Trigger HW ID = CTP Orbit/BX Size Nb. of blocks Status bits SDH +PAYLOAD Type=Heartbeat HW ID = CTP HB global counter HB local counters Orbit/BX Nb. Of blocks Requested actions: start run, pause, resume, end Type=FEE block HW ID=equipment Orbit/BX Size Nb of blocks Status bits SDH(CDH) +PAYLOAD Type=Clusters SW ID = clusterizer version Size Nb. of blocks Status bits SDH +PAYLOAD Heartbeat ~ time stamp + commands (i.e. start, pause, continue, stop) CWG4 - Data model

  5. Data management - FLP Offset in buffer BLi(t,t+dt) HBn HBn+1 Linki &buffer(link1) Linki+1 &buffer(link2) BLi+1(t,t+dt) HBn HBn+1 MDH Type HB ID #12345 Nblocks 10 Link #1 addr1 Link #2 addr2 … MDH Type RAW ID #12346 Nblocks 10 Link #1 addr3 Link #2 addr4 … Local processing Serialize to EPN Minimize searches on EPN for synchronized blocks For continuous readout it can be just the same for data reads correlated in time CWG4 - Data model

  6. The time frame data • The time frames will start and end with O2 “heartbeat” MDH (events) and embed all data blocks collected by a given FLP. The corresponding frames will have to be aggregated on a EPN node in a folder-like structure easy to browse by reconstruction algorithms. The fast (synchronous) persistent reconstruction format will have to achieve the required overall compression. • Note that the HBE (“heart beat event”)summary may be attached to the “end HBE” to allow for asynchronous dispatching of blocks before the frame is fully aggregated by the FLP

More Related