1 / 15

Data Persistency Solution for LHCb

Data Persistency Solution for LHCb. Motivation Data access Generic model Experience & Conclusions. Motivation. Physics software should be independent of the underlying data storage technology Data of different nature has to be accessed Event data, detector data, statistical data, …

beverage
Download Presentation

Data Persistency Solution for LHCb

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Data Persistency Solution for LHCb Motivation Data access Generic model Experience & Conclusions

  2. Motivation • Physics software should be independent of the underlying data storage technology • Data of different nature has to be accessed • Event data, detector data, statistical data, … • The access patterns differ • The data set size varies from several Mbytes to 1 Pbyte • Legacy data was written in ZEBRA format • It is unclear how these data will be stored • Locking into one technology may be a disadvantage GAUDI

  3. Strategy • Transient data representation is separated from the persistent data representation • Each representation can be optimized separately • Transient representation can be used to convert to any other representation • Minimize coupling between algorithms and the transient data • Algorithms see only transient data • Transient data items are not intelligent • Algorithms post and retrieve transient data from a “black-board”, the data store GAUDI

  4. How are Data Accessed? Transient Event Store Event Data Service Algorithm Transient Detector Store Algorithm Algorithm Detec. Data Service Transient Histogram Store Histogram Service GAUDI

  5. Manage ownership • Does cleanup Functionality of Data Stores • Manage objects of similar lifetime • Manage data objects like a librarian • Clients store objects • Other clients pick up objects when needed • Retrieve object collections GAUDI

  6. Structure of the Data Store • Tree - similar to file system • Identification by logical addresses: ”/Event/Mc/MCParticles” • Tree node • has data members (payload) • contains other node objects (directory structure) • Browse capability GAUDI

  7. /node1/A /node1/A/A1 /node1 Data Members /node1/A/A2 Node objects Data Object /node1/B C Symbolic links D /node2/C /node2 Data Members Data Object /node2/D Layout of the Data Object GAUDI

  8. Will be unsuccessful,requested object is not present Data Service Algorithm (3) Request load (1) retrieveObject(…) (2) Search in Store PersistencyService Request dispatcherObjy, SICB, ROOT,.. (5) Register Conversion Service (4) Request creation Data Store Converter Converter Converter Converter Store Dynamics: Loading Try to accessan object data GAUDI

  9. Criteria Persistency Service Output Stream (2) Create persistent representations (1) Collect references ConversionService Data Store Converter Converter Converter Store Dynamics: Storing GAUDI

  10. Objects & pointers Objects, object IDs, collections & DBs C++ pointer >> object ID Generic Persistent Model Persistent Transient GAUDI

  11. Database Technologies • Identify commonalties and differencesNecessary knowledge when reading/writing • RDBMS: More or less traditional • Objy is different: How to match ? GAUDI

  12. Storage type DB name database database Database Objects Cont.name Objects Objects Item ID Objects Objects Objects Container Objects Container Objects Container Generic Model: Assumptions Data Store ConversionSvc Class ID,Object path DiskStorage GAUDI

  13. Technology Converter (3) (2) (4) XID Link ID Link Info ... ... DB/Cont.name,... <number> (1) Lookup table Generic Model: References GAUDI

  14. Generic Model: Extended Object ID • Storage Type • Class ID OR • OID Objectivity • Storage Type dependent part • Link ID • Record ID ZEBRAROOTRDBMS GAUDI

  15. Experience & Conclusions • It is possible to write physics data without knowledge of the underlying store technology • Our approach can adopt any technology based on database files, collections and objects within collections • ZEBRA, ROOT, Objy and RDBMS • We are able to choose technologies according to needs • Overhead of transient-persistent separation looks manageable http://lhcb.cern.ch/computing/Components/html/GaudiMain.html GAUDI

More Related