1 / 19

Persistent references

ATLAS software meeting Database session. Persistent references. David Adams BNL May 28, 2002. Contents. Comment Definitions Use cases Structure ID’s Navigation. Comment. This is a first draft of a talk for the LGC workshop next week

rea
Download Presentation

Persistent references

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. ATLAS software meeting Database session Persistent references David Adams BNL May 28, 2002

  2. Contents • Comment • Definitions • Use cases • Structure • ID’s • Navigation Persistent refs ATLAS SW week – DB session

  3. Comment • This is a first draft of a talk for the LGC workshop next week • It merges the BNL HES model with the LCG persistency RTAG document Persistent refs ATLAS SW week – DB session

  4. Definitions • DO (data object) • Unit of persistent data • HOS (hybrid object store) • Persistent store for DO’s • EDO (event data object) • Unit of event data (i.e. data associated with a triggered beam crossing) • HES (hybrid event store) • Persistent store for EDO’s Persistent refs ATLAS SW week – DB session

  5. Definitions (cont) • OID – object identifier • Unique identifier for a DO • EDOID – EDO identifier • Unique identifier for an EDO • Special case of OID Persistent refs ATLAS SW week – DB session

  6. Definitions (cont) • Event ID • Unique identifier for a triggered beam crossing • E.g. a 32-bit int • Content ID • Identifier which labels the content of an EDO • In StoreGate this is the EDO transient type and a string key Persistent refs ATLAS SW week – DB session

  7. Use cases • HOS access • Insert object into store and receive OID • Present OID to store and obtain access to EDO • HES access • Insert object into store and receive OID • Present OID to store and obtain access to EDO • Restricted view • Define a restricted view of store • Query view for EDO (or any DO) Persistent refs ATLAS SW week – DB session

  8. Use cases (cont) • Replication • File replication • Object replication (between files) • From one storage technology to another • E.g. optimizing for space and then optimizing for fast access Persistent refs ATLAS SW week – DB session

  9. Use cases (cont) • References • One transient DO holds a reference to another • Both are written to the store • Both are read back into a transient store and the reference is restored • Location • In the previous only the first object is read in • There is a mechanism to locate the second object using the “reference” from the first Persistent refs ATLAS SW week – DB session

  10. Use cases (cont) • Replicated object references • The objects in the references use case are replicated • Either the original or replica of each object is read in • The reference is restore for any pair of objects Persistent refs ATLAS SW week – DB session

  11. Use cases (cont) • Location of replicated object • In the previous example either the original or replica of the first object is read in • The mechanism for locating the referenced object may find either the original or the replica of the second object Persistent refs ATLAS SW week – DB session

  12. Structure • Files • Objects reside in files • Files may be implemented in different PT’s (persistence technologies) • Files have a common HOS interface • HES • Layer over HES which takes advantage of the independence of data from different events Persistent refs ATLAS SW week – DB session

  13. Structure (cont) • HOS navigation • Simple means to use a DOID locate a DO in a collection of files • HES navigation • Same for EDOID and EDO • High level navigation • Other mechanisms for locating objects or files • Multiple implementations • Many use relational DB Persistent refs ATLAS SW week – DB session

  14. Structure (cont) Persistent refs ATLAS SW week – DB session

  15. Structure (cont) • HOS file interface • File(OID)  DO • HES file interface • File(EDOID)  EDO • File()  {Event ID} • File()  {Content ID} • File(Event ID)  {Content ID} • File(Event ID, Content ID)  EDO Persistent refs ATLAS SW week – DB session

  16. Identifiers • Options for generation • Standalone  128 bit ID • Central source  64+ bits • Hand out blocks of ID’s to reduce network access • To site, file system, compute node Persistent refs ATLAS SW week – DB session

  17. Identifiers (cont) • Content of HOS identifier • Simplest and smallest if ID has no content (other than uniqueness) • Including File ID provides aid to navigation and path to uniqueness • Should we also add a type ID? • Content of HES identifier • Could add event ID • Could add content ID • Type and string key in ATLAS Persistent refs ATLAS SW week – DB session

  18. Navigation • Low level HOS navigation • Search a collection of files until an OID is found • Might do some simple cataloging to optimize searches and cache results Persistent refs ATLAS SW week – DB session

  19. Navigation (cont) • High level navigation • Implemented above HES/HOS • Is able to restrict data to certain types, e.g. geometry, calibration or event data • Could provide catalogs based on history of data generation • Could identify small (and thus easily searchable) collections of relevant data • Datasets Persistent refs ATLAS SW week – DB session

More Related