210 likes | 290 Views
Hybrid Event Store Integration with Athena/StoreGate. ATLAS Software Week Event Data Model and Detector Description Session. David Adams BNL March 5, 2002. Contents. Introduction Athena/HES components HES StoreView StoreSvc Proxy DataHandle StoreGateSvc DataLink AHesView
E N D
Hybrid Event Store Integration with Athena/StoreGate ATLAS Software Week Event Data Model and Detector Description Session David Adams BNL March 5, 2002
Contents • Introduction • Athena/HES components • HES • StoreView • StoreSvc • Proxy • DataHandle • StoreGateSvc • DataLink • AHesView • Tasks and schedule HES in Athena SW week – EDM session
Athena/HES components • Figure • The following page shows selected components of Athena and HES and their physical dependencies. • The Athena/StoreGate components (DataLink,…) shaded in blue are new or will have to be modified to integrate HES into Athena/StoreGate. • These are discussed in the following sections. HES in Athena SW week – EDM session
Athena/HES components (cont) HES in Athena SW week – EDM session
HES • Definition • Define a subsystem called HES which has no dependencies on StoreGate, Athena or Gaudi. • HesView is the top of this subsystem. • This subsystem will be discussed in the DB meeting on Thursday. • Links to this talk and that one can be found on the hybrid web page • http://www.usatlas.bnl.gov/~dladams/hybrid HES in Athena SW week – EDM session
StoreView • Purpose • StoreView is a new abstract interface which allows Athena to interact with different persistent stores in a generic manner. • Each persistent storage mechanism to be used in Athena will provide an implementation of this interface. • HES • ZEBRA • Objectivity? HES in Athena SW week – EDM session
StoreView (cont) • Interface • Select the next event ID. • Read an input event • Identify the included type-keys and create proxies in StoreGate • Locate the data (on demand from proxy) for any of these EDO’s • Write an output event • Take ownership of selected new EDO’s • (add EDO ID to proxy) • Write EDO’s to selected output streams HES in Athena SW week – EDM session
StoreView (cont) • Configuring a job • The choice of store view (or views) and their configuration is made at run time using Gaudi JobOptions • StoreView is a Gaudi service • No physical dependence on concrete implementation except that library must be provided at run time HES in Athena SW week – EDM session
StoreSvc • Purpose • StoreSvc provides the interface between the store views and the rest of Athena/StoreGate • StoreSvc holds one or more store views • Requests are fulfilled by forwarding to a selected view or looping over all views. • Each view is assigned an ID called store ID. HES in Athena SW week – EDM session
StoreSvc (cont) • Interface for reading • Assign the next event ID • Pass request to a designated store • Or assign in some other manner (e.g. user input) • Define input event • Loop over store views • Fetch data • Given a proxy for an EDO that has not be located, • identify the appropriate view and • ask the view to locate the EDO HES in Athena SW week – EDM session
StoreSvc (cont) • Interface for writing • Stream selection • Run selection algorithms and notify views which streams should be written. • EDO ownership • Give each view the opportunity to take ownership of EDO’s in StoreGate. • Writing data • Give each view the option to to write EDO’s from StoreGate (or from that view). HES in Athena SW week – EDM session
StoreSvc (cont) • Configuring a job • Configure in job options • StoreSvc is a Gaudi service • No physical dependency on Athena/StoreGate HES in Athena SW week – EDM session
Proxy • Purpose • The transient data store (StoreGateSvc) “holds” EDO’s by proxy. • This allows us to • Defer output conversion until all data is produced and selections are made • Defer input conversion until data is accessed • HES environment • It is sensible to also defer location on input • Some data may not be accessible HES in Athena SW week – EDM session
Proxy (cont) • Proxies will be modified for HES • From persistent store, there will be four states: • No data; proxy has type, key, store ID and EDO ID • Above plus IOpaqueAddress (EDO located) • Above plus transient address (EDO converted) • Marked inaccessible (EDO cannot be located) • For proxies from transient store, it will be possible to assign ownership (EDO ID). • Proxy uses StoreSvc to locate EDO when desired. HES in Athena SW week – EDM session
DataHandle • Current usage • A data handle is returned when a user asks StoreGate for an EDO. • Conversion may be deferred. • Changes (implied by change in proxy) • Location may also be deferred. • DataHandle may be invalid because the EDO cannot be located. HES in Athena SW week – EDM session
StoreGateSvc • Purpose and current implementation • StoreGateSvc is the transient data store. • Single instance holds data (by proxy) for current event. • Pileup adds more • Given a type and key, StoreGate returns a DataHandle for the corresponding EDO • Any type-key may not be duplicated in the store HES in Athena SW week – EDM session
StoreGateSvc (cont) • Changes • One instance will hold proxies for visible data • Directly accessed from algorithms • Other(s) will hold proxies for data which is accessible (by reference) but not visible. • Request for data from DataLink will include EDO ID in addition to type-key. • Only EDO with matching ID is returned. • Someone has to search the other transient stores if match is not found in the visible store • Visible store knows the reference stores? HES in Athena SW week – EDM session
DataLink • Purpose and current implementation • DataLink provides a persistent mechanism to reference objects in other EDO’s. • Persistent data includes: • EDO specifier and • index specifying the location of object in EDO. • At present, EDO specifier is the type and key and an event ID implied by context. • Any StoreGate EDO with matching type and key can serve as the reference EDO HES in Athena SW week – EDM session
DataLink (cont) • Changes: • In the new model, we explicitly recognize that persistent store may hold multiple EDO’s with the same event ID, type and key. • Persistent data for the DataLink will also include the ID of the referenced EDO. • May also need to add (transient) store ID to resolve EDO ID ambiguities between different store types. HES in Athena SW week – EDM session
AHesView • Implementation of StoreView for HES • Depends on both Athena/StoreGate and the HES core software. • Creates proxies • Puts proxies in StoreGate • Calls converters to write data • See talk at DB session for information on the HES core. HES in Athena SW week – EDM session
Tasks and schedule • Requirements • Sufficient functionality to use HES as the primary storage mechanism for DC1-2. • We would like a first implementation by June1. • Estimate the Athena/HES piece will require 7FTE-weeks for this implementation. • 2X contingency • We need to adjust priorities to meet this schedule. HES in Athena SW week – EDM session