Events Kernel Sequence Component ESQ One of three components of the E-kernel Subsystem

1. Events KernelSequence ComponentESQOne of three components of the E-kernel Subsystem June 2004

2. EK/ESQ Purpose • The SPICE E-kernel’s sequence component (ESQ) is concerned with mission event data • Which events? • Events relevant to current mission operations activities • Events relevant to providing scientists a complete understanding of science data returned from the spacecraft, both during the mission and years later • Events could be predicted or actual • The EK/ESQ subsystem provides standards and mechanisms for: • browsing event data interactively • conveniently accessing event data within application software • documenting event data • transmitting event data • archiving event data Events Kernel Sequence Component (ESQ)

3. EK/ESQ Implementation • The ESQ component uses a SPICE database mechanism, called DBK (“Database Kernel”), for storing “events” and related data • Suitable for recording sequences of commands, spacecraft configuration, state, or other highly structured, high-volume data • Examples: • Galileo, MGS, or Stardust sequences of events • Galileo observation plans • Clementine image catalog • DBK is a simple, stand-alone relational database system developed by NAIF • References to DBK in the remainder of this tutorial are applicable to the EK/ESQ Events Kernel Sequence Component (ESQ)

4. DBK Subsystem Features • DBK characteristics • All DBK software is provided in the SPICE Toolkit • No licenses or third-party software involved • Data are stored in binary files • Direct access (fast compared to sequential text file access) • Portable to all platforms supported by NAIF • Provides room for metadata (comments), as in other binary kernels • Utilizes operating-system-independent buffered access • Read and write access is provided by SPICE subroutines • SPICE Utilities/Apps that work with DBK files: • Use INSPEKT to browse DBKs interactively • Use COMMNT for comment area access Events Kernel Sequence Component (ESQ)

5. DBK Database Organization - 1 • Relational Model • A DBK database is a set of tables • Each table is made up of rows and columns • Tables are “rectangular”: same number of rows in each column • Column Attributes • Data type • Character (fixed or variable length) • Integer • Double Precision • Time (similar to “DATE” type in SQL) • Dimension (not supported in standard relational model) • Fixed or variable • Null values are allowed • Data values may be indexed Events Kernel Sequence Component (ESQ)

6. DBK Database Organization - 2 • Tables and files • A single table can be spread across multiple DBK files • A single DBK file can contain multiple tables • A single DBK file can contain pieces of multiple tables • SPICE DBK software can load (access) multiple DBK files at one time • If a table is spread across multiple loaded files, this is transparent to INSPEKT and to the high-level API routines Events Kernel Sequence Component (ESQ)

7. DBK Query Language - 1 • The DBK query language is used to select data from DBK tables • A query selects entries in specified columns belonging to rows that satisfy the query’s “constraints.” • DBKs may be queried via the DBK API or INSPEKT • The DBK query language is closely related to the widely used SQL language • Restrictions • Read access only • Use API calls to write or update a DBK • Subqueries are not supported • Arithmetic operations in queries are not supported • Extensions • Recognizes SPICE time formats • May retrieve data from array-valued columns Events Kernel Sequence Component (ESQ)

8. DBK Query Language - 2 • Query syntax SELECT <column list> FROM <table list> WHERE <constraint list> ORDER BY <order-by column list> • Example SELECT event_time, command_name, parameter_name, parameter_value FROM sequence WHERE command_name like “36*” and ((event_time between “1985 Jan 1” and “January 1, 2000”) or ( event_time > “Jan 1, 2002)) ORDER BY event_time DESCENDING, command_name Events Kernel Sequence Component (ESQ)

9. DBK API - 1 • DBK API Components • DBK query interface • Principal interface is a suite of routines supporting query-and-fetch operations. These are: • EKFIND --- issue a query • EKGC, EKGD, EKGI --- fetch character, double precision, and integer data, respectively. EKGD is also used to fetch data of type TIME. • EKNELT --- return the number of elements in a specified column entry • Needed because DBK columns may be array-valued . • EKPSEL --- parse the SELECT clause of a query • Returns, attributes of columns in SELECT clause. • Needed to handle arbitrary queries at run time: enables calling application to determine on the fly which fetch routines to call. • To make DBK data visible to the query interface, one “loads” DBK files of interest via FURNSH. DBK files can be removed from the system by calling UNLOAD. - continues- Events Kernel Sequence Component (ESQ)

10. DBK API - 2 • Low-level DBK reader interface • Supports reading specified column entries from DBK files • Routines EKRCEC, EKRCED and EKRCEI are used to read column entries from character, double precision, and integer columns, respectively. EKRCED reads data from columns of type TIME as well. • The column entry readers are file-oriented and record-oriented: the caller reads an entry from a record specified by its ordinal position in a specified DBK file. • This capability is an extension of the relational model. • Supports retrieval of schema descriptions from DBK files • Routines EKNSEG and EKSSUM are used to retrieve these data. • Routine EKOPR is used to “open” a DBK file for access by the low-level interface. EKCLS is used to close a DBK opened by EKOPR. - continues- Events Kernel Sequence Component (ESQ)

11. DBK API - 3 • DBK record-oriented writer interface • Supports creation of tables record-by-record. • Records may be added in random order. • Data may be taken from an input stream; no need to stage large amounts of data. • File creation is relatively slow. • To create a DBK using the record-oriented writers • Open a new DBK for write access using EKOPN. • Use EKBSEG to start a new “segment.” • DBKs consist of one or more collections of data called “segments.” Each segment contains data belonging to only one table. • Add new, empty records to a segment using EKINSR (“insert record”) or EKAPPR (“append record”). • Add column entries to records using EKACEC, EKACED, and EKACEI for character, double precision, and integer data, respectively. EKACED is also used to write TIME values to records. • Normally, each new record is filled in with data before a new record is started, but this is not required. • When all segments are done, complete the DBK by calling EKCLS. - continues- Events Kernel Sequence Component (ESQ)

12. DBK API - 4 • DBK “fast load” writer interface • Much faster ( roughly 100x) than the record-oriented interface. • Typically requires calling application to buffer large quantities of data in memory. • Builds one segment at a time. • Builds segments by writing the data for each column in one shot. • To use the “fast loader” routines • Prepare a new segment for the fast load routines by calling EKIFLD. • Call EKACLC, EKACLD, EKACLI to add character, double precision, or integer columns to the segment. Use EKACLD to add TIME columns. • Finish fast load by calling EKFFLD. • When all segments are done, complete the DBK by calling EKCLS. - continues- Events Kernel Sequence Component (ESQ)

13. DBK API - 5 • DBK update interface • Allows updating of individual column entries. • Use EKUCEC, EKUCED, or EKUCEI to update character, double precision, or integer column entries. EKUCED updates TIME entries as well. • Allows deletion of records. • Delete records using EKDELR. Events Kernel Sequence Component (ESQ)

14. DBK API - 6 • Example: query-and-fetch with SELECT clause and DBK schema known in advance. • Use EKFIND to issue a query. EKFIND returns the number of matching rows, an error flag, and an error message. • Loop over the matching rows, using EKGC, EKGD, and EKGI to fetch data from each column. All entries are assumed scalar and non-null. QUERY = ‘SELECT CHR_COL, INT_COL, DP_COL, TIME_COL FROM TABLE1’ CALL EKFIND ( QUERY, NROWS, ERROR, ERRMSG ) DO ROW = 1, NMROWS CALL EKGC ( 1, ROW, 1, CDATA(ROW), NULL, FOUND ) CALL EKGI ( 2, ROW, 1, IDATA(ROW), NULL, FOUND ) CALL EKGD ( 3, ROW, 1, DDATA(ROW), NULL, FOUND ) CALL EKGD ( 4, ROW, 1, TDATA(ROW), NULL, FOUND ) END DO Events Kernel Sequence Component (ESQ)

15. DBK Data Access Using Inspekt • INSPEKT is an interactive browsing program for DBK files • Command line interface • Command files • Symbols • Command editing • Extensive formatting options • Log files • Create customized reports • Uses DBK query language, plus various extensions • Read access only • There is currently no generic DBK writing program • (NAIF re-uses a modestly flexible ESQ writing program designed to work with JPL sequence products - PEF2EK) Events Kernel Sequence Component (ESQ)

16. DBK Limitations • The DBK subsystem does not support concurrent file access by multithreaded or multitasking applications. • Multiple users may read a DBK file simultaneously, but there is no built-in mechanism to ensure data integrity when a file is read and written asynchronously. • Various SQL features are not supported: • Commands providing “write access” to DBKs. • Subqueries • Expressions in SELECT clauses • Views Events Kernel Sequence Component (ESQ)