Database requirements overview

1. Prototype Review Meeting Dale E. Gary Professor, Physics, Center for Solar-Terrestrial Research New Jersey Institute of Technology Database requirements overview

2. Disclaimer Two databases Monitor database—the stateframe XML and RDBMS Live vs. static stateframe data Prototype Review Meeting outline

3. We are just getting started on thinking about the databases. I know very little about databases and RDBMS, or how to interface to them, both to write them and to read them. I do have some expectation of what they should be able to do for the project, so that is where I will start. Prototype Review Meeting Disclaimer

4. There are two separate databases: 1) monitor database, 2) metadata database. The actual “visibility” data is actually a third “database,” but is a proprietary format: Miriad. The monitor database is expected to be a standard relational database, for storing a large amount of very similar data (the 1-second stateframe), parts of which will need to be retrieved some time in the past, for tracking down hardware or software issues. We want to be able to create general queries that can bring up past data in potentially new ways. The metadata database is supposed to play well with wider NASA/community solar databases, which both Jim and Dominic have experience with. I would like to hear some details about how it will work. I assume it is not a relational database, correct? Jim will be creating the metadata similar to what is done with RHESSI, so there must be tools that can be adapted. Prototype Review Meeting Two databases

5. The stateframe is supposed to be a complete description of the state of the system, updated once per second. As such, recording it into a database should provide a record of the system that can be perused to view performance over time, or compare one period with another. We have two issues with the stateframe. One is distributing the data, which is expected to be in the form of a binary record. The other is describing the data, which is envisioned to be done via XML. The plan is to be able to make a change to the stateframe (content of the binary data record) and a corresponding change to the XML template that describes it, and all software should be able to adapt to the change without making any change to the code. XML has no standard keywords, so it is extremely general. There exist XML “parsers” that can separate the various parts of the “tree” and allow one to programmatically extract the information about the described structure. We have built our own XML description, based on something that LabVIEW produces, that Python can decode. A similar decoder would be needed for dealing with the stateframe in other languages. Prototype Review Meeting Monitor database—the stateframe

6. Gil has said that using this scheme with SQLServer may be difficult. We need to know how SQLServer likes to get its data. It may be that we need to write a Python translator that converts the stateframe to something SQLServer likes, or perhaps there is another approach. Prototype Review Meeting Xml and rdbms

7. Each live datum in the stateframe is supposed to be “rich,” i.e. have associated with it various other information that does not change, such as a description, units, allowable max and min values, and so on. There will also be a valid flag associated with it. We have yet to create the mechanism for this static information, but one thought is to create a static XML description that parallels the stateframe XML file. The static XML would be used for human readability of items in the stateframe, primarily used by the display system. The RDBMS should be able to create a static database and associate it with the data. Idea: Perhaps the RDBMS should be the entity that serves the static information, and the display system could use the RDBMS as its source of that information. The RDBMS is NOT likely to be a good source of the stateframe live values, however, due to latency. Prototype Review Meeting Live vs. static stateframe data