1 / 11

G.Kruk

Post Mortem data conversion. G.Kruk. Agenda. Conversion overview Why do we want to convert the data? Conversion process BinX library Final data format Current state Issues & further steps. Why do we want to convert the data?.

elke
Download Presentation

G.Kruk

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Post Mortem data conversion G.Kruk CERN AB Controls

  2. Agenda • Conversion overview • Why do we want to convert the data? • Conversion process • BinX library • Final data format • Current state • Issues & further steps

  3. Why do we want to convert the data? • Minimize processing on the client side --> send the data as it is • Data sent by different clients may have different structure • To browse and analyze PM events we want toaccess (read) data coming from different systems in the same way (using the same API) • Let’s convert the different data into one, standardized format

  4. Conversion process PM Server Notifies through “LastDump” property 2 1 PMD file (CMW data object) extracts 3 Conversion program Binary file with data sent by client converts 4 Final data format

  5. Conversion program It consists of two components: • PMManager • Is notified when new PM data arrives (monitors “LastDump” property) • Prepares directories structures • Calls appropriate PMConverter to convert the data • PMConverter (interface) • Performs the actual conversion from binary file to the final format • Uses BinX library • At the moment two implementations: FGCConverter, DefaultConverter (now used only for tests)

  6. BinX library • Library (C++) facilitating reading/writing and conversion of binary files • Uses XML descriptor to decode the binary • Takes care of the endian-ness • Using it we can potentially avoid writing a separate PMConverter for each system/class • Only the conversion program depends on it

  7. Final data format Information that need to be stored • For each signal: • name • type (int, float, string, …) • values (array) • units/precision? • description? • Additional meta-information?

  8. Final data format • After considering several options we decided to use SDDS as the final format • SDDS • Fulfils requirements • Is simple • API in Java, C/C++, Python, … • Supports both: ASCII and binary formats • We have positive experiences with it

  9. Current state • Converter program installed on the PM server machine (cs-ccr-pm1) • Supports conversion of FGC PM data - tested this week • Ready to perform tests with other systems

  10. Current state(directories structure) SYSTEMS PROCESSED PMD FILES FGC FGC 51 51 2005_11_24__14_44_45_240000000 2005_11_24__14_44_45_240000000 binx.xml RPMBB.UA83.RQSX3.L8.pmd RPMBB.UA83.RQSX3.L8.bin RPMBB.UA83.RQSX3.L8-50Hz.sdds RPMBB.UA83.RQSX3.L8-100Hz.sdds RPMBB.UA83.RQSX3.L8-1kHz.sdds RPMBB.UA83.RQSX3.L8-events.sdds

  11. Issues & further steps • Performance ~600-700 ms to convert data from one PC • There are places for improvements e.g. use binary SDDS rather than ASCII • No support for mapping from single bits into flag names e.g. 00000101  “FLAG_A FLAG_C” • For FGCs at the moment it is hardcoded • In the future move the mapping to BinX descriptor or to some other descriptor • Directory structure modification (refinement) • Before extending functionality of the converter program – tests should be performed with other systems (not only with FGCs) and with analysis team: • To validate current functionality, formats used, directory structure • To understand better required refinements

More Related