Managed by UT-Battelle for the Department of Energy

1. Best Ever Archive Utility, Yet Creating a BEAST was easy. BEAUtY seems harder. kasemirk@ornl.gov April 2009 Managed by UT-Battelle for the Department of Energy

2. Previous Setup IOC “ASCII” Config. Channel Access Archive Engine config.xml Binary Data Files Data Server History ~2000: Started by Bob Dalesio ~2003: Index Tools, Data Server ~2007: CSS Client XML-RPC CSS-based OPI 2 Managed by UT-Battelle for the Department of Energy

3. Details Older Java “archive viewer” also available. Archive Engines running on “sampling” computer, data server on “serving” machine. Scripts to restart engines, copy data, update indices. Headaches with data maintenance: – Index time grows with data – Stuck when index files reach 2GB – SNS Users faced with ~80 sub-archives – No clue what needs fixing after network/power problems – No idea who contributes how many samples – No way to remove selected channels or time ranges – Improving on this means implementing an RDB 3 Managed by UT-Battelle for the Department of Energy

4. New Setup IOC Channel Access ArchiveEngine Samples Config. config.xml RDB Other tools for config & samples EngineConfig -Import (Oracle/MySQL) CSS-based OPI 4 Managed by UT-Battelle for the Department of Energy

5. CSS Data Browser Handles Both New URL Just one ‘RDB’ sub-archive Old and new data can be combined in one plot 5 Managed by UT-Battelle for the Department of Energy

6. Configure Sample Engines Configuration is in RDB – Directly use SQL – EngineConfigImport for legacy config files – View/Edit via web Hierarchical (as before) – Sampling engine (name, where to run, …) Groups – Channels No more duplicate channels! 6 Managed by UT-Battelle for the Department of Energy

7. Web Interface to Engine Config in RDB Tomcat/JSP/Servlets to view and edit 7 Managed by UT-Battelle for the Department of Energy

8. Web Config View: Channel Stats 8 Managed by UT-Battelle for the Department of Energy

9. Web Config View: Sample Stats ?? OK 9 Managed by UT-Battelle for the Department of Energy

10. Stats 34 sample engines, 79400 channels Host that runs sampling engine: – CPU load 45%, zero disk I/O wait, very responsive Oracle performance – 8000 samples/sec peak in write tests – Operationally maybe ¼ of that Better configuration would likely have fewer samples/sec 10 Managed by UT-Battelle for the Department of Energy

11. Report Example, SNS Beam Instr. Group May be SNS-specific, but shows usefulness of data in RDB 11 Managed by UT-Battelle for the Department of Energy

12. Summary Still testing, many Oracle setup issues – Ran since Oct 08, then.. – Oracle cluster setup since ~March – Updated partitioning in April Plan for about a year now: Make operational in maybe 2 months Thanks go to – Oracle: Jeff Patton – JSP: Katia Danilova – Beam Instr. Reports: Mariano Padilla, Jeff, Douglas Edwardson 12 Managed by UT-Battelle for the Department of Energy

13. Stuff 13 Managed by UT-Battelle for the Department of Energy

14. Hurdles Months: Get new Oracle server configured – Interface cards for storage array, fiber switches – Firewall holes for office access, backup, admin Changes in 10g – SELECT MIN(stamp), MAX(stamp) -> NULL, NULL – No “range” partitioning on Index-Organized-Tables Configuration issues – ORA-01654: unable to extend … – ORA-00257: archiver error What used to be impossible is now “easy”, but still expensive – DELETE FROM SAMPLES WHERE … 14 Managed by UT-Battelle for the Department of Energy

15. Other Ideas Jlab’s MyA – Operational, but Wrapper code around MySQL to create ‘cluster’ Handles less meta info (units, limits, …) Viewer is one-of C++/TCL/Tk Gabriele Carcassi mentioned RDDTool – Toolset for logging data with data-aging – Command-line RPN tools, web viewer – May be faster than general-purpose RDB but lacks advantage of gpp RDB 15 Managed by UT-Battelle for the Department of Energy

16. SNS Setup for EngineConfigImport Tool – Imports existing XML config files into RDB – On srv02 in …/imported_config *.xml files copied from archive1, includes some editing to remove duplicate channels Review *.log files – 'chmk/chmkhebt/PPS_HEBT:PLC_B:Chmk100AlarmContactA’ already in 'pps/pps_hebt’ Maybe edit “your” *.xml file, refer to import.sh to re- import 16 Managed by UT-Battelle for the Department of Energy

17. Getting Sources, Building binaries Get sources from DESY CVS repository – Maybe binaries “real soon” from ChannelArchiver web page Build binaries – Find somebody who understands “Eclipse RCP”, how to use org.csstudio.archive.rdb/EngineConfigImport.product org.csstudio.archive.engine/ArchiveEngine.product org.csstudio.sns.product/SNS_CSS.product (or DESY CSS.product) – Settings Compile-time defaults usually in “preferences.ini” and “plugin_customization.ini” files. Either change, or prepare file for -pluginCustomization command-line option 17 Managed by UT-Battelle for the Department of Energy

18. RDB Setup Currently supported: MySQL or Oracle – Main table difference: Oracle TIMESTAMP has nanosecs, MySQL has separate nanosecs col. – All tools auto-configure based on JDBC URL Table Schema – ERD, MySQL DBD in org.csstudio.archive.rdb/dbd – Setup for Oracle (table types, partitioning) not final 18 Managed by UT-Battelle for the Department of Energy

19. Run Archive Engine Command-line tool similar to previous engine – More settings (RDB, Logging, JCA/CAJ, …) Compiled-in defaults from “preferences.ini” and “plugin_customization.ini” Can specify overrides via -pluginCustomization my_plugin_customization.ini Mandatory command-line options for instance – engine_name to locate config in RDB – port for consistency, must match settings in RDB Built-in web server for status, restart, stop: http://host:port/: Empty page (used to be ‘main’) …./main: Main page …./reset: Reset statistics …./restart: Stop, re-read configuration, start again …./stop: Stop 19 Managed by UT-Battelle for the Department of Energy