180 likes | 364 Views
Streams Monitoring. 3D mini Workshop January 26th 200 7 Zbigniew Baranowski. What do we use Streams Monitoring for?. Replication topology State of streams connections Process error notifications Monitoring streams performance (latency, throughput etc.) in each phase of replication
E N D
Streams Monitoring 3D mini Workshop January 26th 2007 Zbigniew Baranowski
What do we use Streams Monitoring for? • Replication topology • State of streams connections • Process error notifications • Monitoring streams performance (latency, throughput etc.) in each phase of replication • Monitoring resources that have impact on streams performance(Stream Pool, Redo generation) Streams Monitoring - 2/18
Streams Performance? • Database availability • State of each process (Enabled,Disabled,...) error checking • Queues state (amount of messages, spilling) • Number of LCRs in each phase (captured, propagated and applied) • Replication latency • Number of bytes propagated • Redo generated in DBs • Streams pool size usage Streams Monitoring - 3/18
Monitor architecture • „Strmmon” Daemon • written in Python 2.3.4 • running on central repository server together with 3D OMS • collects streams and instances information • generates and stores logs of these activities in DB • reports about errors and warnings • End-user web application • written in PHP5(using JpGraph and GraphViz) • distributes data to end-user • Performance graphs • Connections status • diagnostic • Etc • Current production node • Intel(R) Xeon(TM) CPU 2.40GHz • 1024 MB of RAM Streams Monitoring - 4/18
Monitor architecture CERN CNAF 3D CERN IN2P3 RAL Server running script End User (Web Browser) PHP
Web User Interface http://oms3d.cern.ch:4889/streams/main.php Username:***** Passwd:***** Features: • Monitor summary • Experiments connection maps • Database list • Active Streams • Graphs • Process and queues connections Streams Monitoring - 6/18
Monitor View Streams Monitoring - 7/18
Connection view Streams Monitoring - 8/18
Database list Streams Monitoring - 9/18
Database detailed view Streams Monitoring - 10/18
Active streams connection dashbord view Streams Monitoring - 11/18
Datailed stream view Streams Monitoring - 12/18
Graph generator Streams Monitoring - 13/18
Graph examples Replication of single transaction Streams Monitoring - 14/18
ExamplesRedo Generated Streams Monitoring - 15/18
Graphs ExamplesLHCb test(propagation) Streams Monitoring - 16/18
Burst of transactions Streams Monitoring - 17/18
What next? • Migration to faster machine • Intel(R) Xeon(TM) CPU 3.00GHz • 4GB of RAM • Improving web script performance • Collecting input from user to improve front page (map with connection and states) and make other clear for users • Reuse of daemon script for other monitoring of T1s for crosschecking with OMS • More data? CPU load, disk I/O etc. • 3D headline with logo is missing Streams Monitoring - 18/18