Keeping Network Monitoring Current using Automated Nagios Configurations (WIP)

1. Keeping Network Monitoring Current using Automated Nagios Configurations (WIP) Greg Wickham APANJuly 2005

2. Is the network being monitored correctly? Greg Wickham APANJuly 2005

3. Contents • Background • Monitoring Overview / Requirements • Solution Architecture • Monitoring Verification • Conclusion

4. Contents • Background • Monitoring Overview / Requirements • Solution Architecture • Monitoring Verification • Conclusion

5. GrangeNet Architecture

6. GrangeNet Monitoring

7. GrangeNet Monitoring

8. GrangeNet Monitoring

9. GrangeNet Monitoring (ACT Edge)

10. GrangeNet Monitoring (ACT Edge) • Is that everything that can be monitored?

11. GrangeNet Monitoring (ACT Edge) • Is that everything that can be monitored? No!

12. GrangeNet Monitoring (ACT Edge) • Is that everything that can be monitored? No! • What else? • BGP address family peerings • Multicast / Unicast / IPv6 • Software versions • Hardware versions • Latency (of links) • Usage (of links) • …

13. Contents • Background • Monitoring Overview / Requirements • Solution Architecture • Monitoring Verification • Conclusion

14. Monitoring Solution • Solution Goals: • Verifying network is correctly monitored • Minimise replication of data • Simplistic integration with existing systems • Easy to maintain • Extensible • Flexible • Efficient

15. Monitoring Overview • Facts: • Networks change • Updating is tedious • Monitoring Difficult to Auditing • Answers Required: • Is the network performing optimally? • Has a change occurred? • What is the status of the network? • Is the monitoring accurate?

16. Contents • Background • Monitoring Overview / Requirements • Solution Architecture • Monitoring Verification • Conclusion

17. Solution Architecture MonitoringConfiguration • Configuration data stored as XML • Describes: • Devices to monitor • How to monitor • Nagios templates • Device Templates

18. Solution Architecture MonitoringConfiguration • Daemon reads configuration data • Verifies devices are monitored correctly • Generates Nagios Configurations • Performs device probes • Runs periodically MonitoringDaemon

19. Solution Architecture NagiosConfiguration MonitoringConfiguration MonitoringDaemon Nagios configurationautomatically generatedby Monitoring Daemon

20. Solution Architecture NagiosConfiguration MonitoringConfiguration NagiosDaemon MonitoringDaemon Nagios uses configuration supplied bymonitoring daemon;Nagios configured to use ‘passive’ checks

21. Solution Architecture NagiosConfiguration MonitoringConfiguration NagiosDaemon MonitoringDaemon Monitoring daemon queries all devicesusing SNMP; Check device telemetry againstknown configurations Network Devices

22. Solution Architecture NagiosConfiguration MonitoringConfiguration NagiosDaemon MonitoringDaemon Monitoring daemon sends Probe status direct to Nagios (Nagios running passive checks) Network Devices

23. Solution Architecture NagiosConfiguration MonitoringConfiguration NagiosDaemon MonitoringDaemon Nagios reports on networkhealth as usual but does noactive checking of its own eMail SMS Web Network Devices

24. Solution Architecture NagiosConfiguration MonitoringConfiguration NagiosDaemon MonitoringDaemon Report generated of devicemonitoring comparison eMail SMS Web Network Devices Report

25. Solution Architecture NagiosConfiguration MonitoringConfiguration NagiosDaemon MonitoringDaemon RRDtool eMail SMS Web Network Devices Report Collected data fed to optional sub-systems

26. Solution Architecture • Result • Only one process communicates to all devices Very Efficient Query time for 34 devices is < 10 seconds • As only one daemon communicates to the devices the load on each network device is minimised (collected data is distributed as necessary) • As Nagios does less work the monitoring server is less loaded (Nagios is heavy)

27. Contents • Background • Monitoring Overview / Requirements • Solution Architecture • Monitoring Verification • Conclusion

28. Monitoring Verification • Templates are used to define pre-requisite monitoring probes • Devices are attached to templates

29. Monitoring Verification Device Description <device> <alias>edge1.vic</alias> <address>202.0.98.68</address> … <module type="nagios"> <template>ibgp-mesh</template> <template>ebgp-peerings</template> <template>ospf</template> <template>system</template> … <probe type=“ibgp-mesh" description="AS18062 - edge1.nsw“ arg=“202.0.98.13” /> <probe type=“ebgp-peering" description="AS64670“ arg=“202.0.98.190” /> … </module> </device>

30. Monitoring Verification Template Description <template name=“ibgp-mesh"> <template>system-health</template> <probe name=“ibgp-mesh" inheirit="bgp-standard"> <attribute type="field">bgpPeerState</attribute> <attribute type="notify">gn-noc</attribute> <attribute type="level">level1-service</attribute> <match> <field name="bgpPeerRemoteAs" value="18062" /> <field name="bgpPeerState" value="up" /> </match> </probe> </template>

31. Monitoring Verification • From the: • Device template; and • Monitoring Template an accurate report can be generated of the status of monitoring. • All probe details are stored in XML so can be easily verified

32. Contents • Background • Monitoring Overview / Requirements • Solution Architecture • Monitoring Verification • Conclusion

33. Conclusion • Due to efficiencies in the monitoring daemon: • Nagios doesn’t load the server • Other applications can share the SNMP data • Doesn’t load the network devices • Device probing is very quick • Reduces complexity of Nagios configuration • Generate reports identifying inaccuracies in existing monitoring • Unified configuration data This is a Work in Progress

34. Status (Work in Progress) • Current functionality: • Separate applications: • Collecting data from devices; feed into Nagios • Generating Nagios configurations • To Do • Integrate applications • Complete Implementation Nagios templates • Documentation! • Software • Perl • net-snmp • Nagios