1 / 58

Traffic Monitoring and Flow Analysis For IP Networks

Traffic Monitoring and Flow Analysis For IP Networks. Overview. Introduction SNMP Introduction to MRTG and RRD-Tool Introduction to Traffic flows. Introduction to SNMP. What is SNMP?. SNMP – Simple Network Management Protocol Industry standard, hundreds of tools exist to exploit it

karif
Download Presentation

Traffic Monitoring and Flow Analysis For IP Networks

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Traffic Monitoring and Flow AnalysisFor IP Networks

  2. Overview • Introduction SNMP • Introduction to MRTG and RRD-Tool • Introduction to Traffic flows

  3. Introduction to SNMP

  4. What is SNMP? SNMP – Simple Network Management Protocol Industry standard, hundreds of tools exist to exploit it Present on any decent network equipment Query – response based: GET / SET GET is mostly used for monitoring Tree hierarchy Query for ”Object Identifiers” (OIDs) Concept of MIBs (Management Information Base) Standard and vendor-specific (Enterprise)

  5. What is SNMP? UDP protocol, port 161 Different versions V1 (1988) – RFC1155, RFC1156, RFC1157 Original specification v2 – RFC1901 ... RFC1908 + RFC2578 Extends v1, new data types, better retrieval methods (GETBULK) Used is version v2c (without security model) v3 – RFC3411 ... RFC3418 (w/security) Typically we use SNMPv2 (v2c)

  6. What is SNMP? Terminology: Manager (the monitoring ”client”) Agent (running on the equipment/server)

  7. What is SNMP? Typical queries Bytes In/Out on an interface, errors CPU load Uptime Temperature or other vendor specific OIDs For hosts (servers or workstations) Disk space Installed software Running processes ... Windows and UNIX have SNMP agents

  8. How does it work? Basic commands GET (manager -> agent) Query for a value GET-NEXT (manager -> agent) Get next value (list of values for a table) GET-RESPONSE (agent -> manager) Response to GET/SET, or error SET (manager -> agent) Set a value, or perform action TRAP (agent -> manager) Spontaneous notification from equipment (line down, temperature above threshold, ...)

  9. The MIB Tree root iso(1) ccitt(0) joint-iso-ccitt(3) org(3) dod(6) 1.3.6.1 internet(1) directory(1) mgmt(2) experimental(3) private(4) mib-2(1) enterprises(1) host(25) system(1) snmp(11) cisco(9) interfaces(2) hrDevice hrStorage ip(4) hrSystem

  10. The MIB Tree root iso(1) ccitt(0) joint-iso-ccitt(3) org(3) ciscoMgmt(9) dod(6) ciscoEnvMonMIB(13) 1.3.6.1 ciscoEnvMonObjects(1) internet(1) ciscoEnvMonTemperatureStatusTable(3) directory(1) mgmt(2) experimental(3) private(4) ciscoEnvMonTemperatureStatusEntry(1) mib-2(1) enterprises(1) ciscoEnvMonTemperatureStatusValue(3) cisco(9) system(1) snmp(11) interfaces(2) ip(4) ...

  11. The Internet MIB directory(1) OSI directory mgmt(2) RFC standard objects experimental(3) Internet experiments private(4) Vendor-specific security(5) Security snmpV2(6) SNMP internal

  12. OIDs and MIBs Navigate tree downwards OIDs separated by '.' 1.3.6.1.4.1.9. ... OID corresponds to a label .1.3.6.1.2.1.1.5 => sysName The complete path: .iso.org.dod.internet.mgmt.mib-2.system.sysName How do we convert from OIDs to Labels (and vice versa ?) Use of MIBs files!

  13. MIBs MIBs are files defining the objects that can be queried, including: Object name Object description Data type (integer, text, list) MIBS are structured text, using ASN.1 Standard MIBs include: MIB-II – (RFC1213) – a group of sub-MIBs HOST-RESOURCES-MIB (RFC2790)

  14. MIBs - 2 MIBs also make it possible to interpret a returned value from an agent For example, the status for a fan could be 1,2,3,4,5,6 – what does it mean ?

  15. MIBs - SAMPLE sysUpTime OBJECT-TYPE SYNTAX TimeTicks ACCESS read-only STATUS mandatory DESCRIPTION "The time (in hundredths of a second) since the network management portion of the system was last re-initialized." ::= { system 3 } sysUpTime OBJECT-TYPE This defines the object called sysUpTime. SYNTAXTimeTicksThis object is of the type TimeTicks. Object types are specified in the SMI we mentioned a moment ago. ACCESS read-onlyThis object can only be read via SNMP (i.e., get-request); it cannot be changed (i.e., set-request). STATUS mandatory This object must be implemented in any SNMP agent. DESCRIPTIONA description of the object ::= { system 3 } The sysUpTime object is the third branch off of the system object group tree.

  16. MIBs - SAMPLE CiscoEnvMonState ::= TEXTUAL-CONVENTION STATUS current DESCRIPTION "Represents the state of a device being monitored. Valid values are: normal(1): the environment is good, such as low temperature. warning(2): the environment is bad, such as temperature above normal operation range but not too high. critical(3): the environment is very bad, such as temperature much higher than normal operation limit. shutdown(4): the environment is the worst, the system should be shutdown immediately. notPresent(5): the environmental monitor is not present, such as temperature sensors do not exist. notFunctioning(6): the environmental monitor does not function properly, such as a temperature sensor generates a abnormal data like 1000 C. ”

  17. Querying SNMP agent Some typical commands for querying: snmpget snmpwalk snmpstatus Syntax:snmpXXX -c community -v1 host [oid]snmpXXX -c community -v2c host [oid]

  18. Querying SNMP agent Let's take an example snmpstatus -c s3cr3t -v1 169.223.142.1 snmpget -c s3cr3t -v1 169.223.142.10 .iso.org.dod.internet.mgmt.mib-2.interfaces.ifNumber.0 snmpwalk -c s3cr3t -v1 169.223.142.20 ifDescr

  19. Querying SNMP agent Community: A ”security” string (password) to define whether the querying manager will have RO (read only) or RW (read write) access This is the simplest form of authentication in SNMP OID A value, for example, .1.3.6.1.2.1.1.5.0, or it's name equivalent .iso.org.dod.internet.mgmt.mib-2.system.sysName.0 Let's ask for the system's name (using the OID above) Why the .0 ? What do you notice?

  20. References Essential SNMP (O’Reilly Books) Douglas Mauro, Kevin Schmi Basic SNMP at Ciscohttp://www.cisco.com/warp/public/535/3.html http://www.cisco.com/univercd/cc/td/doc/cisintwk/ito_doc/snmp.htm Wikipedia:http://en.wikipedia.org/wiki/Simple_Network_Management_Protocol IP Monitor MIB Browserhttp://support.ipmonitor.com/mibs_byoidtree.aspx Cisco MIB browser: http://tools.cisco.com/Support/SNMP/do/BrowseOID.do Open Source Java MIB Browserhttp://www.kill-9.org/mbrowsehttp://www.dwipal.com/mibbrowser.htm (Java) SNMP Link – collection of SNMP resourceshttp://www.snmplink.org/ Net-SNMP Open Source SNMP toolshttp://net-snmp.sourceforge.net/ Integration with Nagios http://www.cisl.ucar.edu/nets/tools/nagios/SNMP-traps.html

  21. Optional Materials SNMP Version 3

  22. SNMP and Security • SNMP versions 1 and 2c are insecure • SNMP version 3 created to fix this • Components • Dispatcher • Message processing subsystem • Security subsystem • Access control subsystem

  23. SNMP version 3 (SNMPv3) The most common module is based in user, or a “User-based Security Model” • Authenticity and integrity: Keys are used for users and messages have digital signatures generated with a hash function (MD5 or SHA) • Privacy: Messages can be encrypted with secret-key (private) algorithms (DES) • Temporary validity: Utilizes a synchronized clock with a 150 second window with sequence checking.

  24. Security Levels • noAuthPriv • No authentication, no privacy • authNoPriv • Authentication with no privacy • authPriv • Authentication with privacy

  25. Cisco SNMPv3 configuration snmp-server view vista-ro internet included snmp-server group ReadGroup v3 auth read vista-ro snmp-server user admin ReadGroup v3 auth md5 xk122r56 Or alternatively: snmp-server user admin ReadGroup v3 auth md5 xk122r56 priv des56 D4sd#rr56

  26. Net-SNMP SNMPv3 configuration # apt-get install snmp snmpd # net-snmp-config --create-snmpv3-user -a "xk122r56" admin /usr/sbin/snmpd # snmpwalk -v3 -u admin -l authNoPriv -a MD5 -A "xk122r56” 127.0.0.1

  27. Introduction to MRTG

  28. MRTG: Multi Router Traffic Grapher • MRTG is a tool to monitor the traffic load on network-links. MRTG generates HTML pages containing PNG images which provide an almost live visual representation of this traffic. You can find MRTG at http://oss.oetiker.ch/mrtg/. • MRTG has been the most common network traffic measurement tool for all Service Providers. • MRTG uses simple SNMP queries on a regular interval to generate graphs.

  29. MRTG • External readers for MRTG graphs can create other interpretation of data. • MRTG software can be used not only to measure network traffic on interfaces, but also build graphs of anything that has an equivalent SNMP MIB - like CPU load, Disk availability, temperature, etc... • Data sources can be anything that provides a counter or gauge value – not necessarily SNMP. • For example, graphing round trip times • MRTG can be extended to work with RRDTool

  30. MRTG: Issues • MRTG generates each graph (what if you have hundreds of graphs!) every 5 minutes, creating a lot of overhead. • It also has very few customizable graphing options. • Disk space is always an issue. • MRTG management itself can be tedious work.

  31. Using MRTG • Get the required packages • Compile and install the packages • Make cfg files for router interfaces with cfgmaker • Create html pages from the cfg files with indexmaker • Trigger MRTG periodically from cron or run it in daemon mode

  32. RRDTool • Round Robin Database for time series data storage • Command line based • From the author of MRTG • Made to be faster and more flexible • Includes CGI and Graphing tools, plus APIs • Solves the Historical Trends and Simple Interface problems as well as storage issues • Find RRDtool here: http://oss.oetiker.ch/rrdtool/

  33. Defining the Output (Archives) • RRA:AVERAGE:0.5:1:24 • RRA:AVERAGE:0.5:6:10 • RRA = Round Robin Archive • AVERAGE = consolidation function • 0.5 = up to 50% of consolidated points may be UNKNOWN • 1:24 = this RRA keeps each sample (average over one 5 minute primary sample), 24 times (which is 2 hours worth) • 6:10 = one RRA keeps an average over every six 5 minute primary samples (30 minutes), 10 times (which is 5 hours worth) • Clear as mud! • All depends on original step size which defaults to 5 minutes

  34. RRDTool Database Format

  35. So simple… • rrdtool create /var/nagios/rrd/host0_load.rrd -s 600 DS:1MIN-Load:GAUGE:1200:0:100 DS:5MIN-Load:GAUGE:1200:0:100 DS:15MIN-Load:GAUGE:1200:0:100 RRA:AVERAGE:0.5:1:50400 RRA:AVERAGE:0.5:60:43800 • rrdtool create /var/nagios/rrd/host0_disk_usage.rrd -s 600 DS:root:GAUGE:1200:0:U DS:home:GAUGE:1200:0:U DS:usr:GAUGE:1200:0:U DS:var:GAUGE:1200:0:U RRA:AVERAGE:0.5:1:50400 RRA:AVERAGE:0.5:60:43800 • rrdtool create /var/nagios/rrd/apricot-INTL_Ping.rrd -s 300 DS:ping:GAUGE:600:0:U RRA:AVERAGE:0.5:1:50400 RRA:AVERAGE:0.5:60:43800 • rrdtool create /var/nagios/rrd/host0_total.rrd -s 300 DS:IN:COUNTER:1200:0:U DS:OUT:COUNTER:600:0:U RRA:AVERAGE:0.5:1:50400 RRA:AVERAGE:0.5:60:43800

  36. What it looks like… http://noc.ws.nsrc.org/mrtg/device.html

  37. MRTG • In Ubuntu / Debian • $ sudo apt-get install mrtg • Configuration • /etc/mrtg/<device.mrtg> • Global directory : /var/www/mrtg/ • Run MRTG against the configuration file from cron

  38. cfgmaker • Uses snmpwalk and creates an mrtg configuration file • /usr/bin/cfgmaker \--output=/etc/mrtg/router.mrtg \--global 'workdir: /var/www/mrtg’ \ --global 'options[_]: growright,bits’ \ NetManage@10.10.0.254

  39. Sample • Part of /etc/mrtg/device.mrtg • ### Interface 1 >> Descr: 'FastEthernet0/0' | Name: 'Fa0/0' | Ip: '' | Eth: '' ### • Target[10.10.0.254_Fa0_0]: #Fa0/0:NetManage@10.10.0.254: • SetEnv[10.10.0.254_Fa0_0]: MRTG_INT_IP="" MRTG_INT_DESCR="FastEthernet0/0" • MaxBytes[10.10.0.254_Fa0_0]: 12500000 • Title[10.10.0.254_Fa0_0]: Traffic Analysis for Fa0/0 -- rtr.ws.nsrc.org.nsrc.org • PageTop[10.10.0.254_Fa0_0]: <h1>Traffic Analysis for Fa0/0 -- rtr.ws.nsrc.org.nsrc.org</h1>

  40. Creating HTML with indexmaker • Execute indexmaker like this: • /usr/bin/indexmaker \ • --output=/var/www/mrtg/device.html \ • /etc/mrtg/device.mrtg • If your mrtg configuration file is well commented, the html is nice and detailed.

  41. Introduction to Traffic Flow

  42. What Are Flows • Traffic flows: • A sequence of packets. • Exchanged between discrete sources + destinations. • Flow data is not necessary a 1:1 mapping. • Can also be a group of packets in the network.

  43. How Are Flows Accessible • Three well-known methods today: • NetFlow (and friends) • IPFIX • sFlow

  44. How Are Flows Accessible • NetFlow: • Originally developed by Cisco Systems. • Developed to capture IP traffic information. • Most common implementations are NetFlow v5 + v9. • NetFlow v9 adds support for MPLS and IPv6 data. • The following data can be gleaned from flows: • SNMP data. • Source + Destination IP addresses. • IP protocol. • TCP + UDP Source + Destination information. • IP ToS data.

  45. How Are Flows Accessible • NetFlow Friends: • Several vendors have developed NetFlow. • Are called by other names, but are the same. • Other implementations are: • AppFlow – Citrix. • Cflowd – Alcatel Lucent. • JFlow/cflowd – Juniper. • NetStream – 3Com. • NetStream – Huawei. • Rflow – Ericsson. • sFlow – Allied Telesis.

  46. How Are Flows Accessible • IPFIX: • Internet Protocol Flow Information Export • A standard-based flow capture protocol. • Is based on Cisco’s NetFlow v9. • Developed for any system that requires flow captures. • IPFIX uses SCTP as the transport protocol. • But also supports TCP and UDP.

  47. How Are Flows Accessible • sFlow: • Captures a sample of all “packets” on the network. • Naming of the technology is a misnomer. • sFlow does not capture flows. It captures packets. • Used mainly on switches for Layer 2 information.

  48. How Are Flows Accessible • NetFlow/IPFIX Differences with sFlow: • With NetFlow, flow cache is built on the router. • With sFlow, packet headers exported immediately. • With NetFlow, devices are complex, expensive. • With sFlow, devices are simpler, cheaper. • NetFlow provides great accuracy, even when sampled. • sFlow samples packets, which is less accurate. • NetFlow permits data set flexibility. • sFlow just samples packets.

  49. How Can Flows Be Analyzed • Three stages in flow analysis: • Collect the flows. • Export the flows. • Analyze the flows. • In some cases: • Flow export is not really necessary. • Collection + analysis can be done locally. • Not scalable, but can be a “quick & dirty” fix.

  50. How Can Flows Be Analyzed • Collect the flows: • Device must support a NetFlow implementation. • NetFlow configured on participating interfaces. • Flow cache is built as data crosses the device.

More Related