7. Monitoring & Analysis Tools

1. 7. Monitoring & Analysis Tools

2. 7. Monitoring & Analysis Tools • Tremendous variety of tools are available for monitoringmany aspects of networks • Simple commands usually included in operating systems • Free (open-source) applications • Commercial packages and systems • Category • Passive monitoring tools • Traffic Flow Analysis • NetFlow (C), cflowd (F), FlowScan (F), Sniffer Pro (C), argus (F), i-Flow (C) • Network Utilization • MRTG (F), RMON (C) • Visualization • RRD (F) • Active monitoring tools • Network Performance • ping (S), traceroute (S), Network Vantage (C), NetPerf (F), etc.

3. 7. Monitoring & Analysis Tools - NetFlow • Cisco IOS NetFlow Infrastructure Network Planning RMON Probe RMON Application Accounting/Billing NetFlowFlowCollector: • Data Collection • Data Filtering • Data Aggregation • Data Storage NetFlow Data Export: • Data Switching • Data Aggregation • Data Export Network Data Analyzer: • Data Presentation • Flow Control and Configuration Partner Applications Image From NetFlow PPT by Michael Lin, Cisco Systems

4. 7. Monitoring & Analysis Tools - NetFlow • NetFlow FlowCollector • provides fast and scalable data collection from multiple NetFlow Export-enabled devices • performs data volume reduction through selective filtering and aggregation • stores flow information in flat files on disk for post-processing by consumer applications. NetFlow Enabled Devices NetFlow FlowCollector NetFlow Consumer Applications Image From NetFlow PPT by Michael Lin, Cisco Systems

5. 7. Monitoring & Analysis Tools - NetFlow • Network Data Analyzer • Receives flow data from NetFlow FlowCollector(s) • Performs time-based analysis and data sorting • Configures FlowExports and FlowCollectors • Produces histograms, bar charts, and pie charts NetFlow FlowCollectors NetFlow FlowAnalyzer Image From NetFlow PPT by Michael Lin, Cisco Systems

6. 7. Monitoring & Analysis Tools - cflowd • Freely available NetFlow analysis tool from CAIDA • Functionality • Input • NetFlow export data from Cisco routers • Collect • collect: Information of flow obtained from NetFlow • store: uses arts++ file format (binary file format specification for storing network data) • Analyze • predetermined statistics text format, using ARTS utility (e.g., xartsprotos) • query and visualize using java front-end

7. 7. Monitoring & Analysis Tools – cflowd Source: http://www.caida..org

8. 7. Monitoring & Analysis Tools – FlowScan • Traffic Reporting & Visualization Tool • developed by Dave Plonka(U. Wisconsin) • analyzes and reports on flow data exported by routers • produces graph images which provide a continuous, near real-time view of the network traffic across a network's border • freely available • FlowScan binds together (1) a flow collection engine (a patched version of cflowd) (2) a high performance database (Round Robin Database - RRD) (3) a visualization tool (RRDtool)

9. 7. Monitoring & Analysis Tools – FlowScan loads and executes report modules of the administrator’s choosing Source: “FlowScan”, Dave Plonka

10. 7. Monitoring & Analysis Tools – arts++ • ART is a binary file format specification for storing network data • ART was initially developed at ANS (American Nuclear Society) by David Bolen (1992) • ARTS was licensed to CAIDA (1998) • ARTS data objects are generally composed of three parts: a header, a list of attributes and a data section • CAIDA has developed a C++ class library for ARTS called arts++ • arts++ Functionality • efficient data archival • aggregation in the time domain (AS, net, port, protocol, interface..) • version-specific formats • support for iostreams and UNIX file descriptors

11. 7. Monitoring & Analysis Tools – ARGUS • Audit Record Generation and Utilization System • A powerful flow-based,passive monitoring tool for IP networks • Provides tools for various analysis of network activity • Probe system: argus • Collector/Analysis tools: ra, racount, ragator, ramon, rasort, raxml • Developed originally by CMU in 1993, now coordinated by QoSient LLC as open source project • Current Release Version : 2.0.5 • Current Developing Version: 2.0.6 • http://www.qosient.com/argus • Fixed model Real-Time Flow Monitor after IETF RTFM

12. Argus Architecture Source: QoSient LLC

13. Argus Data Model • Argus flow modeled after IPPM Framework • Type-P and Type-P1-P2 flows • Bidirectional flow model <- RTFM • Packets of Type-P • Defined in RFC 2330 from IETF IPPM WG • To remove the ambiguity in the definition of Network Performance Metrics • The generic notion where in some contexts P will be explicitly defined(Type-P), partially defined(Type-P1-P2), or left generic • Example: • IP-connectivity • IP-Type-P-Connectivity, IP-Port-HTTP-Connectivity

14. Argus Flows • An Argus Flow is simply a set of datagrams that share a common set of datagram attributes. • Destination Address • Network Addresses • Addresses, Protocol, NSAPs, TTL, Session IDs, Application data, etc. • Supports 13 simultaneous flow models, enabling Layer 2, 3, 4, and 5 based flow tracking and reporting

15. Argus Flow Models • Layer 5 • RTP and RTCP (Type-P) • 8-tuple:  SrcIPAddr,DstIPAddr,L4Protocol,SrcPort,DstPort,  rh_ver, rh_seq, rh_ssrc • Layer 4 • TCP and UDP (Type-P) • 5-tuple:  SrcIPAddr, DstIPAddr, L4Protocol, SrcPort, DstPort • ESP (Type-P) • 4-tuple:  SrcIPAddr, DstIPAddr, L4Protocol, SPI • ICMP ECHO (Type-P1-P2) • 7-tuple:  SrcIPAddr, DstIPAddr, L4P, type, code,id, seq             where the type is either ECHO REQUEST or REPLY. • ICMP INFO TYPE (Type-P1-P2) • 5-tuple:  SrcIPAddr, DstIPAddr, L4P, type, code             where the type is either REQUEST or REPLY. • ICMP UNREACHABLE/REDIRECT (Type-P1-P2) • Mapped to any supported Argus flow type. • 6-tuple:  SrcIPAddr, DstIPAddr, L4P, type, code, object      • IGMP (Type-P) • 4-tuple:  SrcIPAddr, DstIPAddr, L4P, type

16. Argus Flow Models • Layer 3 • IPv4 (Type-P) • 3-tuple:  SrcIPAddr, DstIPAddr, L4Protocol    • Fragments (Type-P1-P2) • Mapped to any supported Argus flow type. • Fragments (Type-P) • 4-tuple:  SrcIPAddr, DstIPAddr, L4Protocol, ip_id • Layer 2 • LLC SNAP Encapsulation (Type-P) • 5-tuple:  SrcMACAddr, DstMACAddr, L3Proto, SrcSAP, DstSAP • ARP (Type-P1-P2) • 3-tuple: ARP_SPA, ARP_TPA, Eaddr • where the EAddr value is either the SrcMacAddr of the REQUEST or the dstMACAddr of the REPLY. • All other traffic: (Type-P) • 3-tuple:  SrcMACAddr, DstMACAddr, L3Protocol

17. Argus Flow Record Format • Common Type Length Value (TLV) Structure • Common 16 byte header struct ArgusRecord {   unsigned char type, cause;   unsigned short length;   unsigned int status;   unsigned int argusid;   unsigned int seqNumber;   union {      struct ArgusMarStructmar;      struct ArgusFarStructfar;   } ar_union;}; • A Start MAR must be the first record in an ArgusRecord Stream • A Stop MAR should be the last record

18. Argus Record Format • Type • Type of Argus Record: MAR or FAR • Length • Length of entire argus record • Status • Connectivity status, transition status • Argus ID • A unique identifier for the source argus • Sequence Number • Management Audit Record (MAR) • Provides information about argus itself • Start MAR --- Status MAR ---- Stop MAR • Flow Activity Record (FAR) • Provides information about network transaction flows that argus track • The FAR are generated either because of state or because of time • Start FAR: transaction started, Stop FAR: transaction stopped • Status FAR • Default time out : every 60 seconds

19. Argus Flow Record -MAR struct ArgusRecord {   unsigned char type, cause;   unsigned short length;   unsigned int status;   unsigned int argusid;   unsigned int seqNumber;    union {      struct ArgusMarStructmar;      struct ArgusFarStructfar;   } ar_union;}; struct ArgusMarStruct { struct timeval startime, now; unsigned char major_version, minor_version; unsigned char interfaceType, interfaceStatus; unsigned short reportInterval, argusMrInterval; unsigned int argusid, localnet, netmask, nextMrSequenceNum; unsigned long long pktsRcvd, bytesRcvd; unsigned int pktsDrop, flows, flowsClosed; unsigned int actIPcons, cloIPcons; unsigned int actICMPcons, cloICMPcons; unsigned int actIGMPcons, cloIGMPcons; unsigned int actFRAGcons, cloFRAGcons; unsigned int actSECcons, cloSECcons; int record_len;};

20. Argus Flow Record - FAR struct ArgusFarStruct { unsigned char type, length; unsigned short status; unsigned int ArgusTransRefNum; struct ArgusTimeDesctime; struct ArgusFlowflow; struct ArgusAttributesattr; struct ArgusMetersrc, dst; }; struct ArgusTimeDesc { struct timeval start; struct timeval last; }; struct ArgusFlow { union { struct ArgusIPFlowip; struct ArgusICMPFlowicmp; struct ArgusMACFlowmac; struct ArgusArpFlowarp; struct ArgusRarpFlowrarp; struct ArgusESPFlowesp; } flow_union; }; struct ArgusAttributes { union { struct ArgusIPAttributesip; struct ArgusARPAttributesarp; } attr_union; }; struct ArgusIPAttributes { unsigned short soptions, doptions; unsigned char sttl, dttl; unsigned char stos, dtos; }; struct ArgusMeter { unsigned int count, bytes, appbytes; }; struct ArgusARPAttributes { unsigned char response[8]; };

21. Argus Flow Record – FAR - Argus Flow icmp ip struct ArgusIPFlow { unsigned int ip_src, ip_dst; unsigned char ip_p, tp_p; unsigned short sport, dport; unsigned short ip_id; }; struct ArgusICMPFlow { unsigned int ip_src, ip_dst; unsigned char ip_p, tp_p; unsigned char type, code; unsigned short id, ip_id; }; 7 8 arp struct ArgusArpFlow { unsigned int arp_spa; unsigned int arp_tpa; unsigned char etheraddr[6]; unsigned short pad; }; mac struct ArgusMACFlow { struct ether_header ehdr; unsigned char dsap, ssap; }; 3 4 rarp struct ArgusRarpFlow { unsigned int arp_tpa; unsigned char srceaddr[6]; unsigned char tareaddr[6]; }; 3 esp struct ArgusESPFlow { unsigned int ip_src, ip_dst; unsigned char ip_p, tp_p; unsigned short pad; unsigned int spi; }; 6

22. Argus Flow - Canonical Record structArgusCanonicalRecord { struct ArgusRecordHeader ahdr; struct ArgusFarStruct far; struct ArgusMacStruct mac; union { struct ArgusTCPObject tcp; struct ArgusESPStruct esp; struct ArgusICMPObject icmp; struct ArgusIGMPObject igmp; struct ArgusDHCPObject dhcp; struct ArgusRTPObject rtp; struct ArgusRTCPObject rtcp; struct ArgusARPObject arp; struct ArgusAHObject ah; struct ArgusFRAGObject frag; } acr_union; struct ArgusAGRStruct agr; struct ArgusTimeStruct time; struct ArgusVlanStruct vlan; struct ArgusMplsStruct mpls; }; struct ArgusRecordHeader { unsigned char type, cause; unsigned short length; unsigned int status; unsigned int argusid; unsigned int seqNumber; }; struct ArgusMacStruct { unsigned char type, length; unsigned short status; union { struct ArgusETHERObject ether; } phys_union; }; struct ArgusETHERObject { unsigned char ethersrc[6]; unsigned char etherdst[6]; }; struct ArgusAGRStruct { unsigned char type, length; u_short status; unsigned int count; struct timeval laststartime, lasttime; struct ArgusTimeObject act, idle; }; struct ArgusTimeStruct { unsigned char type, length; u_short status; struct ArgusTimeEntity src, dst; }; struct ArgusVlanStruct { unsigned char type, length; unsigned short status; unsigned short sid, did; }; struct ArgusTimeEntity { struct ArgusTimeObject act, idle; }; struct ArgusTimeObject { int n; unsigned int min; unsigned int mean; unsigned int stdev; unsigned int max; }; struct ArgusMplsStruct { unsigned char type, length; unsigned short status; unsigned int slabel; unsigned int dlabel; };

23. Argus Flow Record – acr union tcp struct ArgusTCPObject { unsigned char type, length; unsigned short status; unsigned int state; unsigned int options; unsigned int synAckuSecs, ackDatauSecs; struct ArgusTCPObjectMetrics src, dst; }; struct ArgusTCPObjectMetrics { unsigned int seqbase, ackbytes; unsigned int bytes, rpkts; unsigned short win; unsigned char flags, pad; }; dhcp struct ArgusDHCPObject { unsigned int respaddr; }; rtp struct ArgusRTPObject { unsigned char type, length; unsigned short status; struct rtphdr src, dst; unsigned short sdrop, ddrop; unsigned short ssdev, dsdev; }; struct ArgusESPObject { unsigned int spi, lastseq, lostseq; }; esp struct ArgusESPStruct { unsigned char type, length; u_short status; struct ArgusESPObject src, dst; }; rctp struct ArgusRTCPObject { unsigned char type, length; unsigned short status; struct rtcphdr src, dst; unsigned short src_pkt_drop, dst_pkt_drop; }; icmp struct ArgusICMPObject { unsigned char type, length; unsigned short status; unsigned char icmp_type, icmp_code; unsigned short iseq; unsigned int osrcaddr, odstaddr; unsigned int isrcaddr, idstaddr; unsigned int igwaddr; }; arp struct ArgusARPObject { unsigned char respaddr[6]; unsigned short pad; }; ah struct ArgusAHObject { unsigned int src_spi, dst_spi; unsigned int src_replay, dst_replay; }; igmp struct ArgusIGMPObject { unsigned char igmp_type, pad; unsigned int igmp_group; }; frag struct ArgusFRAGObject { int fragnum, frag_id; unsigned short status, totlen, currlen, axfraglen; };

24. Argus Transport Model • Record generator (server) supports multiple access methods. • Local storage • Near-real time record access • Collector (client) initiated associations • TCP based control exchange • Proprietary protocol for capability negotiation • TCP or UDP based data transfer • SASL (Simple Authentication and Security Layer, RFC 2222) mediated security

25. Access Methods • Local Storage • Information Base for Transport Reliability • Enable retransmission capability • Support guaranteed delivery • Provide bulk transfer capability • Near-Real Time Access • Push based record transfer • Integrated management capabilities • Keep Alive/Heartbeat • Probe status and state reporting

26. Argus Record Stream • Collection of Management and Flow Activity Records • Management records convey Argus status/state (MAR) • Flow Activity Records (FAR) convey monitored flow state • Argus Stream/Files have same structure Start MAR argus Record(required) FAR Argus Record(optional) …. Status MAR Argus Record(optional) FAR Argus Record(optional) ... Stop MAR Argus Record(required)

27. Argus Practical Experiences • Data Model Supports a lot of applications • Security Assurance • Detect Service Failure • Detect DoS attack • Detect Network Configuration Problem (Policy enforcement Validation ) • Accounting/Billing • Bidirectional Flow Model • Performance Monitoring in Passive mode (IPPM Metrics) • Connectivity and reachability : unidirectional and bidirectional • Packet Loss: TCP state machine, sequence number tracking logic • Round-Trip Delay: -R option, TCP handshake establishment round trip delay metrics are provided by default • Packet Jitter and Jitter variance • Traffic Management • Operations Management

28. NG-MON • Next Generation Network Traffic MONitoring and Analysis System • Developed at DPNM Lab, POSTECH • Targeting 10 Gbps or higher networks • To support various analysis applications • Multimedia streaming & conferencing, P2P, game traffic analysis • Network security attack detection and analysis • SLA monitoring • Usage-based billing, Customer relationship management

29. NG-MON - Requirements • Distributed, load-balancing architecture for scalability • subdivide monitoring system into several functional components • efficient load sharing between phases and within each phase • pipelined and parallel architecture • Lossless packet capture • Flow-based analysis • aggregate packet information into flows for efficient processing • Considerations for small storage requirements • Support for various applications

30. raw packet packet header information flow information NG-MON - Design • NG-MON is composed of 5 phases • Packet Capture • Flow Generation • Flow Store • Traffic Analysis • Presentation & Reporting Traffic Analyzer Flow Store Packet Capturer Flow Generator Presenter Web Server Network Device User Interface Web browser analyzed data stored flows

31. Probe #1 Probe #2 Probe #3 NG-MON - Packet Capture Splitting Device Network Link divided raw packet pkt header messages • Distribution of raw packets • by using splitting function provided by an optical splitter • by using mirroring function provided in network devices • Probe • captures all packets coming into probe • export buffer-queues: one to one with flow generators • fills buffer-queues with packet header’s 5-tuple based hashing • collect the scattered packets in the same flow into the same buffer-queue

32. Flow Generator #1 Flow Generator #2 Flow Generator #3 Flow Generator #4 NG-MON - Flow Generation • Distribution of packet header information • 5-tuple based hashing in the probe • Packet header messages of potentially the same flow get delivered to the same flow generator • Flow generator receives packet header messages and generates flows and exports flow messages to flow store pkt header messages flow messages

33. Flow Store #1 t 1 Flow Store #2 Flow Store #3 NG-MON - Flow Store • Separation of write operations from read operations • the destination address of flow message is assigned to the flow store according to the time • While one or more flow stores are inserting flow data, the other flow stores are queried by the traffic analyzers • Flow store provides traffic information to support various analysis applications • provides an analysis API to analyzers Database Query / Response Traffic Analyzer #1 Traffic Analyzer #2 flow messages t 2 t 3 Write operations Read operations

34. Web Server Flow Store #2 Flow Store #3 NG-MON - Traffic Analysis & Presentation • Analyzer extracts information from Flow Stores and can perform application specific analysis • Separate analyzer is needed for each application Traffic Throughput Analyzer Flow Store #1 Usage-based billing application Presenter DDoS or DoS Attack Analyzer Other applications

35. NG-MON - Implementation

36. Router Router NG-MON - Deployment at POSTECH http://ngmon.postech.ac.kr 141.223.182.[31,32,33,34] POSTECH Computer Center INTERNET 141.223.182.36 EnterFLEX at Computer Center 141.223.182.40 EnterFLEX at Computer Center Packet Capture Flow Generator Flow Store Packet Capture Flow Generator Analyzer Presenter Packet Capture Flow Generator Flow Store Packet Capture Flow Generator 141.223.182.38 EnterFLEX at Computer Center 1Gbps Optical link 141.223.182.37 EnterFLEX at Computer Center NetOptics 1Gbps Optical Splitter POSTECH Gigabit Campus Network

37. NG-MON - Host Data SentMinute View

38. NG-MON - Host Data Exchanged Minute View

39. NG-MON - Detailed Host Data Received Minute View

40. NG-MON –Network Security Analysis Minute View

41. NG-MON –Detailed Security Analysis Minute View

42. NG-MON -Application Protocol Minute View

43. NG-MON -Application Protocol Minute View

44. Flow-based Passive Monitoring Tools Summary • Input: L – LAN, W – WAN, G - Giga • Measurement: A – Active, P – Passive, P – Protocol distribution, U – Utilization, R – RTT, L – Packet Loss • Scope: R – Real time, O - Offline