260 likes | 371 Views
Now and Then, How and When? June 16 th , 2009 Stephen Donnelly Technologist | Endace Technology SHARK FEST '09 Stanford University June 15-18, 2009. Endace. Potted history 1996 The University of Waikato 2001 Endace created 2005 Publically Listed Specialists in packet capture
E N D
Now and Then, How and When? June 16th, 2009 Stephen Donnelly Technologist | Endace Technology SHARKFEST'09 Stanford University June 15-18, 2009
Endace Potted history 1996 The University of Waikato 2001 Endace created 2005 Publically Listed Specialists in packet capture High data/packet rates Accurate time stamping Wide variety of network interfaces
Network Monitoring Interfaces DAG cards cover many network technologies 8000 bps to 39813120000 bps TDM - T1/E1/J1 PDH - T3/E3 SONET/SDH - OC-3, 12, 48, 192, 768 InfiniBand – SDR, DDR
Platforms and Appliances Open Platforms Full access Managed Appliances Packet Capture Trace Replay Applied Watch IDS Flow Export Lawful Intercept CACE Pilot
Lossless Packet Capture Capture all packets on link Categorize Filter Present to user Debugging Security Forensics Lawful Intercept
Network Interface Cards Designed to provide inexpensive network connectivity for diverse applications Web, Email, File transfer Generally applications are the bottleneck E.g. a web server generating content Protocols are fault tolerant so NIC need not be LAN traffic is bursty
NIC Device Model Packet Buffers Application Libpcap Tx Descriptor Ring Rx Descriptor Ring Network Stack Packet Filter NIC Driver
Performance Testing Simple Libpcap app counting packets Packets Captured vs. Applied CPU Load Single processor core AMD Opteron 248 (2.2GHz) 2GB DDR 400 DRAM Linux 2.6.12
DAG cards Optimized for packet capture and replay Efficient transfer to and from user applications Capture 100% of received packets Full or partial packet capture Account for any packet loss that does occur Record accurate timestamps Synchronized clocks for timestamp comparisons ERF Format with rich per-packet metadata
DAG Internals Power Supply Circuits 1 to n Network Physical Layer Interface/s Network Interface / Framer JTAG / Test Connector/s Clock Oscillator FPGA CPLD LEDs ROM Sync Connector Bus Connector Features only on subset of cards Processor RAM Coprocessor FIFO
DAG Stream Buffer Large Static Ring Buffers 4MB to 2GB each Window-based Handshaking Minimize per-packet overhead Memory-mapped to User space Zero copy
DAG Device Model Tx Stream Rx Stream Rx Stream Application Libpcap Libdag Network Stack Packet Filter DAG Driver
Accurate time stamps Debugging/Benchmarking/Optimization QoS/SLA Service response time Storage networks Network equipment HPC Financial services Time=Money, Latency=Risk
Reference Clocks GPS Worldwide Clear view of sky CDMA Works indoors Limited coverage Unknown distance to tower Radio (Shortwave) Limited by RF Propagation