1 / 103

Internet Measurement Tutorial

Internet Measurement Tutorial. Yuval Shavitt School of Electrical Engineering. http://www.eng.tau.ac.il/~shavitt. Motivation. Wide area networks are too complex to grasp Many protocols at various levels interact and effect behavior Many applications have performance requirements

Roberta
Download Presentation

Internet Measurement Tutorial

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Internet Measurement Tutorial Yuval Shavitt School of Electrical Engineering http://www.eng.tau.ac.il/~shavitt

  2. Motivation • Wide area networks are too complex to grasp • Many protocols at various levels interact and effect behavior • Many applications have performance requirements • End-to-end delay and loss, reliability

  3. Motivation (2) • Its an interesting complex system • Has emergent characteristics like many living systems: • Biological systems • Social networks

  4. TELNET FTP SMTP DNS TCP UDP IP LAN wireless WAN TCP/IP Protocols Application Transport Network Physical+ Data link

  5. Internet Measurement Challenges

  6. Internet Measurement Challenges (1) • Network size: • 100,000,000s hosts, 1,000,000s routers, ~30,000 ASes • Network Complexity • Interaction between components, protocols, applications, users • All change over time • New applications are added • New protocol versions (TCP) • New router design (AQM)

  7. Internet Measurement Challenges (2) • Not engineered for measurement: • Initial design had no measurement thinking • Distributed management • Tendency not to share data • Blocking measurement attempts (“don’t ping my network”) • NATs, Firewalls, …

  8. Success Stories “On the self-similar nature of Ethernet traffic” W. E. Leland, M. S. Taqqu, W. Willinger, and D. V. Wilson IEEE/ACM Transactions on Networking, February 1994. • Thorough analysis of Bellcore LAN traces established self-similar properties of packet arrival process. “On power-law relationships of the internet topology” M. Faloutsos, P. Faloutsos, and C. Faloutsos, ACM SIGCOMM 1999, Aug./Sept. 1999. • Analysis of the RouteViews BGP database establish the power-law characteristics of the Internet topology. Pr(k) <k> k

  9. Why do we measure the Internet? • Already mentioned: • Because it is there! • Operational reasons • We cannot improve the Internet if we don’t understand it • We cannot understand it if we don’t measure • We cannot build effective models or simulators if we don’t measure

  10. Long term objectives • Monitor the Internet at real time • Manage the Internet • Monitor and react before things go bad

  11. What can we measure in the Internet? • Structure • Topology (router/network) connectivity, link capacities, link loss, available bandwidth, routing • Traffic • End-to-end performance, packet arrival process (congestion built-up) • Users and applications • WWW, peer-to-peer, streaming • Malicious behavior • Attack patterns, port scans

  12. Where can we measure the Internet? How to chose representative measurement points? Example: traffic samples • LAN traffic vs. WAN traffic, • Inside an ISP vs. between continents • Country biases • Commercial location vs. educational • More locations is better

  13. How can we measure the Internet? • Active measurements • Probes: Traceroute, ping, packet trains • Application simulation • Passive measurement • Logs (WWW) • Monitors, sniffers

  14. Measurement resources on the WWW CAIDA: www.caida.org/tools/taxonomy SLAC: www.slac.stanford.edu/xorg/nmtf/nmtf-tools.html

  15. When should we measure the Internet? • Diurnal and weekly traffic cycles • Time scales depend on “what” and “how” • Passive measurement are typically continuous • Can generate huge data sets • Log access problems • Privacy concerns • Active measurements are typically discrete • Important characteristics can be missed • Probes can be filtered and/or detected

  16. Who is measuring the Internet? • Businesses do a great deal of measurement • Mostly do not share with the research community • examples: • Akamai: http delay from server side • HP (Mercury): http delay from client side • Google: everything • Academia and Research institutes • Publish papers, but data may not be always available • Internet Statistics and Metrics Analysis (ISMA) • CAIDA attempt to create a global meta-data database

  17. Publishing Internet Measurement Studies • All major networking conferences & journals accept measurement papers • ACM SIGCOMM, IEEE INFOCOM, ACM SIGMETRICS • Dedicated meetings: • ACM Internet Measurement Conf. (IMC, IMW) • Passive & Active Measurements Conf. (PAM) • TridentCom

  18. Active Measurement Techniques

  19. Active Probes • Active probes send stimulus (packets) into the network and then measure the response • Done on network, transport and application layers • Active probes are useful to measure various things: • Delay, delay jitter, and loss • Topology and routing behavior • Capacity, bandwidth, and throughput

  20. Simple delay/loss probing with ping C:\>ping www.fer.hr Pinging www.fer.hr [161.53.72.111] with 32 bytes of data: Reply from 161.53.72.111: bytes=32 time=113ms TTL=49 Reply from 161.53.72.111: bytes=32 time=111ms TTL=49 Reply from 161.53.72.111: bytes=32 time=113ms TTL=49 Reply from 161.53.72.111: bytes=32 time=118ms TTL=49 Ping statistics for 161.53.72.111: Packets: Sent = 4, Received = 4, Lost = 0 (0% loss), Approximate round trip times in milli-seconds: Minimum = 111ms, Maximum = 118ms, Average = 113ms

  21. ICMP ICMP is the IP error diagnosis protocol.

  22. PING

  23. Application layer “ping” • One can generate application layer messages to test application reaction time • Most common: • TCP SYN message to port 80

  24. traceroute • Useful to learn the route characteristics between two hosts. • Sends a series of probes to successive nodes along a route to an intended destination and records the source address and time delay of the message returned by each. • Based on ICMP “TTL expired” message

  25. IP datagram format IP protocol version number 32 bits total datagram length (bytes) header length (bytes) type of service head. len ver length for fragmentation/ reassembly fragment offset “type” of data flgs 16-bit identifier max number remaining hops (decremented at each router) upper layer time to live Internet checksum 32 bit source IP address 32 bit destination IP address upper layer protocol to deliver payload to E.g. timestamp, record route taken, pecify list of routers to visit. Options (if any) data (variable length, typically a TCP or UDP segment)

  26. TypeCodedescription 3 0 dest. network unreachable 3 1 dest host unreachable 3 2 dest protocol unreachable 3 3 dest port unreachable 3 6 dest network unknown 3 7 dest host unknown traceroute

  27. traceroute time A B C D E • Regular UDP packets • successive TTLs • ICMP “TTL expired” message • ICMP “port unreachable” message

  28. traceroute versions • UNIX: • default send UDP packets • Start at port 33435, and increment port per packet! • traceroute –l sends ICMP “ECHO request” • tcptraceroute uses TCP SYN messages • If port is close gets RST reply • If port is open gets SYN ACK and reply with RST • Best to overcome firewalls • Windows • ICMP “ECHO request”

  29. C:\>tracert www.fer.hr Tracing route to www.fer.hr [161.53.72.111] over a maximum of 30 hops: 1 <1 ms <1 ms <1 ms 192.168.200.254 2 19 ms 20 ms 19 ms vxr.tau.ac.il [132.66.8.10] 3 17 ms 22 ms 20 ms c6509.tau.ac.il [132.66.8.20] 4 21 ms 19 ms 19 ms tel-aviv.tau.ac.il [132.66.4.1] 5 19 ms 23 ms 18 ms gp1-tau-fe.ilan.net.il [128.139.191.70] 6 20 ms 20 ms 20 ms iucc.il1.il.geant.net [62.40.103.69] 7 69 ms 69 ms 69 ms il.it1.it.geant.net [62.40.96.154] 8 82 ms 82 ms 82 ms it.ch1.ch.geant.net [62.40.96.33] 9 101 ms 98 ms 98 ms ch.at1.at.geant.net [62.40.96.1] 10 105 ms 105 ms 105 ms at.hu1.hu.geant.net [62.40.96.178] 11 117 ms 112 ms 113 ms hu.hr1.hr.geant.net [62.40.96.145] 12 113 ms 115 ms 115 ms carnet-gw.hr1.hr.geant.net [62.40.103.218] 13 120 ms 122 ms 123 ms 193.198.228.6 14 114 ms 112 ms 119 ms 193.198.229.10 15 120 ms 119 ms 119 ms 161.53.16.14 16 114 ms 114 ms 113 ms duality.cc.fer.hr [161.53.72.111] Trace complete.

  30. C:\>tracert www.colbud.hu Tracing route to www.colbud.hu [81.182.250.153] over a maximum of 30 hops: 1 <1 ms <1 ms <1 ms 192.168.200.254 2 19 ms 21 ms 18 ms vxr.tau.ac.il [132.66.8.10] 3 20 ms 21 ms 21 ms c6509.tau.ac.il [132.66.8.20] 4 21 ms 20 ms 19 ms tel-aviv.tau.ac.il [132.66.4.1] 5 20 ms 22 ms 19 ms gp1-tau-fe.ilan.net.il [128.139.191.70] 6 26 ms 22 ms 21 ms iucc.il1.il.geant.net [62.40.103.69] 7 91 ms 92 ms 92 ms il.nl1.nl.geant.net [62.40.96.117] 8 97 ms 97 ms 97 ms nl.de1.de.geant.net [62.40.96.101] 9 95 ms 96 ms 93 ms ffm-b2-pos2-3.telia.net [213.248.77.89] 10 96 ms 96 ms 150 ms ffm-bb2-pos2-3-0.telia.net [213.248.64.177] 11 110 ms 112 ms 114 ms bpt-b1-pos2-0.telia.net [213.248.64.26] 12 * * * Request timed out. 13 112 ms 110 ms 111 ms 10ge-0-0.core0-ip2.net.telekom.hu [145.236.85.2] 14 112 ms 114 ms 110 ms tenge1-2.core0.adatpark.hu [145.236.89.10] 15 114 ms 112 ms 114 ms fixip-lns2.adatpark.hu [195.228.253.58] 16 120 ms 122 ms 124 ms 153-250-182-81.adsl-fixip.axelero.hu [81.182.250.153] Trace complete.

  31. Probing for link characteristics • Packet dispersion techniques can be used to infer characteristics of each link along an Internet path. • Bandwidth, queuing delays, propagation delay • Cross traffic may cause problem • Many tools are available: • bprobe [CC97], clink [D99], nettimer [LB99], pathchar [J97], pchar [M00], pathrate [DRM01]

  32. Capacity • Maximum IP­layer throughput that a flow can get, without any cross traffic link 2 link 1 link 3 sink source C • Ci = capacity of link i • Path capacity C=mini{Ci}

  33. Available Bandwidth • Maximum IP­layer throughput that a flow can get, given (stationary) cross traffic link 2 link 1 link 3 A sink source C • ui = utilization of link i • Path available bandwidth A=mini{Ci(1- ui)}

  34. Packet Pair Dispersion • Packet transmission time: τ=L/C • Send two packets back-to-back • Measure dispersion  at the reciever • Estimate C as L/ • But cross-traffic ‘noise’ can effect . C 3C L/C L/3C L/C

  35. Pathchar • Developed by Van Jacobson to: “allows any user to find the bandwidth, delay, average queue and loss rate of every hop between any source & destination on the Internet” • Measure the path hop by hop • Default: 32 probes per hop

  36. Self-Loading Periodic Streams (SLoPS) [Jain Dovrolis 02] • SND sends a periodic UDP packet stream at rate R. • R=L/T, L=packet size, T=period, K=number of packets • Measure one way delay (OWD): Dk=tarrive-tsend • OWD variation: Dk=Dk+1-Dk(independent of clock offset) • With stationarity & fluid model for the cross traffic, and FIFO queues:

  37. Illustration of SLoPS Periodic Stream: K packets, size L bytes, rate R = L/T

  38. Trends in Real Data U. Oregon to U. Delaware (12 hops) A=74Mbps (MRTG), K=100, T=100S, L=1200B R= 96Mbps and 37Mbps

  39. When RA

  40. Passive Measurement Techniques

  41. Passive packet measurement • Capture packets as they pass by • Packet capture applications (tcpdump) on hosts use packet capture filter • Requires access to the wire • Promiscuous mode or mirror ports to see other traffic • Hardware-bases solutions • Endace, Inc.’s DAG cards … OC12/48/192 (0.622/2.5/10Gbps) • Programmable NIC cards (<$100) • Issues: • Timestamps • Data volumes • Privacy

  42. tcpdump • Can capture entire packet or n first bytes • Timestamps each packet • Can filter based on any combination of header field 12:40:18.501228 IP bakara.eng.tau.ac.il.23 > amirotem-pc.eng.tau.ac.il.2260: P 1:3(2) ack 1 win 8760 (DF) 12:40:18.692431 IP amirotem-pc.eng.tau.ac.il.2260 > bakara.eng.tau.ac.il.23: . ack 3 win 64162 (DF) 12:40:18.692775 IP bakara.eng.tau.ac.il.23 > amirotem-pc.eng.tau.ac.il.2260: P 3:10(7) ack 1 win 8760 (DF) 12:40:18.893601 IP amirotem-pc.eng.tau.ac.il.2260 > bakara.eng.tau.ac.il.23: . ack 10 win 64155 (DF)

  43. Full Packet Capture 12:22:42.401784 IP (tos 0x0, ttl 128, id 37074, len 41) AMIROTEM.dummy.net.3214 > bakara.eng.tau.ac.il.23: P [tcp sum ok] 3535692137:3535692138(1) ack 1410929928 win 16196 (DF) 0x0000 4500 0029 90d2 4000 8006 2d02 c0a8 c803 E..)..@...-..... 0x0010 8442 300c 0c8e 0017 d2be 6169 5419 1508 .B0.......aiT... 0x0020 5018 3f44 1d9e 0000 6c P.?D....l 12:22:42.426889 IP (tos 0x0, ttl 252, id 33630, len 41) bakara.eng.tau.ac.il.23 > AMIROTEM.dummy.net.3214: P [tcp sum ok] 1:2(1) ack 1 win 9324 (DF) 0x0000 4500 0029 835e 4000 fc06 be75 8442 300c E..).^@....u.B0. 0x0010 c0a8 c803 0017 0c8e 5419 1508 d2be 616a ........T.....aj 0x0020 5018 246c 3875 0000 6c88 8888 8888 P.$l8u..l..... 12:22:42.600874 IP (tos 0x0, ttl 128, id 37075, len 41) AMIROTEM.dummy.net.3214 > bakara.eng.tau.ac.il.23: P [tcp sum ok] 1:2(1) ack 2 win 16195 (DF) 0x0000 4500 0029 90d3 4000 8006 2d01 c0a8 c803 E..)..@...-..... 0x0010 8442 300c 0c8e 0017 d2be 616a 5419 1509 .B0.......ajT... 0x0020 5018 3f43 169d 0000 73 P.?C....s 12:22:42.617003 IP (tos 0x0, ttl 252, id 33631, len 41) bakara.eng.tau.ac.il.23 > AMIROTEM.dummy.net.3214: P [tcp sum ok] 2:3(1) ack 2 win 9324 (DF) 0x0000 4500 0029 835f 4000 fc06 be74 8442 300c E..)._@....t.B0. 0x0010 c0a8 c803 0017 0c8e 5419 1509 d2be 616b ........T.....ak 0x0020 5018 246c 3173 0000 7388 8888 8888 P.$l1s..s.....

  44. Passive IP flow measurement • An IP flow is defined by the five-tuple: • src addr, src port, dst addr, dst port, protocol • Cisco’s NetFlow • Part of the IOS • Provide template based flow records • Many tools can manipulate NetFlow data

  45. FlowScan [Plonka00] • Combines flow collection engine, database, visualization tool • Provides a near real-time visualization of network traffic • Breaks down traffic into well known service or application

  46. Analysis of Flows • Examining flows of packets one can determine OOO packets: • Losses • Reorders • TCP state machine • Retransmissions • Duplicates • Analysis can be done on 1- or 2-directional flows

More Related