Interdomain Routing Policies in the Internet: Inference and Analysis

1. Interdomain Routing Policiesin the Internet:Inference and Analysis Massimo Rimondini Defense of the PhD in Computer Science and Engineering Mar 16th, 2007

2. Interdomain Routing Autonomous Systems ISP BGP LAN WAN Internet

3. 2 2 1 Routing Policies CLASSIFIED 1 2

4. Why Routing Policies? Directions • Routing dynamics • Routing consistency • Oscillation prevention • Debugging • Traffic engineering • Interconnection strategies • Interdomain topology discovery • Inference of commercial relationships • Traffic engineering • Interplay of routing policies • Emulation of computer networks ? ?

5. N W E S Interdomain Topology Discovery

6. State of the Art • University of Oregon Route ViewsRIPE Routing Information Service • W. Mühlbauer, A. Feldmann, O. Maennel, M. Roughan, S. Uhlig. Building an AS-topology Model that Captures Route Diversity. ACM SIGCOMM 2006 • B. Zhang, R. Liu, D. Massey, L. Zhang. Collecting the internet AS-level topology. ACM SIGCOMM Computer Communication Review, 2005 • X. Dimitropoulos, D. Krioukov, G. Riley. Revisiting Internet AS-level Topology Discovery. Proc. PAM ’05 • H. Chang, R. Govindan, S. Jamin, S. J. Shenker, W. Willinger. Towards Capturing Representative AS-level Internet Topologies. Computer Networks, 2004 • Internet Routing Registry • P. Mahadevan, D. Krioukov, M. Fomenkov, X. Dimitropoulos, kc claffy, and A. Vahdat. The Internet AS-Level Topology: Three Data Sources and One Definitive Metric. ACM SIGCOMM Computer Communication Review, 2006 • G. Siganos, M. Faloutsos. Analyzing BGP Policies: Methodology and Tool. Proc. IEEE INFOCOM ’04 • CAIDA Skitter • B. Huffaker, D. Plummer, D. Moore, kc claffy. Topology Discovery by Active Probing. Proc. SAINT ’02 • kc claffy. Internet Measurement and Data Analysis: Topology, Workload, Performance and Routing Statistics. NAE Workshop, 1999 • University of Washington Rocketfuel • N. Spring, R. Mahajan, D. Wetherall, T. Anderson. Measuring ISP Topologies with Rocketfuel. IEEE/ACM Trans. on Networking, 2004 • N. Spring, R. Mahajan, T. Anderson. Quantifying the Causes of Path Inflation. Proc. ACM SIGCOMM 2003 • R. Mahajan, N. Spring, D. Wetherall, T. Anderson. Inferring Link Weights using End-to-End Measurements. Proc. IMW ’02

7. State of the Art AS level Registry Router level BGP data Active probing Passive observation

8. BGP Probing 3 C 9 9520 5 520 2 7 20 0 0 0 0 8 4 C 1 6 C

9. BGP Probing 3 C 9 30{12} 9530{12} 5 530{12} 2 7 0 0{12} 0{12} 0{12} 8 4 C 1 6 C

10. BGP Probing: Contributions • Probing primitives • AS-set stuffing • Withdrawal observation • Exploration algorithm • Experimentation on the IPv4/IPv6 Internet • Route flap dampening analysis • Publications • Technical report • ISCC 2006 • Journal (pending acceptance)

11. BGP Probing: Results

12. IRR Data Extraction • Internet Routing Registry • assignment of Internet resources (IP addresses, AS numbers) • routing policies • Volunteered updates • inconsistent registrations • out of date information

13. IRR Data Extraction Routing Policy Specification Language aut-num: AS137 import: from AS20965 action pref=100; from AS1299 action pref=100; accept ANY [...] export: to AS1299 announce AS-GARR [...] changed: noc@garr.it 20000830 source: RIPE import: from AS20965 action pref=100; from AS1299 action pref=100; accept ANY export: to AS1299 announce AS-GARR

14. IRR Data Extraction Routing Policy Specification Language aut-num: ASX5 import: { from ASX2:AS-Z2 accept 100.0.0.0/8; } refine { from ASX1 ASX2 accept 100.1.0.0/16; } except { from ASX3 accept 100.1.1.0/24;} export: to ASX1:PRNG-Y1 to ASX1:AS-Z1 except ASX9 announce 100.1.1.0/24 mp-export: to ASX11 at 2001::1 announce 2001::/48 default: to ASX12 action pref=10 default: to ASX13 100.1.1.1 at 100.1.1.2 ASX2:AS-Z2 } refine { ASX1 ASX2 } except { ASX1:PRNG-Y1 ASX1:AS-Z1 except ASX9 mp-export: to ASX11 at 2001::1 announce 2001::/48

15. IRR Data Extraction aut-num: AS24336 as-name: DIGITALBANK-JP descr: d-b net Backbone import: from AS17685 accept ANY export: to AS17685 announce AS24336 admin-c: DM210-JP tech-c: DM211-JP notify: matsuo@po.d-b.ne.jp mnt-by: MAINT-AS24336 changed: matsuo@po.d-b.ne.jp 20050220 source: RADB AS24336 aut-num: AS24336 as-name: DIGITALBANK-JP descr: DIGITALBANK, Inc., Regional ISP in Japan country: JP import: from AS17685 action pref=100; accept ANY import: from AS7682 action pref=100; accept ANY export: to AS17685 announce AS24336 export: to AS7682 announce AS24336 admin-c: DM210-AP tech-c: DM211-AP notify: matsuo@po.d-b.ne.jp mnt-routes: MAINT-JP-DIGITALBANK mnt-by: MAINT-JP-DIGITALBANK changed: hm-changed@apnic.net 20050210 source: APNIC AS24336 import: from AS17685 accept ANY export: to AS17685 announce AS24336 import: from AS17685 action pref=100; accept ANY import: from AS7682 action pref=100; accept ANY export: to AS17685 announce AS24336 export: to AS7682 announce AS24336 20050220 RADB 20050210 APNIC

16. IRR Data Extraction: Contribs • A methodology to extract BGPlinks from the IRR • A classification of the linksinto confidence levels • An on-line service • Extracted links • Statistics about the health of the IRR • Collaboration with the RIPE NCC • Publications • Technical report • ACM SIGCOMM MineNet 2006

17. IRR Data Extraction: Results

18. N W E S Inference ofCommercial Relationships

19. Commercial Relationships 3 C 9 5 2 7 0 8 4 C 1 6 C

20. Core Commercial Relationships 0 1 2 3 Provider € 4 5 Customer Peer Peer € 7 6 C 8 9 C C

21. State of the Art • X. Dimitropoulos, D. Krioukov, B. Huffaker, kcclaffy, G. Riley. Inferring AS Relationships: DeadEnd or Lively Beginning? LNCS, 2005 • G. Di Battista, M. Patrignani, M. Pizzonia. Computingthe Types of the Relationships between AutonomousSystems. Proc. IEEE INFOCOM ’03 • T. Erlebach, A. Hall, T. Schank. ClassifyingCustomer-Provider Relationships in the Internet. Proc.CCN ’02 • L. Subramanian, S. Agarwal, J. Rexford, R. H. Katz. Characterizing the Internet Hierarchy from Multiple Vantage Points. Proc. IEEE INFOCOM ’02 • L. Gao. On Inferring Autonomous System Relationships in the Internet. IEEE/ACM Transactions on Networking, 2001 • Z. Ge, D. R. Figueiredo, S. Jaiswal, L. Gao. On the Hierarchical Structure of the Logical Internet Graph. Proc. SPIE ITCom ’01 • L. Gao. On Inferring Autonomous System Relationships in the Internet. Proc. IEEE Global Internet Symposium ’00 • combinedapproach • SAT based • heuristics • SAT based • apx algorithms • combinatorialformulation • vantage points • hierarchy • policies • degree based

22. No valley please Validation

23. Algorithm independence report Stability report Validation Inference algorithms Data set (AS paths) Comparative validation

24. Validation: Contributions • Methodology • Set of measures • Software suite • Validation of [sark02]against [dpp03] • Publications • IPS 2004

25. Commercial Relationships:Results • Algorithm independence(% consistent assignments on same data set) • >90% • Stability (% consistent assignments over consecutive data sets) • [dpp03]: between 95% and 99% • [sark02]: between 96% and 99% 10 100 # edges # changes

26. N W E S Traffic Engineering

27. Traffic Engineering 2 1 3 0 0 0 0 00 0

28. State of the Art • R. K. C. Chang, M. Lo. Inbound Traffic Engineeringfor Multihomed ASes Using AS Path Prepending. Proc.NOMS ’04 • B. Quoitin, C. Pelsser, L. Swinnen, O. Bonaventure,S. Uhlig. Interdomain Traffic Engineering with BGP. IEEE Communications Magazine, 2003 • N. Feamster, J. Borkenhagen, J. Rexford. Guidelines for Interdomain Traffic Engineering. ACM SIGCOMM Computer Communication Review, 2003 • B. Quoitin, S. Uhlig, C. Pelsser, O. Bonaventure. Internet Traffic Engineering Techniques. TR, 2002 • L. Swinnen, S. Tandel, S. Uhlig, B. Quoitin, O. Bonaventure. An Evaluation of BGP-based Traffic Engineering Techniques. TR, 2002 • D. Awduche, A. Chiu, A. Elwalid, I. Widjaja, and X. Xiao. Overview and Principles of Internet Traffic Engineering. RFC 3272, 2002. • L. Gao, F. Wang. The Extent of AS Path Inflation by Routing Policies. Proc. IEEE Global Internet Symposium ’02 • H. Tangmunarunkit, R. Govindan, S. Shenker, D. Estrin. The Impact of Routing Policy on Internet Paths. Proc. IEEE INFOCOM ’01 guidelines observation

29. Seeking for Optimal Prepending • ILP formulation • Different objective functions • Multiple prefixes • Tie break • Computational geometry prepending on ui ui X 1 2 3 X 0

30. Traffic Engineering: Contribs • Two formal models • Avoid trial-and-error • “Pluggable” requirements • Algorithmic optimization • Publications • IPS MoMe 2005

31. N W E S Interplay of Routing Policies

32. Policy Probing • N. T. Spring. Efficient Discoveryof Network Topology andRouting Policy in the Internet.PhD Thesis U. Washington. 2004 • F. Wang, L. Gao. Inferring and Characterizing Internet Routing Policies. ACM SIGCOMM IM ’03 • See commercial relationships

33. ! ? Policy Probing 0 0{1,6,7} 0{1,3,5,6,7,9} 1 2 3 4 5 6 8 9 7 10 C C 7 4 1 0 9 6 3 0 C 7 4 2 0 ... 10 9 5 2 0 10 8 4 2 0 10 8 4 1 0 10 9 5 3 0 10 8 4 2 0

34. Policy Probing: Contributions • Path feasibility determination • Path preference comparison • Experiments on the IPv4/IPv6Internet

35. Stability • J. L. Sobrinho. Network Routing with PathVector Protocols: Theory and Applications.Proc. ACM SIGCOMM ’03 • T. G. Griffin, F. Bruce Shepherd, G. Wilfong.The Stable Paths Problem and Interdomain Routing.IEEE/ACM Transactions on Networking, 2002 • L. Gao, T. G. Griffin, J. Rexford. Inherently Safe Backup Routing with BGP. Proc. IEEE INFOCOM ’01 • T. Griffin, G. T. Wilfong. A Safe Path Vector Protocol. Proc. IEEE INFOCOM ’00 • K. Varadhan, R. Govindan, D. Estrin. Persistent Route Oscillations in Inter-domain Routing. Elsevier Computer Networks, 2000 • L. Gao, J. Rexford. Stable Internet Routing without Global Coordination. Proc. ACM SIGMETRICS ’00 • T. G. Griffin, F. B. Shepherd, G. Wilfong. Policy Disputes in Path-Vector Protocols. Proc. ICNP ’99)

36. Stability 130 10 1 0 210 20 320 30 2 3

37. Stability: Contributions • Event timings • Two kinds of stability • strict stability • forwarding stability • Properties • Transitions & reachability

38. N W E S Emulation of Computer Networks

39. State of the Art Royal Institute of Technology, Sweden IMUNES - An Integrated Multiprotocol Network Emulator / Simulator University of Zagreb Technical University of Madrid UMLMON - Virtualization with User Mode Linux Gerd Stolpmann

40. blah blah blah... Netkit The poor man's system to experiment computer networking • Easy to install and use • Lightweight • Easy setup of complex experiences • Compact all-in-one virtual labs • Corpus of ready-to-use labs & teaching material

42. Contributions • Improved user interface • Stability • Functionalities • Usability • Improved labs • Self tests • Lecture slides • Publications • Technical report • Nearly submitted paper F2.0 Beta - Virtual machines boot messages have been deeply cleaned up. - Maximum number of days between mounts and maximum number of mounts before filesystem check have both been reset to 0. You won't go through those annoying file system checks any more! - Support for two console terminals is provided. - All console terminals now use automatic login as root. You will no longer have to enter the password hundreds of times when starting your labs... - Filesystem now provides support for kernel modules. That is, it provides a set of modutils to manage modules and some scripts automatically mount a host directory containing kernel modules, if this is found to exist. - Netkit startup scripts have been cleaned up and improved. - Fixed wrong minor number for device /dev/ubd/disc0/part1. Also, the following devices have been created: /dev/ubda /dev/ubda1 /dev/ubda2 /dev/ubdb /dev/ubdb1 /dev/ubdb2 - Support for automatically performed lab testing has been introduced. - Static mappings in /etc/hosts and the resolver configuration in /etc/resolv.conf have both been cleaned up. - Default passwords for zebra, bgpd, and ripd have all been set to 'zebra'. - Several tools have been installed and a list of installed packages is now being distributed. --------------------------------------------------------------------------------- K2.2 - Simplified (and fixed) deploying of Netkit kernel modules. The simplification consists in the fact that explicitly dealing with symbolic links is not needed any more: just run `make modules_install' with a seasonable INSTALL_MOD_PATH and that's it. Try `man netkit-kernel' for more information. The fix consists in the fact that, before this release, it was not possible to use modules for a kernel whose version did not match the one expected by the Netkit filesystem. K2.1 Beta - Fixed a bug which limited the maximum number of arguments on the UML kernel command line (as a consequence, virtual machines could not be configured with more than about 6 interfaces, depending on the other options passed to vstart). - The port-helper patch supplied with the Netkit kernel was erroneously reversed. Fixed. K2.0 Beta - A new 2.6.11.7 kernel has been recompiled from scratch. The kernel configuration has been completely revised. Minimal patches have been applied to provide for better stability. - Support for module loading has been introduced, so that the kernel configuration now includes basic commonly used functionalities as built-in pieces, while most of the other features are available as modules. - The kernel should now work flawlessly also with SKAS-patched host kernels. 2.4 - Very improved ltools: - Some new options have been introduced, allowing to change the behaviour of the underlying vtools. The most interesting improvement is the possibility to start multiple virtual machines simultaneously. Depending on user needs, this can also be done while keeping an eye on dependencies (e.g., "pc3 can only start after pc1 and pc2"). In a few words: lab.dep is now supported! - lstart now warns if there are duplicate interface definitions inside lab.conf. Also, any option of vstart can be used inside lab.conf. - lclean now also removes log files. - lrestart has disappeared, and is now simply a link to lstart. The same holds for ltest, which is a link to lstart. Depending on the name with which it is invoked, lstart's behaviour changes as needed. - lstart now produces much terser output. - By default, lcrash removes virtual machines filesystems, so that a lab can be restarted without having to use lclean first. - No more need to use the "." to tell the ltools that the lab is in the current directory. - The command line help is now much clearer. - A new lcommand has been introduced: linfo. It provides general information about a lab without starting it. Optionally, it may also generate a PostScript file with a sketch of the link-level topology of the lab. - (More or less slightly) improved man pages. - Enriched the documentation with several warnings about installing Netkit on FAT32 filesystems and using scripts and/or lab configurations which use the CR+NL line break convention. - Removed a vcrash warning stating the impossibility to delete a socket file for a virtual hub. - Fixed a bug which prevented vlist and halt/crash related commands from working properly with accounts using long user names (maximum tolerated user name length is now 35 characters - I know it's weird to set a limit on it, but it's really necessary ;-). - Improved check_configuration.sh (now it has a modular structure and also warns about VDSO support in the host kernel). - The --version option now reports information about the version number of all the installed Netkit components as well as that of the host kernel. - Improved vcrash mechanism. Hangups of the mconsole helper tool are now detected and properly handled (any of you who has ever seen those everlasting "Crashing..." lines knows what I am talking about). Moreover, if crash via mconsole fails, then vcrash automatically attempts to kill virtual machines without user intervention. Last but not least, the process killing routine has been revised; before this fix, there were chances of leaving some processes running even after a "hard kill". Now there are very fewer chances for virtual machines to survive ;-) - Depending on PIDs usage, the uptime information returned by vlist could be incorrect. Uptime has now been completely removed from the output of vlist, as it can be obtained by using the `uptime' command inside virtual machines. - Improved support for Konsole tabs. Better management of stale locks and of simultaneously started virtual machines. - The fuser command is not required any more: it has been replaced by an invocation of lsof. - Missing parameters in netkit.conf are now assigned default values. 2.3 - Improved checks for the presence of terminal emulator applications (both in check_configuration.sh and in the Netkit scripts). - Fixed a bug inside vcrash and vclean (wrong signal spec when killing processes broke both vcrash and vclean). - Placed a workaround to avoid early hangs of the port-helper process when booting virtual machines. - Removed some minor warnings issued by vstart and vclean. - Fixed a small bug in the output of vlist. 2.2 Beta - Updated check_configuration.sh (there was no check for the correctness of the PATH environment variable). - Updated INSTALL file. 2.1 Beta - Fixed a bug that prevented correct recognition of running processes (caused vlist and related commands - vcrash, etc. - not to work). - Suppressed a warning popping out when killing a virtual machine having two (or more) network interfaces attached to the same virtual hub. - Fixed minor bug that caused reporting wrong console information when using vlist to get detailed info about a running virtual machine. 2.0 Beta - Netkit now consists of three separate pieces: a "core", a filesystem, and a kernel. Changelogs are maintained separately as well. - Netkit is now documented! The distribution includes a plethora of man pages which guide you throughout Netkit usage and setup. - Introducing a more robust `check_configuration.sh' script, replacing the old `config'. - Environment variable pointing to the Netkit directory has been renamed to NETKIT_HOME (VLAB_HOME is still supported). - Default configuration parameters for newly started virtual machines and, in general, for the Netkit environment, can now be easily altered by editing the netkit.conf file. - All Netkit commands can now limit their scope of action to virtual machines owned (=started) by the desired user. - All Netkit commands are now more robust and provide a clearer output. - Virtual machines can now be configured with network interfaces connected to a real network. This can be done automatically by using an option of the vstart command, which will take care of asking you for the root password when needed. - Network interfaces can now be attached to a running virtual machine ``on the fly'' by using the vconfig command. - The vstart command no longer requires the use of the (very very boring) --new option when starting new virtual machines. - Virtual hubs are automatically stopped when the last machine that uses them quits (no matter whether by graceful halt or by crash). - The vstart command allows the use of alternative UML kernels and filesystems for the virtual machines. - Better management of consoles has been introduced (virtual machine consoles can now be attached to different terminal emulators in a more flexible way). Also, each machine supports two consoles. - Vstart now allows to pass parameters directly to the virtual machine kernel command line. - The vlist command can now be used to ask for detailed information about a running virtual machine. - Vcrash supports different shutdown strategies (by management console or by process signaling). It also allows to remove a virtual machine filesystem upon its crash. - Vhalt now allows to remove a virtual machine filesystem after shutdown. - A new command (vclean) has been introduced to clean the host system from hanging processes and unused virtual hubs. It also allows to remove any configuration settings that have been altered to setup interfaces connected to a real network. - All the ltools now allow to perform operations on a subset of the virtual machines of a lab. This is useful, for example, when only few machines are to be restarted. - Lstart now allows to configure a different amount of emulated memory for each virtual machine. - Lstart now supports an option inside `lab.conf' which can be used to restrict the set of enabled virtual machines for a lab. - A new testing system for the labs has been introduced. It can be used to automatically perform arbitrary checks on a Netkit lab. This is especially useful to ease the distribution of a lab, because it provides with a one-shot solution to test that the lab works properly on a different machine or Netkit distribution. - Something else we may not have mentioned...... --------------------------------------------------------------------------------- F2.2 - Cleaned up /etc/hosts (it contained some wrong entries associating 127.0.0.1 with "pc1"). - Lab .startup files are taken into account even if they are not executable. This impacts the Netkit filesystem because it requires a change in the virtual hosts init scripts. - Boot time scripts processed the lab.conf file improperly. This caused warning messages when using the [mem] parameter or when defining interfaces for virtual hosts having particular names (e.g., containing a dash). - The "expect" package has been installed. F2.1 - Moved the /lib/tls/ directory away (UML does not support Thread Local Storage yet). Its presence caused some tools not to work (e.g., nslookup). - Foreign (i.e., user provided) Netkit kernel modules were properly recognized only if the kernel version matched 2.6.11.7. Fixed.

43. Overall contributions To Do • Interdomain topologydiscovery • Commercial relationships • Traffic engineering • Policy discovery • Stability • Emulation • complexity & effectivenessof algorithms • network measurements • validate other algorithms • describe the space of solutions • hierarchy • bounds on complexity • case study • game theory • efficiency • accuracy • locate stable states • system characterization • upgrade of networking tools • new features • new & improved labs • integration with other products

44. Thank you!