Structure of the Internet

1. Structure of the Internet • Update for 1st H/Wk • We will start lab next week • Paper presentation at the end of the session • Next Class MPLS

2. Other Attempts at the problem • IDPR (and not IDRP) • Link state! • Prunes the ASes and keeps only the transit ones • Assumes information is very static • Topology databases need not be synchronized • Use source routing in order to avoid loops • Establish a “connection” through the ASes in the Path • Between Border routers • And repair it when it breaks • More complex than OSPF and BGP-4 together • Did not go anywhere

3. How is the Internet in reality? • Provider relationships • PoPs and Internet structure? • Provider internal structure • Traffic Egnineering • BGP monitoring tools • Examples

4. AS relationships • The internet is a BIG AS graph • How does it look? • Ideally, we would like to see a nice hierarchy: customer, local ISP, regional ISP, national ISP, transit ISP • Not really… • Locality is determined based on cost • Reliability requires multiple redundant paths • To whom I talk can have important business implications

5. Types of relationships • Customer – Provider • Customer pays money for the service • Customer is usually smaller than the provider • Paid Transit • ISP A and B pay ISP C to connect them • Transit ISPs have big global networks (tier-1) • Peering • Two ISPs exchange routes that they originate into the internet • I.e. their own customer routes • No upstream routes • Nobody pays

6. Things are complicated • Network connectivity does not imply reachability • Policies may prevent it, for example a multi-homed customer can not transit traffic between its providers • Need to know the relationships between ASes and this is not easy: • Policies are not widely advertised • Treated as sensitive business information

7. Tiers • Large transit ISPs are Tier-1 • (MCI, AT&T) • They have no parent provider • Smaller national/regional ISPs are Tier-2 • GEANT • And small local ISPs are Tier-3 or edge • ForthNet • It is possible to find more structure in Tier-1 • See “characterizing the internet hierarchy from multiple vantage points” • Some tier-1 form the “dense core” of the Internet • Almost fully connected graph, tier-0 • Then tier-1 and tier-2, less connected large ISPs • Then small ISPs and customers • In 2001, 20 ISPs in the dense core

8. ISPs need to talk to each other • Depends on the relationship • Customer provider over a single link • Transit and peering? • Do it in Internet Exchanges • Also known as Network Access Points (NAPs) and Points of Presence (PoPs) • No need for n^2 connections • Exchange provides a stable environment for peering • Backup power, administration etc • Providers need to “co-locate” in the exchange • Exchanges are not free • Although peering is • Can always have private peerings between two ISPs

9. Exchange architectures • Centralized • A single or multiple routers • Router may have to enforce policies, not too good • Switched • Just connectivity, BGP enforces the policies • Need to co-locate • More expensive • Co-location costs and cost to send traffic to the exchange • Distributed • No need to co-locate • Not so stable as the centrally administered exchange

10. Peering Costs • Peering • How to share the cost of an end-to-end path • Cost of sending a packet is almost 0 • Try to split the cost down the middle between the sender and the receiver • Zero cost peering • Slowly emerging paid peering

11. Peering economics • When is it better to peer? • How much traffic I will be able to send through the peering • So I will not pay for it anymore? • Hard to measure how must traffic goes “behind” certain peers • How much will I have to pay for the exchange peering • Transit costs, exchange costs, operational costs

12. How to charge? • Charging models in customer-provider • Say I have a OC-12 (622 Mbit/sec) connection • Pay flat rate for the whole thing • Expensive probably • Pay for a fraction of it (say 200 Mbit/sec only) • Can not send more • Burstable fractional • Pay for a fraction but I can send more • Extra traffic charged per Mbyte • 95% charging • Drop the 5% highest samples and use the next one to charge • For the whole month! • How often do I sample the traffic? Usually 5 min… • Volume based charging • And flat rate (DSL style)

13. Structure of provider networks • Three levels (example) • Aggregation • Distribution • Core • Make sure IGP scales • Do not send it full BGP routes • Neither customer prefixes • Aggregation and distribution may not run iBGP • Core has to run iBGP • In transit ISPs core carries full BGP routes • In edge ISPs core may not have to run iBGP

14. Routing policy best practices • Do not re-advertise to provider B routes you learn from provider A • Customers should not allow transit • Do not advertise internal networks • Do not advertise prefixes that are aggregated • If you have a single provider no need for full routes • Always check routes you get for bogons • Limit the maximum number of routes you receive from other so that their errors do not kill you

15. Multi-homing • Two types • Provider assigned prefix • Secondary provider has to agree to advertise it • Provider independent prefix • Both providers will advertise it • But connectivity is only part of the problem • How to I use this multi-homing effectively? • How do I decide where I send traffic? • How can I control how I receive traffic?

16. Traffic Engineering and BGP • BGP conveys only connectivity information • Can not tell me which is the best/cheapest/least load path to use • I have minimum influence on what paths are used to reach me from other providers • There are some hacks to do something about it • AS prepending: make some paths I export longer so they are not used too much • Selectively advertise my external networks • Breaks aggregation • Use help from my providers • They may advertise communities that allow me to have little bit of control on the incoming path • By telling provider where to advertise my paths • These are not real solutions • Spawned a market for route analytics • But these only address my outbound traffic

17. Traffic Engineering inside the AS • Need to be able to control how transit and customer traffic flows in my network • It was believed that it is necessary to have circuit based transit to achieve this • ATM • Now MPLS • But IGP may be sufficient if I set the weights in a smart way • More to come…

18. Hot Potato Routing • IGP cost to reach a BGP next-hop can make all the difference • May affect a lot of traffic and cause instabilities • And cause BGP forwarding loops • BGP routers compute their paths on a timer, in between route computations there may be inconsistencies • IGP cost is low in the BGP path selection process • Paths have to be otherwise the same • Common in tier-1 providers • Also rule for preferring eBGP over iBGP can result in asymmetric paths

19. BGP Tools • WHOIS • Registration information for AS • Some examples, show the community stuff too • Looking glass • A provider opens ups its routing tables • I can see how my routes look from there • RADB • Route policy registry, some providers do not accept announcements that do not have correct entries there • BGP reports for scaling, CIDR etc • http://bgp.potaroo.net • http:/www.cidr-report.org

20. General Tools • Ping • Trace route