1 / 41

NANOG24 Miami 12th February 2002

Inter-domain Multicast in European Research Networking: TEN-155 Operational Experience and Deployment on GÉANT. NANOG24 Miami 12th February 2002. Agnes Pouélé, DANTE Ltd. Network Engineer Jan Novak, Cisco Systems Inc. Network Consulting Engineer. 1. Agenda. DANTE

kempe
Download Presentation

NANOG24 Miami 12th February 2002

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Inter-domain Multicast in European Research Networking:TEN-155 Operational Experience and Deployment on GÉANT NANOG24 Miami 12th February 2002 Agnes Pouélé, DANTE Ltd. Network Engineer Jan Novak, Cisco Systems Inc. Network Consulting Engineer 1

  2. Agenda • DANTE • TEN-155 Operational Experience • Evolution of the TEN-155 multicast topology from 1998 to 2000 • Operational Experience • GÉANT deployment • Overview of GÉANT Network • GÉANT’s Multicast design • GÉANT Multicast and Unicast Coverage • GÉANT Multicast Service and Monitoring • Conclusion 2

  3. DANTE, TEN-155, GÉANT • DANTE • DANTE is a not-for-profit company set up in 1993 by European National Research Network organizations. • TEN-155 • was an ATM based network built initially on OC3 links and then upgraded in 2000 • GÉANT • successor of TEN-155 • launched in December 2001 • 10 Gbps Pan-European Network 3

  4. Part I TEN-155 Operational Experience 4

  5. Starting point MBONE TUNNELED TOPOLOGY Typical mess of DVMRP tunnels on Sun WSs with usual tunnel routing problems. 5

  6. 1999: PIM-SM and DVMRP set-up PIM-SM domain OSLO IETF transmitted over both native STM-1 ATM based network and DVMRP tunnelled infrastructure 6

  7. Based on this first set-up and successful operation, we concluded to enable multicast on all production routers in TEN-155. Multicast code stable, CPU usage ok, M-BGP ok PIM-SM “only” for production service PIM-SM to DVMRP border works, but not possible to operate routinely (NOC) From 1999 to 2000 each country was migrated from the DVMRP cloud to a interconnection with TEN-155 using PIM-SM/MBGP/MSDP Multicast deployment in TEN-155 7

  8. Belgium Luxembourg UUNET Poland ABILENE Czech Rep Germany United Kingdom Israel AS8933 OSPF + internal MBGP Greece Slovenia Hungary France Nordics Netherlands AT& T Spain Italy Portugal Switzerland 2000: TEN-155 final topology e x t e r n a l NL SE DE UK e x t e r n a l e x t e r n a l US AT FR CH tunnel IT dedicated mcast B G P unicast/mcast line 8

  9. Parameters to be considered for the exploitation of Multicast CPU - parameters about 1600 forwarding (mroute) entries max 8 outgoing interfaces (average 2-3) max 20 Mbit/s of data forwarded by one router resulting in 5-10% of CPU increase (mainly PIM) TEN-155 Operational ExperiencePerformance Impact 9

  10. Parameters to be considered for the exploitation of Multicast Memory about 3000 SA messages in the cache about 1600 forwarding entries (mroute) about 10 000 routes in the MBGP table These parameters didn’t represent a significant memory usage TEN-155 Operational ExperiencePerformance Impact 10

  11. Default peer in redundant topology “Redundant” mesh-groups Group A Group C Group B Usage of MSDP – new component – new storms MSDP peers default peer Impact of a MSDP storm on the CPU load 11

  12. Usage of MSDP – early implementation problems • Cisco IOS 12.0.6S and lower • ghost SA entries in the MSDP cache • SA messages recreated by the incoming (S,G) joins • FIXED • Origination of SA messages only when source registered to the originating RP • “ip msdp redistribute” command – without arguments • re-originated all known SAs • caused huge increase of the SA counts worldwide • FIXED 12

  13. WS - DANTE DE.TEN-155 router TEN-155 MSDP monitoring • Monitoring set-up MSDP peering • Software – adapted C++ MSDP • implementation of Steve Rubin • MSDP usage monitoring • number of RPs, average 90 (40 EU) • number of groups • number of pairs source, group (S,G) • number of SA messages per minute 13

  14. TEN-155 MSDP monitoring number of RPs Number of RPs announced to TEN-155 before and after the loss of US connectivity 14

  15. WS - DE PoP DE.TEN-155 router TEN-155 MBGP monitoring • Monitoring set-up MBGP peering • Software - Merit’s MRTD - • modified SAFI definition for MBGP • MBGP monitoring • stability/updates • number routes, max about 9000 (760 EU) • number of ASNs, average 240 (80 EU) 15

  16. TEN-155 MBGP monitoring stability/updates • In blue, • the number of routes originated from one AS • In green, • the number of updates originated by the same AS 16

  17. TEN-155 Multicast Monitoring tools • http://www.dante.net/pubs/dip/40/40.html • http://www.dante.net/pubs/dip/41/41.html • http://www.dante.net/mbone/ • http://sigma.dante.org.uk/stats/mrtg/msdp/data/ • http://www.dante.net/mbone/mbgp • The graph values from mbgp and msdp monitoring are historical values. 17

  18. TEN-155 Operational ExperienceConclusion • Move to native and SM multicast in national networks (NRENs) • Unicast and multicast non congruent • MSDP peer doesn’t need to be RP. • Concept of two BGP tables and “multi-protocol” RPF check often still misunderstood. • Inter-domain Multicast debugging => Almost impossible to fix problems in just one week 18

  19. PART IINEXT GENERATIONGÉANT DEPLOYMENT 19

  20. www.dante.net/nep/GÉANT-MULTICAST/map.html 20

  21. GÉANT Services • GÉANT Standard IP Service • IP traffic from NREN to NREN and Research peerings. • MulticastService (rolling out now) • Replacement of TEN-155 Managed Bandwidth Service • GÉANT Premium IP Service • Layer-2 VPNs (forthcoming) • Upcoming • Security and Dos attack detection, IPV6 21

  22. GÉANT Unicast Customers 27 countries in Europe GÉANT Multicast Customers 24 countries in Europe GÉANT Unicast and Multicast research and commercial peerings Abilene, Canarie and ESnet via GTREN Infonet, UUNET GÉANT’s Customers and Other Peerings 22

  23. GÉANT Multicast Customers www.dante.net/nep/GÉANT-MULTICAST/map.html 23

  24. Current design built on The experience gained from TEN-155 Guidelines from multicast experts Juniper Laboratory tests GÉANT Multicast services Multicast transit domain NRENs to NRENs Multicast transit domain NRENs to other PEERS Beacon monitored backbone GÉANT Multicast routing policy at: http://www.dante.net/nep/GEANT-MULTICAST/routing-policy.html GÉANT Multicast Implementation 24

  25. Group 233.0.1.1 Source 10.0.1.2 I-MBGP full mesh I-MSDP full mesh Group 233.0.3.1 Source 10.3.30.2 TEST-BED LAB 26/27th Nov 2001 • Junos 5.0R3.3 HostX Group 233.1.10.1 Network 10.1/16 -AS 101 Source 10.1.10.2 Sulfur • mesh group Iridium Mangan ese M-BGP peering Network 10.0/16 -AS 100 Gallium Helium MSDP peering HostZ Xenon Tin HostY Network 10.2/16 –AS 102 Network 10.3/16 –AS 103 Group 233.2.20.1 Group 233.3.30.1 Source 10.2.20.2 Source 10.3.30.2 25

  26. GÉANT single PIM-SMv2 Version 2 , enabled on all interfaces. Three Rendez-vous Points with private anycast address. backup for internal sources and receivers. Private Anycast address (filtered out) closest RP based on the OSPF cost All other interconnected administrative domain have to be PIM-SM v2 enabled with their dedicated RP. PIM SMv2 GÉANT domain 26

  27. 9 se pl uk 40 630 40 ie 35 7 nl 640 5 40 10 be 640 40 lu 640 10 de 10 cz fr 159 40 40 20 es 7 sk 7 35 20 7 hu it ch 40 170 10 Rendez-vous Point at si 160 160 Multicast access gr PHYSICAL TOPOLOGY: RP Eenet Litnet Janet Nordunet NY4-1 Latnet IUCC Posnan Heanet Surfnet Belnet PoP DFN NY4-2 Restena Cesnet Infonet Renater Rediris Sanet FCCN Hungarnet RoEduNet INFN Arnes Switch&Cern STM64/OC192 STM16/OC48 Carnet STM4/OC12 Aconet Grnet STM1/OC3 Cynet Unicom-b www.dante.net/nep/GÉANT-MULTICAST/map.html 27

  28. MBGP Separate multicast routing table (inet.2) Currently congruent BGP and MBGP topology in Europe iMSDP MSDP is fully meshed between 19 PoPs Use of mesh group i-MSDP Peering with loopback addresses (Not the anycast address !!) eMSDP NREN <---> GÉANT Access Router GÉANT Design: MBGP and MSDP 28

  29. MSDP peering Logical view Janet Ny4-1 Iucc GTREN GRnet Eenet Cynet Nordunet Unicom-b External MSDP peering Litnet Renater uk Latnet gr se fr Switch&Cern iMSDP Mesh Group iMBGP Full Mesh Abilene de2 ch NRN ----- GÉANT router Infonet Rediris es de1 DFN be NY4-2 Belnet nl RP it Surfnet INFN pt si Arnes at lu FCCN Carnet hu ie Aconet Restena sk pl cz RoEdunet Heanet Hungarnet Posnan Sanet Cesnet www.dante.net/nep/GEANT-MULTICAST/map.html 29

  30. A list of filtered SA is defined at: http://www.dante.net/nep/GEANT-MULTICAST/deployment-msdp.html We filter the recommended list. We authorise 239.194.0.0/16 from the IPV4 Organisation Local Scope through GÉANT. MSDP SA Filtering 30

  31. GÉANT Multicast and Unicast Coverage CAnet Esnet Abilene GTREN RESEARCH Peerings www.dante.net/nep/gtren.html DFN GÉANT AS20965 JANET DE UK NREN4 NREN2 ... ... Third party provider European Distributed Access STM-16 Commodity Internet Access STM-4 31

  32. Access to the service Via the primary access to GÉANT Via a GRE tunnel (currently nobody) Support of PIM-SM v2 only Operational procedures (rolling out now) Goal: same level of service as Unicast. Troubleshooting Extension of the trouble ticket systems to multicast incidents GÉANT Multicast Service 32

  33. Beacon Tool initially developed by Kai Chen from NLANR dast.nlanr.net/projects/Beacon/ Relies on a number of Agents spread over the network which simultaneously send and receive multicast packets carrying a packet sequence number and a timestamp. Communicates with a central server which displays matrices of Agents via web pages. GÉANT multicast monitoring 33

  34. Recommended by TF-NGN group (www.dante.net/tf-ngn) To use from day 1 monitoring of multicast inside and outside of GÉANT. Server code enhanced with historical functionality http://noc.man.poznan.pl/noc/index/strony (Menu item “Applications”) romradz@man.poznan.pl Multicast Beacon Agent written in C http://www.cesnet.cz/tf-ngn/multicast/ GÉANT multicast monitoring 34

  35. Beacon’s matrices One for the internal sources of GÉANT Each GÉANT POP has a beacon agent installed One for the external sources of GÉANT http://beaconserver.geant.net:19999/ We have assigned two multicast groups from GLOP range [RFC2770] for each matrix Parameters monitored Loss Delay Jitter GÉANT multicast monitoring 35

  36. Beacon internal/external matrix www.dante.net/nep/GÉANT-MULTICAST/deployment-beacon.html 36

  37. Per group monitoring from TEN-155 Based on the IETF IP-MROUTE MIB Shows traffic per multicast group per interface Under installation on ws1.se.geant.net MSDP and MBGP monitoring tools Not yet available Other Monitoring Tools 37

  38. Conclusion From TEN-155 to GÉANT Unicast and multicast moves to a congruent topologyacross Europe and towards research peerings. Deployment status Links http://www.dante.net/nep/GEANT-MULTICAST/ http://beaconserver.geant.net:19999/ CONCLUSION 38

  39. THANKS We would like to thank and acknowledge the help of the people who worked and are working with us on these projects, mainly from all EU and US research networks 39

  40. Questions ? 40

  41. SA Filter list 41

More Related