1 / 24

Design and Implementation of the REANNZ KAREN network Jörg Micheel

Design and Implementation of the REANNZ KAREN network Jörg Micheel. Outline for this talk. Network design goals for KAREN Layer 1+2+3 network architecture Network services and implementation International transit network design Network performance

taurus
Download Presentation

Design and Implementation of the REANNZ KAREN network Jörg Micheel

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Design and Implementation ofthe REANNZ KAREN networkJörg Micheel

  2. Outline for this talk • Network design goals for KAREN • Layer 1+2+3 network architecture • Network services and implementation • International transit network design • Network performance • Checklist for KAREN connectors (REANNZ members) • Summary and references

  3. MoRST/REANNZ/KAREN design goals • A high performance network for the NZ R&E community! • 10 Gbps capable backbone interconnecting all major cities in NZ • Access speeds at 1 Gbps and 10 Gbps • Unconstrained end-to-end performance at (multi-)Gigabits/second • Tailored on-demand performance for specific applications or experiments (bandwidth, delay, jitter) • International connectivity at 155 Mbps to AU, 622 Mbps to US • New services: multicast, IPv6, Jumbo frames (9000 Bytes MTU) • Virtual Private Network functionality for members • Telco-grade implementation and network management • Security, redundancy, high availability • Range of network measurement facilities (wire tap, NetFlow, SNMP data collection, active measurement) and development environment • Most importantly: stick to budget and timelines!!!

  4. L1/L2/L3 Network Design • L1 Network core as rings on TCL OPTera DWDM • L1 Dark fiber spur to neutral POP and AAP • L2 Nationwide network based on Extreme X450a and BD10K • L3 using Juniper M320 in Auckland and Wellington • Note: 10GigE WANPHY is 9.287 Gbps! • ANOPS management network based on TCL PIP service and CISCO 2801

  5. Extreme Networks Black Diamond – metro core switch • Black Diamond 10808 (BD10K) • 22 rack mount units • 1280 Gbps capacity (blocking) • Up to 48 10-Gigabit ports • Up to 480 10/100/1000 ports • Powerful VLAN, Virtual router Layer2 and Layer3 capabilities • Proprietary EAPS link-protection protocol provides continuity in case of fiber cut • L2/L3 Quality-of-Service • L2/L3 hardware filtering and priority • Jumbo frames at 9212 • High availability, hardware redundancy

  6. Extreme Networks Summit X450a – edge switch • X450a-24t with 24 ports 10/100/1000 copper, four combined SFP GigE ports • X450a-24x with 24 ports 1-GigE SFP, four combined 10/100/1000 ports • Optional dual 10-Gigabit Ethernet uplinks • 1 RU form factor • 160 Gigabits-per-second capacity • 65 million packets-per-second forwarding performance • Stacking capability with XOS 11.7 (April 2007) • Other features similar to Black Diamond series

  7. Juniper Networks M320 Multi-service Edge Router • 320 Gbps switching capacity • 8 FPC slots with 20 Gbps FD capacity • ½ rack size • 32 PICs per chassis • 10 GigE capable

  8. L2/L3 design • Connector joins KAREN via dark fiber • Switch access into one or more VLANs • BGP peering with core • L2 packet switched data nationally • L3 routing overseas

  9. KAREN Service Matrix

  10. Internet Exchange model (L2 switching + BGP Route Reflector) “Switch – don’t route” “Peer with two – route with many (others)”  scalable Internet Exchange model

  11. Challenges of a L2 network • Redundant links will be pruned (Spanning Tree, etc), creating a star topology • Only difference between L2 resilience protocols is speed • Issue: capacity not utilised • Issue: shortest path • Issue: protect all VLANs • Solution: VLANs EAST/WEST for public IP services • Solution: Extreme EAPS for protection

  12. KAREN Multicast – two options for connectors • Bootstrap as part of KAREN multicast cloud (quick start for small sites, no MSDP, but doesn’t scale) • Create your own multicast domain (requires MSDP, scales well)

  13. Multi-Protocol BGP and routing tables

  14. KAREN International • Separate to domestic KAREN, but co-joined • As a static 3-point transit network has to implement all services (IPv4/IPv6 uni- and multicast, jumbo frames) • Routing policy ensures traffic flows between NZ and overseas peers (but not between other peers) • Pacific Wave landing point in Seattle poised for peering expansion • Bulk of 9K routes from Internet2 ITN

  15. REANNZ POP

  16. KAREN performance tests • Network commissioning in October and November successfully demonstrated capacity, delay and jitter parameters • Bandwidth tests carried out as 1 Gbps VLANs POP-to-POP • All L2 components stressed at or near capacity limits (see next slide for example) • Delay and jitter tests carried out as RTT measurements using hardware loopbacks

  17. KAREN performance tests (as shown on WAND weathermap) See recorded animations at http://erg.cs.waikato.ac.nz/weathermap/ for other tests carried out during November and December.

  18. KAREN delay and jitter tests

  19. KAREN delay and jitter results

  20. KAREN Connectors 101 (and shopping list) • KAREN is a Tier 1 network – you need to behave like a Tier 2 – control your own routing (policy). • Consider the services you want: IPv4 unicast, multicast, IPv6, Jumbo • Are you a heavy hitter ? Thinking of 10 Gbps ?  Router that speaks BGP, holds 20K+ routes and does 1 Gbps • If you are a heavy hitter, you need VLAN support and 40K+ routes • If you want jumbo frames, you need VLAN support and 40K+ routes • If you want multicast, you need PIM-SM, preferably MSDP and M-BGP • If you want IPv6, you need M-BGP and space for even more routes

  21. KAREN Connectors to date • About a dozen connectors at around 15 sites, wide range of equipment • CISCO 6500 series edge routers • Allied Telesyn AT-9924Ts • Juniper M and J series – J6350 • Linux PC and Quagga Routing Suite • Check Point Firewall on Nokia platform

  22. 2007 outlook • Connectors, connectors, connectors! • Access solutions for schools and other non-BGP speakers • International IPv6 peering • Peering with US FedNets; China, Japan R&D networks • IPv6 DNS • Better solution to the 2/4/8 peering problem for complex sites • Pushing traffic around EAST/WEST for dual attached sites • L2 PIM-SM snooping (on top of IGMP snooping) • More security, core hardening • Stacking support in Napier and 10 Gbps services to Havelock North • Measurement infrastructure (active and passive) – capability build fund

  23. Summary • KAREN creates a fast lane for the R&E community, inside New Zealand and with overseas R&D networks. • It offers a range of new services previously unavailable or inaccessible in New Zealand, such as multicast, IPv6. • It offers a test bed for novel tools and applications. • Performance is stunning – go and use it!

  24. Acknowledgements and references • REANNZ KAREN http://www.karen.net.nz/ • WAND weather map http://erg.cs.waikato.ac.nz/weathermap/ • A cast of dozens of hands at TelstraClear and JazzTech • Questions: please contact myself or David Brownlie and Clayton Ejiofor at REANNZ. Thank you!

More Related