Taiwan Advanced Research and Education Network (TWAREN) - Current status & Future Plan

1. Taiwan Advanced Research and Education Network (TWAREN) - Current status & Future Plan Dr. Te-Lung Liu Researcher National Center for High-Performance Computing tlliu@nchc.narl.org.tw

2. Outline • TWAREN Network Overview • Development and Research Technologies

3. TWAREN Network Overview • Development and Research Technologies

4. TWAREN TaiWan Advanced Researchand Education Network

5. What is TWAREN • A physical network serves multiple purposes and logical networks • TANet, connects to commodity Internet • TWAREN research network • experiment, testbed, special research • Provisioning services on multiple layers • L1 lightpaths • L2 VLAN • L3 IP • has been successfully migrated from old backbone in Oct 2006

6. TWAREN Architecture • 4 core nodes • 20G backbone • 12 GigaPops • Connects HPC resources in North and South Taiwan

7. Goals of TWAREN • TWAREN is part of “Challenge 2008”, a comprehensive six-year national development plan formulated by the government • Build a highly reliable, stable and flexible R&E network for academic and research community in TW • Provide advanced network services to satisfy the needs of academia field in TW. • Increase the International and domestic collaboration • Future infrastructure drives today’s research agenda

8. TWAREN GigaPoPs

9. TWAREN Services • Broadband Connection Service • International Research Network Transit (Internet2) • Measurement / Network Management • Multimedia / Multicast • Lightpath provisioning • Virtual Private Network(VPN) • Native IPv6 Service • Internet access • MCU • Proxy Server • SourceForge • File Download Center • Consultation • Applications support

10. TWAREN Achivements • High reliability & availability(99.9%  99.99%)‏ • fault tolerance • automatic protection if possible • automatic failure detection and locating • Better performance: minimum number of routers between GigaPoPs • Flexible: can be easily and quickly to set up a logical network per user’s request • People skills: Optical network OAM

11. ASCC STM-64 NTU NIU STM-16 NCU NDHU TP TC HC NCTU NCHU TN NTHU ONS15600 ONS15454 NSYSU CCU NCKU Optical Backbone

12. NTU NIU ASCC NCCU NDHU NCU NCNU NHLTC Taipei Hsinchu NCTU Taichung Tainan NCHU NTTU NTHU 7609 ONS15600 STM64 STM16 6509 ONS15454 NCKU CCU NSYSU 10GE GE GSR 3750 Interconnecting with L2/L3 devices

13. Protection Mechanism • Circuit break:2 levels of protection • By carriers: SDH protected • By architecture: • Link b/w core nodes: VLAN are reconfigured with rapid spanning-tree protocol. (5s)‏ • Link b/w GigaPOP and core node: the backup SNCP lightpaths are configured for automatic fail-over. (50ms)‏

14. Protection Mechanism • Equipment protection • Core node failure: Manually configure emergency lightpaths to re-route traffic from affected GigaPoPs to another core node. Emergency lightpaths need to be designed and documented. • GigaPoP failure: Spare line cards

15. NIU ASCC NCCU NDHU NCU NCNU NHLTC Taipei Hsinchu NCTU Taichung Tainan NCHU NTTU NTHU 7609 ONS15600 STM64 STM16 6509 ONS15454 NCKU CCU NSYSU 10GE GE GSR 3750 Normal Traffic Flows NTU

16. NIU ASCC NCCU NDHU NCU NCNU NHLTC Taipei Hsinchu NCTU Taichung Tainan NCHU NTTU NTHU 7609 ONS15600 STM64 STM16 6509 ONS15454 NCKU CCU NSYSU 10GE GE GSR 3750 In case of circuit break... NTU

17. NTU NIU ASCC NCCU NDHU NCU NCNU NHLTC Taipei Hsinchu NCTU Taichung Tainan NCHU NTTU NTHU 7609 ONS15600 STM64 STM16 6509 ONS15454 NCKU CCU NSYSU 10GE GE GSR 3750 In case of core node failure...

18. TWAREN NOC • NOC (Network Operation Center) • Located at NCHC southern business unit in Tainan Science Park • Goals: To ensure the 7x24 network operation • Major works: • Providing 7x24 network maintenance and operation • Enhance the security capacity • Provide network service • Peering • Light path provision • Network architecture design TWAREN NOC

19. MOEcc6509 NDHU6509 NCCU6509 NHLUE6509 One Subnet L2 VLAN NTU6509 NCU6509 NCHU6509 TP7609C L2 Switch HC7609 TANet VLAN TC7609 TC7609C L2 Switch HC7609C L2 Switch NCTU6509 TN7609C L2 Switch NTHU6509 NSYSU6509 CCU6509 NTTU6509 NCKU6509 TN7609P TANet VPN

20. TANet (MOEcc6509) TAIWANLight TWGATE Internet ASCC APAN ISP Peering TP12816P TP12816R NDHU7609P iBGP RR NIU7609P ASCC7609P NCTU7609P NTU7609P NCU7609P TP7609C Switch NCHU7609P Research VLAN HC12816P TC12816P TC7609C Switch HC7609C Switch HC12816R TC12816R TN7609C Switch HC7609P NCNU7609P NTHU7609P NSYSU7609P CCU7609P TN7609P NCKU7609P TN12816R TN12816P iBGP RR TAIWANLight TAIWANLight ISP Peering TWAREN ResearchVPN

21. VPN Services • Multipoint-to-Multipoint Layer2 VPN (VPLS) • Multiple VPNs over single architecture • Cross-area campuses and offices can be connected within single administrative domain • Provide dynamic creation of VPNs for National-wide integrated projects • User-based SSL VPN Access • Access to different VPN according to login name and password authentication • Researchers and Professors could access their own research resources from home or outside

22. VPLSArchitecture

23. User-Based SSL VPN Access Core node@ HsinChu SSL VPN Org 1 Users TWAREN VPLS Backbone Org 2 Org 3 Web Browser SSL VPN 。。。 Core node@ Tainan Org n

24. TWAREN’s International Connections • Pacific Crossing to USA’s west coast upgraded to 5 Gb/s • Connections between LA, Palo Alto, Chicago, and New York are 2.5 Gb/s • Connects to the rest of the world via the U.S.’s Abilene Network • Connection expanded to Europe in 2006 (IEEAF donated 622 Mbps of bandwidth/fiber optic cable)‏

25. ASCC-15454 NCTU-15454 NCU-15454 NTU-15454 NTHU-15454 Chicago 15454 TP-15454 HC-15454 HC-15600 TP-15600 Palo Alto 15454 NY 15454 NIU-15454 TN-15454 TN-15600 TC-15454 NDHU-15454 LA 15454 NCSYSU-15454 NCKU-15454 CCU-15454 NCHU-15454 TWAREN Optical Network TAIWANLight CombinedTWAREN/TAIWANLightLambda Testbed

26. TWAREN’s International Peerings • TWAREN made peerings with international NRENs at Los Angeles, Chicago, New York and Seattle (through Pacific Wave).

27. TWAREN’s Direct Peerings Coverage • TWAREN's direct peering covers most area in America, Asia, Australia and New Zealand, and will soon be expanded to Europe.

28. TWAREN/TAIWANLight and GLIF TWAREN is a member of GLIF (Global Lambda Integrated Facility)‏ TAIWANLight is an official optical exchange - GOLE (GLIF Open Lightpath Exchange)

29. TWAREN Network Overview • Development andResearch Technologies

30. Future Internet Testbed @ Taiwan

31. Future Internet There are many serious limitations in current Internet. Scalability Security QoS Virtualization Future Internet is a summarizing term for worldwide research activities dedicated to the further development of the original Internet. (From Wiki)

32. Future Internet Testbed For innovations and researches in Future Internet, the testbed requires some advanced concepts: Programmability Virtualization End-to-end slice

33. OpenFlow Make deployed networks programmable Makes innovation easier No more special purpose test-beds Validate your experiments on production network at full line speed

34. TWAREN OpenFlow Testbed in 2010 iCAIR NCHC Capsulator TWAREN L3 Network NOX OpenFlow Network @KUAS OpenFlow Network @NCKU Capsulator Capsulator OpenFlow Switch • NCKU and KUAS are pilot universities that connected with the Testbed • The OpenFlow Testbed is extended to iGENI@iCAIR • Capsulator (Ethernet-in-IP tunnel) is used to emulate pure L2 network for OpenFlow 34

35. TWAREN OpenFlow Testbed in 2011 NCU NCHC lightpath TWAREN VPLS Capsulator NCKU CHT-TL KUAS NTUST Capsulator OpenFlow Switch OpenFlow Switch OpenFlow Switch OpenFlow Switch OpenFlow Switch OpenFlow Switch OpenFlow Switch • NTUST, NCU and CHT-TL joined the Testbed. • For TWAREN connectors (NCKU, KUAS and NCU), a dedicated VPLS VLAN is allocated for better transmission performance. 35

36. Emulab/ProtoGENI Testbed • TWISC (Taiwan Information Security Research and Education Center) operats 206 nodes of Emulab Testbed in Taiwan. • Third largest Emulab in the world • Testbed@TWISC is operated by NCKU team and co-located in NCHC • A portion of the testbed is planned to try ProtoGENI test with University of Utah. • A lightpath is provisioned between NCHC and iCAIR shared by both OpenFlow and Emulab/ProtoGENI 36

37. Lightpath and VLAN setup Lab Vlan 2782 iCAIR Trunk Trunk port Vlan 462 Vlan 2782 7609P@HC Vlan 462 Vlan 2782 Vlan 462 Vlan 2782 7609P@TN Emulab/ProtoGENI – Vlan 462 Vlan 462 Vlan 1548 Vlan 1548 Vlan 1555 iCAIR OF (with NCKU) – Vlan 2782 Vlan 462 Vlan 1548 Vlan 1555 Vlan 2782 NCKU 7609V NCKU 7609V NCKU EE NCHC Emulab @NCHC OF sw A OF sw B Vlan 462 Vlan 1548 Emulab/ProtoGENI – Vlan 462 NCKU OF (with iCAIR) – Vlan 1548 37

38. iGENI - Taiwan Integrated Research Network 38

39. Multi-Domain OpenFlow Management • Each network domain has its own OF Controller • Each Controller manages topology and flow provisioning inside the domain • Inter-domain flow could be made by connecting partial flows provisioned by controllers of each cloud • Lack of global view for inter-domain flows • No loops allowed for inter-domain topology • Difficult to support QoS or SLA functions across domains • Inter-domain topology auto-discovery is required for multi-domain management 39

40. Inter-Domain Topology Discovery (I) • OpenFlow Controller just only knows its directly connected switches. • ENVI is a useful GUI tool to show OpenFlow topology under single controller. OFA OFB OFC OFD Controller1 Controller2 Domain UI Topology of Domain1 Topology of Domain2 OFA OFB OFC OFD 40

41. We add additional contents in LLDP packet to let Controllers have its neighbors’ connectivity details. ENVI is also modified to show the whole topology. Inter-Domain Topology Discovery (II) OFA OFB OFC OFD Controller1 Controller2 Domain UI Topology of Domain1 & 2 OFA OFB OFC OFD 41

42. Results Multi-Domain Network Topology shown in GUI Physical OpenFlow Network Topology 42

43. GLIF & SC11 Demo Joint Demo among NCHC/TW, iCair/US, and CRC/Canada

44. Information Security Activity Detection over High-Speed Backbone

45. Security Detection over High-Speed Backbone Normally, we don’t install IDS/IDP in backbone for performance issue. IDS/IDP are placed at user’s local sites Backbone traffic is hard to mirroring due to its large amount and high-speed It’s impossible to do packet analysis Packet header analysis is available with Netflow/sFlow Information Security Activity Detection over High-Speed Backbone Integrate fast packet header analysis with attack information from user’s local site

46. System Architecture Invasion and attack info from user’s local sites Netflow Data from Backbone/User Routers Users’ IDS/IDP Backbone’s Netflow data Users’ HoneyPot Security Detection @Backbone Users’ Log analyzer Users’ Netflow data Collect Search Orientation Trace-back Block Notification Notify User with Suspicious Activities Backbone network, peering partner, User network

47. Design Concepts Distributed Computing For monitoring netflow data in real-time Fast Search Effective Tree-Searching algorithm Expandable Simply add more machines when larger data analysis is required Remote Backup Separate different computing nodes in order to provide robust analysis service Single Portal All input can be submit to single portal with Global Server Load-Balancing technology Cooperate with Researchers/Developers Will design an open API for developers to contribute their own ideas

48. Design Blocks Router1 Router2 Router3 RouterN Netflow packet Blacklist Search Tree Distributor 1 Distributor 2 Netflow packet Netflow packet Filter 1 Filter 2 Filter 3 Filter N Matched Netflow raw Matched Netflow raw Analyzer 1 P3333 Analyzer 2 P4444 Analyzer 3 P5555 Analyzer 1 Analyzer 2 Analyzer 3 Analyzer N Update Search Tree result result result Controller 1 Update Blacklist Controller 2 Blacklist IDS/IDP Honey... Syslog

49. Numerical Results of Tree Creation

50. Numerical Results of Real-time Matching