460 likes | 613 Views
Network Architectures. OARTech Paul Schopis October 13, 2005. Topics. TFN/OARnet background General MPLS Description What problem are we trying to solve anyway? Early Experiments at ITEC TFN implementation. OARnet Background. Founded in 1987 as part of the Ohio Supercomputing Center
E N D
Network Architectures OARTech Paul Schopis October 13, 2005
Topics • TFN/OARnet background • General MPLS Description • What problem are we trying to solve anyway? • Early Experiments at ITEC • TFN implementation
OARnet Background • Founded in 1987 as part of the Ohio Supercomputing Center • 90+ higher ed member institutions • Board of Regents funding • OSTEER advisory council • Internet2 GigaPOP
Third Frontier Network • Phase 1: replace backbone with dark fiber • Phase 2: connect 17 universities to network with dark fiber or gig circuits • Phase 3: connect other universities and colleges • Phase 4: connect other partners
Dark Fiber Acquisition • RFP issued during Summer of 2002 • Dark fiber was strongly preferred, but leased services considered • Vendors who bid dark fiber were required to offer a minimum of a single pair of fiber over their network
Dark Fiber Acquisition • Determined that leased lambdas were too expensive and not widely available • Selected a bid from Spectrum Networks for single pair of fibers • American Electric Power (AEP) • Williams Communications (Wiltel) • American Fiber Systems
Spectrum • We had various responses • The providers in the Spectrum offer bid individually • No price increase for using Spectrum as integrator • SBC and others reported no bid bit desired to bid on future last mile • inter-lata issue
Dark Fiber Acquisition • $4.6 M for 20 year IRUs • $342K/yr for maintenance • 1600+ route miles • Truewave, SMF-28, LEAF or Terra Light Fiber • Aerial and buried
TFN Financing • $21M investment • Financing from Ohio State University • Loan for fiber ($7M) • Short-term financing ($2M) • Financing from state capital budget ($8.5M) • Equipment • Last mile to 17 institutions
Community • We desired to make this a true community owned network • Committees with schools participating in decisions and recommendations
Equipment • Cisco 15454 integrated solution (DWDM) • all of the amps, mux/demux etc. integrated • Multi Service Transport Platform (MSTP) • ITU G.709 compliant • Cisco routers (GSR 12000) and switches • Juniper M7i routers
Last Mile • RFP issued in Dec 2003 for last-mile connectivity to all higher education and K-12 sites • OC3, gig circuits and10 gig circuits • We did make contact with local fiber providers on backbone bid ex. Buckeye Telesys
General MPLS Description Packet have a 20 bit label that routes it along a “Label Switched Path”. Values range from 0 to 1,048,575. 0 through 15 are reserved for special uses. Some label ranges have special meanings for specific vendors.
General MPLS Description • 0 IPv4 Explicit Null Label - No label stacking, must POP label • 1 Router Alert Label - delivered to local router for local processing • 2 IPv6 Explicit Null Label - Same rule as IPv4 except forwarded to IPv6 routing instance. • 3 Implicit Null Label - Control protocol (LDP or RSVP) request for down stream router to POP Label
General MPLS Description Choosing the next hop can be thought of as the composition of two functions. The first function partitions the entire set of possible packets into a set of"Forwarding Equivalence Classes (FECs)". The second maps each FEC to a next hop. In many ways an IP prefix is a FEC IP routing protocols are the mechanisms to map IP FECs to a next hop.
General MPLS Description What are the advantages of MPLS?
General MPLS Description MPLS forwarding can be done by switches which are capable of doing label lookup and replacement, but are either not capable of analyzing the network layer headers, or are not capable of analyzing the network layer headers at adequate speed.
General MPLS Description Since a packet is assigned to a FEC when it enters the network,the ingress router may use, in determining the assignment, any information it has about the packet, even if that information cannot be gleaned from the network layer header. For example,packets arriving on different ports may be assigned to different FECs. Conventional forwarding, on the other hand,can only consider information which travels with the packet in the packet header.
General MPLS Description A packet that enters the network at a particular router can be labeled differently than the same packet entering the network at a different router, and as a result forwarding decisions that depend on the ingress router can be easily made. This cannot be done with conventional forwarding, since the identity of a packet's ingress router does not travel with the packet.
General MPLS Description Sometimes it is desirable to force a packet to follow a particular route which is explicitly chosen at or before the time the packet enters the network, rather than being chosen by the normal dynamic routing algorithm as the packet travels through the network. This may be done as a matter of policy,or to support traffic engineering. In conventional forwarding,this requires the packet to carry an encoding of its route along with it ("source routing"). In MPLS, a label can be used to represent the route, so that the identity of the explicit route need not be carried with the packet.
General MPLS Description Some routers analyze a packet's network layer header not merely to choose the packet's next hop, but also to determine a packet's"precedence" or "class of service". They may then apply different discard thresholds or scheduling disciplines to different packets.MPLS allows (but does not require) the precedence or class of service to be fully or partially inferred from the label. In this case, one may say that the label represents the combination of a FEC and a precedence or class of service.
The Problem • Goal create an Abilene Premium Service • Need to create “Virtual Wire” ( Smells a lot like a light path) • Need predictable bandwidth • Need to meet DiffServ EF requirements • Need to be able to signal request for resources • Needed admission control
The Solution • DiffServ Code Point • Queuing mechanisms High Priority • Policy on edge to mark and forward via high priority queue • Admission control for LSP (MPLS Tunnels) via marked packets that conformed to requirements
The Solution • LSPs anchored to WRED Queues on WAN side • All CPE side used High Priority • Tested across multiple BGP Domains • Tested QPPB for discovery of QoS resources
The Solution • Used RSVP to signal request for “sub-pool” reservation, e.g. guaranteed BW • Resulted in primitives being incorporated into DSTE-MPLS • Results used to write RFC 3270
AS 1 AS 3 AS 3 AS 2 AS 4
AS 1 AS 3 AS 3 AS 2 AS 4
AS 1 AS 3 AS 3 AS 2 AS 4
AS 1 AS 3 AS 3 AS 2 AS 4
TFN implementation • Needed to migrate to new network • Needed to provide services such as multicast and IPv6 • Needed to solve fish problem • Executed test plan based on Abilene test plan
Legacy POP Design I1 I1 ATM I2 ATM I2 ATM I1&I2 POP Campus
Legacy POP Design I1 I1 ATM I2 ATM I2 ATM I1&I2 POP BGP for Route diff Campus
New OARnet Design Goals • Reduce Costs • Reduce Complexity • Reduce Maintenance Fees • Deliver Services
MPLS Requirements • CPE device • PE Provider Edge • P Provider Core LSP Switching Router • We can collapse P and PE to one device • Need CPE for Label to IP binding • I1 will be standard routing • I2 will be Label Switched with BGP multihop to find correct path • Must deliver advanced services to I2 community • IPv6, Multicast, Jumbo Frames etc.
New Architecture PE/P CPE CPE GigE Aggregator GigE POP Campuses
New Architecture MPLS for I2 Routes LFIB IP for I1 Routes FIB Red = LDP tagged CPE PE/P BGP Multihop LDP Exchange with Core CPE BGP Multihop LDP Exchange with Core GigE Aggregator GigE AS600 Campuses AS3112
New Architecture CPE PE/P LR 1 LR 2 GigE Aggregator GigE AS600 Campuses AS3112
Rate Cap Architecture Red = I2 Cap Blue = Commodity Cap Green = Intra State Cap CPE PE/P CPE GigE Aggregator GigE AS600 Campuses AS3112
Some Implementation Issues • Had to come up with more robust naming convention • Old ALP1, SWALP1 • Required DNS overhaul • Pseudo CILLY code • CLMBN-R0, CLMBN-E0, CLMBN-O1, CLMBN-OT1