What’s MAX Been Up to Lately? Presentation to MAX membership

1. What’s MAX Been Up to Lately?Presentation to MAX membership Spring 06 Member Meeting Dan Magorian Director of Engineering & Operations

2. MAX Production Net and Research Activities MAX Activities Production Net Activities ResearchActivities Layer 3 IP Network Services Layer 1 Optical Services Internet2 HOPI Test Bed support DRAGON Experimental Networking NGIX/East, AtlanticWave peer points Disaster Recovery Colocation GIG-EF Research Technologies Additional Funded Research

3. BALT Baltimore M40e Router NGIX T640 Router OC48c POS CLPK College Park OADM Aggregation Switch M160 Router Washington D.C. DCGW DCNE OC48c POS Abilene Northern Virginia Qwest OC48c POS ARLG M160 Router Where we were a year ago

4. Qwest Cogent R&E Nets Abilene NLR Qwest Where we are Spring 06 State of MD pop, Baltimore 1. Original Zhone dwdm over Qwest fiber BALT M40E 2. Movaz dwdm over State Md fiber 3. HHMI dwdm over Abovenet fiber Ring 4 4. (July) USM dwdm over State Md fiber Level3 pop, Mclean VA CLPK NGIX Ring 2 T640 MCLN CLPK T640 UMD pop, College Park MD CLPK MCLN Ring 1 Ring 3 DCGW DCNE ASHB ARLG Equinix pop, Ashburn VA GWU & Qwest DC pops, ISI/E Arlington VA pop

5. Has been/will be completed by July 06 • Protected ring out of fiber CLPK – Level3 pop Mclean • Built out to Equinix public peering facility with help of HHMI (thanks!). • Redundancy/doubled capacity Qwest ISP via Equinix. Pioneered low-cost “backup” 50mb rate vs 200mb with Qwest for Quilt folks. • After extended legal process, 400M Qwest ISP and I2 sold to HHS via Qwest Federal Systems with MAX as subcontractor. • At Equinix, likely will transit IPv6 carrier peerings to Internet2, making MAX primary east coast IPv6 exchange for them. • Brought up Cogent ISP through GWU. Anyone interested in $25/mb? • Good uptake on 2.5 & 10G lambdas to NLR: 4 leased, 6 in works • Phased out WhatsUp, deployed new Nagios-based open-source monitoring and alerting system for bgp routes as well as link state. • Hosted I2 performance workshop, built portable 10G test pcs available to members, deployed new noc and backup systems. • Hired 2 engineers, search for AtlanticWave engineer in progress.

6. Projects Underway FY06-07 • 10G L3 backbone upgrade (more detail on following slide) • Baltimore dwdm ring. USM selected MRV, was ready to go, but MDOT wanted in on project at 11th hour with additional sites and lambdas. Project expected to be online July. There will also be new 10G-capable Force10 switches at ring locations. • More use of external lambdas, dropping costs, new vendor Ekinops (French) has cost-effective dwdm add-in gear. • Atlantic Wave switch partnership with Ciena, will be used for upcoming GEANT peering (more detail on following slide) • Advanced Exfo chromatic dispersion/polarization mode dispersion test gear just ordered, needed as we provision 10G waves over contributed unknown fiber. Will get additional test gear in 07 for sonet & ethernet analyzers. Loanable members. • Continuing measurement and alerting infrastructure buildout for redundancy, alerting on optical, full info tracking database.

7. 10G L3 backbone/Abilene backup upgrade • Sun still intending to join MAX at 10G, but their grid computing project is delayed. Still expect to see 10G connectors in FY07. • However, Abilene backup with another gigapop, either MAGPI or SOX, may happen sooner in 07. Will need 10G for this, also NLR wave or other connection. • Need to move beyond current gige Abilene redundancy with CLPK and LVL3 routers as grow, to avoid having “eggs in same basket”. • Hard part has been getting Juniper 10G T640 router interfaces at reasonable cost. Finally found one pair used. • Finally convinced Juniper that need to offer 10G ints at high discount to all of higher ed to avoid missing out on 10G market. They will offer this to Quilt members at about 70% discount in 1-2 months.. • The way lambda costs have been dropping, only makes sense to provision 10G lambdas these days, and do 2.5G as subrate. FY05 10G transponder pair: $77k. FY06: $54k. Now, just paid $24k. Now as soon as we can get Juniper to follow suit…

8. Mid-Atlantic Crossroads DREN vBNS GEANT ESnet Abilene DRAGON NREN Cogent Qwest ISP NISN MAX Regional Infrastructure National/International Peering Networks Regional Network Participants NGIX-East GIG-EF Univ System Md Network Virginia Institutional Connectors

9. Strong Carrier Partnerships • With 7 MAX pop locations around region, participants can connect at one or more locations via dark fiber or lit services via aggressive low-cost carriers, as well as traditional telcos. • 80% & growing of participant circuits ethernet. • Strong partnerships with carriers (12 & growing!) • Qwest, Level 3, Fibergate, Starpower/RCN, Yipes, Allied Telecom, AboveNet, Looking Glass Nets, Verizon, MCI, ATT, Comcast.

10. MAX and NGIX/E upstream network peerings • Abilene (diverse: 2.5 gb primary /1 gb backup, 10 gb) • vBNS network (1 gb) • Defense Research & Exp Net (DREN) (1 gb) • NASA’s NISN and NREN (1 gb ea) • USGS DOI net (1gb) • Qwest DIA ISP service offering (1 gb, 2nd 1 gb) • Cogent ISP service (1 gb) • National Lambda Rail, potentially NLR L3 service. • GEANT2, “European Abilene”, (10 gb, May) • Energy Sciences net (ESnet) (1 gb, becoming 10 gb) • Atlantic Wave distributed peer fabric, (summer 06)

11. Atlantic Wave over NLR lambda CHI NYC LON STK SEA WDC TOK LAX CER ATL AMS A-Wave MIA • East Coast confederation of • Network Exchange points • Production level IP peering • Experimental “Light Path” services AUS Sao Paulo (SPB)

12. Atlantic Wave switch partnership with Ciena • Awave switch was in FY05, carried over to FY06 budget. Best guess, real costs were unknown. • SURA funding of Awave 10G lambdas over NLR is protracted process due to contractual issues with 4 universities. • Meanwhile, GEANT 10G sonet circuit arriving in May needing to be terminated using GFP framing. We need to move forward. • We held sonet switch vendor presentations by Ciena, Nortel, Sycamore, and Alcatel (used by GEANT), got quotes. • Ciena out of blue decided to >>donate<< CoreDirector switch & sonet interfaces to MAX, do co-development gmpls MOU with DRAGON. Were going donate 5 to Internet2, now MAX instead. • Talked Ciena into 2nd free switch, more interfaces: 1 production, 1 research. Will need to pay FY07 only for 6 newly released 10G ethernet ints, maint, training, etc. • Very signification donation to MAX, value $4-500k for 12 OC192 ints, 16 OC48 ints, 20 gige ints in 1 full height & 1 half chassis.

13. MAX expanded customer colo and disaster recovery sites • Disaster recovery sites have been a continuing request from members at MAX meetings. People want affordable colo space to host their own gear offsite, yet accessible. • In response, we have investigated 4 options: trade deals with Northrup Grumman and InterAmerican Development Bank, both new members with colo facilities; visited GWU’s VA facility, and striking a much better price per sq ft and per amp for a lot of space on the second floor of Level3 pop in Mclean. All still in progress, Level3’s would be most convenient if they can bring price down. • Meanwhile, Max unexpectedly acquired a fully equipped 1200 sq ft machine room across the hall with generator, dual large UPS, overhead power distribution, underfloor dual AC units. Will hold 38 cabinets. Need because MAX test lab out of space, but would like to make available to members also. Quote out to ring out 8400 fiber. • Would charge folks upfront for standardized cabinets, then lease lambdas back to their institution so have private dedicated bandwidth and looks like part of their network with no MAX L3 in between.

15. 1200 sq ft, can hold 38 cabinets, independent access to outside ramp

16. One of two large Liebert AC units feeding underfloor cooling

17. Dual 50 kva UPS with generator feeding overhead dual 208 VAC power rails

20. That’s fine for folks who can use DR in the DC area, but what about farther out? • Have begun talks with UMBC about a separate machine room space to be leased to MAX that could hold around 28 cabinets. • Location (40 miles from College Park) is good, already on new Baltimore dwdm ring so can provision colo lambdas. • UMBC was original UMD partner in “BWI gigapop” in vBNS days (97), so long history of good working relationship. • Power is good, space is good, but they need to install additional cooling for new space. Study underway, hopefully reasonable cost • Intend to have standardized offering both machine rooms for about $1500/cabinet one-time plus $500/cabinet/ month. Wanted to be free as benefit of membership, but needed recoup lease, maint, etc • Will likely lease private lambdas, mpls tunnels over shared 10G. • If can have online summer (8400) and early fall (UMBC), hoping to have good uptake in cabinet leases. Are you interested?

21. MAX Video Lab/Access Grid node • DRAGON has successful ongoing work with ISI/E and others for development of UltraGrid video codecs for HD over IP. • This work now is crammed into MAX “bank vault” conf room. • Also, we have lot of requests from MAX membership (eg, Holo- caust Museum this week) for help with video projects like multicasting events. Right now, nowhere to test against. • Idea is to take old MAX lab after equipment moved to new machine room, convert it to video lab and access grid node. Already well equipped with separate ac, good power, carpet and good lighting. Hope to have new video lab in by fall. Ladan Gharai of ISI and Chris Tracy of MAX compare parking permits over high def (“better than life”)

22. Recruiting for Engineering and Operations Advisory Group • Ongoing MAX Strategy and Research Advisory Groups. • Have had Core Engineering Group for years, but that needs to be expanded and meet more regularly (quarterly). • Want to offer chance for techies to help us plan future MAX engineering projects and directions. • Caution: this is not for management folks and those wearing suits. It will be highly technical. We hope that the lead network engineers of each institution will be interested. • The intent is for some sessions to have presentations of plans, others to have brainstorming sessions on selected topics, and periodically to have invited vendor presentations. • Interested? Please mailto://magorian@maxgigapop.net.

23. Thanks!