What’s MAX Production Been Up to? Presentation to MAX membership

1. What’s MAX Production Been Up to?Presentation to MAX membership Spring 07 Member Meeting Dan Magorian Director of Engineering & Operations

2. National LambdaRail Cogent ISP Internet2 NewNet Qwest ISP Qwest ISP R&E Nets Old Abilene MAX Production topology State of MD pop, Baltimore 1. Original Zhone dwdm over Qwest fiber BALT M40E 2. Movaz dwdm over State Md fiber 3. Gige on HHMI dwdm over Abovenet fiber Ring 4 4. Univ Sys Md MRV dwdm, various fiber Level3 pop, Mclean VA CLPK NGIX Ring 2 T640 MCLN CLPK T640 UMD pop, College Park MD CLPK MCLN Ring 1 Ring 3 DCGW DCNE ASHB ARLG Equinix pop, Ashburn VA GWU & Qwest DC pops, ISI/E Arlington VA pop

3. What has the Production Side of MAX been doing since the Fall member meeting? • Provisioned lambdas to help Internet2 transition to Newnet & ESnet routers in Level3 MCLN pop seamlessly. • Old Qwest router leaving for Kansas City Friday! • Designing new dwdm system for Baltimore BERnet • Uses “client dwdm”, lower end-to-end transponder costs. • Inexpensive idea usable anyone w/ local fiber, eg NOAA. • Interconnecting with new MIT pop in Baltimore at Level3 • DWDM ring replacement: Phase 1 this FY • listening to vendor presentations now • TAC meetings for community input next month • New lab across hall MAX hdq (almost) done, tours welcome. • Buying advanced 1G/10G ethernet/sonet tester

4. Ongoing projects (con’t) • 10G backbone upgrade underway, Juniper pics here soon • HHMI very generously providing 10G protected lambda to Equinix Ashburn replacing 1G. Thanks! • GWU provided old Juniper M20 and M10 routers for MAX lab. Thanks! Community welcome to use MAX test lab. • Provisioning Juniper VRFs for experimental service testing. • Ciena CoreDirectors only took 9 months to buy (!), will be installed for NGIX & Internet2 dynamic wave (DCS) fanout. • Want to convene community focus group to listen to member colo needs, mesh with what’s happening with I2. • AtlanticWave: slowly going production, governance group service document work, MAX providing measurement pcs.

5. New Service: Layer 2 vlans • MAX has traditionally run Layer 1 (optical) and Layer 3 (routed IP) service. • Only NGIX/E exchange point is Layer 2 service. • Continues to be demand for non-routed L2 service (vlans), similar to NLR’s FrameNet service. • As replace old Luxn dwdm system and put in new, • quite economic to put in dwdm filters and 10G ethernet switches with “colored” optics (XFPs) • This means that folks will be able to stretch private vlans from DC to Mclean to Baltimore over shared 10G channel. • Also will be able to provision dedicated ethernet bandwidth • Haven’t figured out service costs yet. Will ask TAC.

6. New Service: Flow analysis • Turns out it would be useful to be able to characterize traffic flows passing thru MAX infrastructure. • So MAX buying Juniper hardware assists to do this. • Starting to look at open-source Flow Tools analysis. • Have in mind web interface where people put in AS #s • then look at flows to/from their institutions • By date range (yr), ASN, source & dest, traffic type. • The data is anonymized, packet contents not collected. • Have the cost for this one figured out: free! • Will ask TAC about details of what would like to see after have a prototype to comment on. • Also just heard about a local flow analysis workshop by expert kc claffey of CAIDA to be held at Loyola Columbia.

7. Flash: MIT is building Nortel dwdm system into Baltimore & offering to trade lambdas • They have well-funded DR initiative paying for ring build • from Boston to Albany and NYC, • spur down coast thru Philly to Baltimore Level3 pop. • USM already has 2 fiber pairs 660 Redwood – 300 W Lexington, donating one for this (thanks!) • So now possibility of not just: • Baltimore lambdas south to MAX and Mclean to NLR and Internet2, • But also north to NYC and Boston, eg NOX/Harvard • A bit like AtlanticWave “north” • MIT costs/lambda still under discussion, but eager to trade, may give us a shared 10G in exchange.

8. MIT Nortel dwdm Albany MIT Nortel dwdm Boston Long-distance high-level diagram of newdwdm system (meeting MITin Baltimore) MIT Nortel dwdm NYC Lambdas to Europe MIT Nortel dwdm BALT BERnet dwdm 6 St Paul NLR and I2 lambdas BERnet Participants BERnet Participants MAX dwdm MCLN MAX dwdm CLPK

9. MAX view of current Baltimore L1/L3 topology (both lambdas pass thru CLPK router) BALT M40e BERnet Participants East & west OC48 lambdas on USM MRV ring Internet2 CLPK T640 MCLN T640 MAX Movaz lambdas NLR

10. Proposed BERnet Production L1/L2 topology USM F10 Redwood St USM F10 6 St Paul BERnet Participants BERnet Participants Xconnect New BERnet Res DWDM USM MRV USM MRV Xconnect New BERnet Res DWDM Participant Production lambdas BERnet Production 10G lambdas MAX Movaz Dwdm USM MRV New MAX F10 MCLN MCLN T640 New MAX F10 CLPK CLPK T640 New DC ring

11. High Level Regional Diagram of Balt Research Net JHU MIT 300 Lex. JHMI 40 wavelength MUX w ITU XFP’s Client Side Path UMBC 660 Redwood 6 St. Paul 40 wavelength Amplified Line Side Path One Transponder Pair to Pay for and Provision End to End MCLN NLR & I2 College Park

12. Node Sites 6 St. Paul Three nodes and room for 40 10 Gbps channels ~ $130K per site with Mclean Level 3 and College Park nodes being paid for by MAX. Pre Amplifiers at all three sites MCLN NLR & I2 College Park

13. “Client” and “Line” dwdm provisioning example Client-side: instead of “normal” 1310 optic, “colored” dwdm XFP pair on an assigned wavelength 40 km reach $6K JHU 660 Redwood 6 St. Paul Line side: Transponder Pair makes the 185 km reach with Amps and holds the client XFP $32K MCLN NLR & I2 College Park

14. DWDM DWDM DWDM DWDM DWDM DWDM More “Client Dwdm” examples between participants XFP pairs on an assigned wavelengths on 40 channel dwdm filters (40 km reach $6K). Red lambda is to DC, blue to NYC, green local to UMBC JHU switch 6 St. Paul 660 Redwood 300 W Lexington UMBC All each participant fiber needs is filter pair, not full dwdm chassis

15. MAX will replace some dwdm nodes this FY(summer install) and some next FY • Phase 1: College Park, Mclean, Baltimore research net. • Since have protected ring, can cut one side, then other. • Balt research fiber is new (actually old Ciena OC48). • Will select and procure vendor soon. • Phase 2: GWU and Qwest Eckington next FY • Replacing original Luxn dwdm ring • Demand for lambda services to Dc unknown • Have met with Qwest for comarketing to justify DCNE upgrade • Might do client dwdm with shared 10G ethernet instead • This means that MAX working to unified new dwdm system! • Get involved with TAC to hear details: tac@maxgigapop.net

16. Closing thought: “Lots of lambdas in your future, what are you going to do about it?” • This was the title of a BOF at the I2 MM yesterday hosted by Joe St Sauver, U Oregon. • Idea was to get people thinking about the sea change happening in the community • appropriate campus strategies for fanout • to researcher high-bandwidth projects. • Not replacement of “regular” ip routing, but in addition • Circuit-like services for high-bandwidth low-latency projects • People may say, “I don’t have any of those apps right now” • But suddenly may have researchers with check in hand • Need to plan for how will deliver, just as BERnet doing • Eg, in planning phase now for DC ring, need to forecast. • Talk to us about what you’re doing and thinking!

17. Thanks!