The Next Five Years - or … A blind man ’ s attempt to describe an elephant

1. The Next Five Years - or…A blind man’s attempt to describe an elephant A perspective on future evolution of Research and Education networks and their organizations Presented to the Swedish Universities Network consortium October 27, 2004 Stockholm, SE Jerry Sobieski Director, Network Research Initiatives Mid-Atlantic Crossroads

2. What will the network/elephant look like? The three blind men and the elephant…

3. The Three Blind Men need to expand their scope of efforts • Each had an accurate perspective of their piece of the overall picture…But they all disagreed on what the elephant looked like, and *none* of them were right. • They lacked integration • They didn’t try to understand the other man’s perspective about what an elephant was… • They never tried to understand why/how the elephant existed • My comments today are those of one of the blind men…() • To really understand elephants, we can’t just keep looking at the same old elephant – we need to study other elephants, we need to integrate many different perspectives, we need to look around the elephant to study the environment in which they exist, and the way they interact…

4. Outline • The Network Organization • Expanding its roles • The Network Evolution • Factors forcing change • Emerging technologies • A brief review of current experimental network testbeds • Conclusions

5. Research and Education Networks are Evolving • The networks we support today are changing – even more dramatically and rapidly than 5 or 10 years ago… • The “network” over the next few years will certainly become more important, but the measure of network effectiveness will be less about speeds and feeds and cost, and more about • How invisible and pervasive the network becomes to general users • And how the R&E networks collaborate with the emerging demanding applications to create new services and capabilities. • The organizations charged with addressing the networking needs of the R&E community will have to support a broader set of activities, initiatives, services, and technologies to accomplish these things.

6. Research and Education • Our success as “networking” organizations over the next 5 to 10 years will be measured by how we participate in and contribute to the Research & Education mission of our members. • Perhaps we already know this…but how well have we really executed? • Deploying and operating well engineered production IP networks is necessary - but no longer sufficient. • We must be an integral part of research initiatives • “supporting” research is not enough. we need to be in the trenches with the researchers • both in the networking realm and other disciplines • We must collaborate with and integrate the educational curricula of our institutions into our strategic objectives • We have unparalleled [network] resources in which we can create a richer environment for teaching and learning

7. The Value Proposition • The value our organizations provide to the members has historically been • High performance and “early adopter” [IP] network services that were not commercially available • Substantial cost savings over commercial service offerings • The private sector is rapidly chipping away the cost advantage (e.g. ISP services, capital equipment costs, etc) • And core backbone performance of most T1 ISPs is now as good as (or better than) our own.

8. Time for a New Value Proposition • What are the programmatic activities a central networking organization should be pursuing? • How do these create *real value* for our members? • How do we stitch these together into a compelling strategic picture for the future?

9. Expanding our Scope • Operational Network Services continue to be critical • High Performance IP services will continue to form the basis • Move upstream into network research and development • help define and to create the future network technologies • Move up layers into middleware and applications R&D and services • Build the “techknowledge transfer” to the IT departments and research applications. • Move down the stack into lower layer network services– • Our IP networks depend upon these, why shouldn’t these capabilities be part of our service portfolio for our community? • Expand the geographic scope of all these activities to include international collaborations

10. Middleware Initiatives Middleware Initiatives IP Services IP Services Applications Collaborations Applications Collaborations Middleware Initiatives Applications Collaborations IP Services Light Path Services (ethernet,,Sonet…) Light Path Services (ethernet,,Sonet…) Light Path Services (ethernet,,Sonet…) Research Programs Experimental Deployments Optical Services Optical Services Optical Services The Network Technology Space Production

11. Production Maturity Experimental Research Time Research and Development Activities Traditional new technology insertion activities consist of waiting for mature product availability, then integration testing into production core. Future activities must include research and development of new network technologies and standards, and experimental deployments including applications integration and service model development.

12. Research and Development Activities • A new set of applications is emerging that view the network quite differently… • Its not simply about [theoretical] bandwidth, or TCP… • (Congratulations to SUnet on being awarded the LSR this year!!!) • They do not really care to “play fair” and share network resources • “Best Effort” does not equate to “Good Enough” • The R&E networks must gain a much better understanding of these applications • They require deterministic, predictable, and repeatable network performance • They require dedicated network resources • They are global in scope – distributed facilities, distributed science teams

13. The Lunatic Fringe • These applications will make up fewer than 1% of our users, yet will consume “significant” (50% or more?!) portions of our backbone capacity • These research teams do not care to “play fair” with dorm MP3 traffic, web browsing, or even other research projects • They want their own dedicated network resources • These applications incorporate non-network resources such as computational clusters or special scientific instruments/sensors • The required network resources must be available when the science teams have access to the instruments • Flows will be real time (or near R/T) and large (multi-gigabit) • They will require “application specific” network topologies • Multiple links, instantiated between possibly many different locations simultaneously • Reserved in advance, instantiated when needed

14. The Lunatic Fringe in Sweden • electronic-Very Long Baseline Interferometry –“e-VLBI” • The radio telescope at Onsala Space Observatory makes radio observations simultaneously with telescopes at Westford MA, Koke Park HI, and Kashima Japan, and NASA Goddard in Maryland. • These recordings are sent to Haystack Observatory in MA for correlation. • Realtime correlation is being tested in the US • 512 Mbs per antenna has been demonstrated (Sep 2004) • 1024 Mbs per antenna will can be expected by early 2005 • 2046 Mbs by 2006 • 4 Gbs – per telescope – by 2007. • Correlator sites will require capacity of from 10 to 40 Gbs real time in three years.

15. Very Long Baseline Interferometry (VLBI)

16. Very Long Baseline Interferometry (VLBI)

17. Other Lunatics on the Fringe • Optiputer – • multi [10]gigabit channels interconnecting computational clusters, storage arrays, remote sensors & instruments… • CaveWave – multi-display ultrahigh resolution interactive visualization • 12 displays x 16 Mpixels/display x 32 bit color x 30fps • = 6 Gbits/frame  then update that display 30 times/sec (!) • The computational graphics requirements are daunting • And the network required to move this data interactively is mind-numbing • Ultra-Grid and HD-CVAN – High Definition video collaboratories • 900 Mbs realtime IP flows • 1.6 Gbs raw SMPTE292 … ~6-8 Gbs/stream as Super HiDef emerges • Multicast (IP) and Light Trees in optical domain • HEP – Large Hadron Collider coming online in 2007 in Cern • NASA Earth Observing System – hi resolution remote sensing

18. Key Emerging Technology Trends for the next few years • Photonic networks - will be the substrate of our future network services • “Light Path” services – connection oriented services for a new millennium • 10 Gbs Ethernet NICs in PCIe based PCs • The 40 Gbs hurdle – how will it impact R&E networks? • New application<->network interactions – Application specific topologies, Grids, scheduling,… • Security and privacy – hardening the new network and services against malicious and unscrupulous interactions • Wireless access – mobile, nomadic, and remote access

19. 10 Gigabit to the Desktop • 10 gigabit NICs for PCs = $3500 (25,000 SEK) • How many will it take to fill a 10gbs backbone? • Application cost for point to point 10gbs link is order of magnitude less than each router along the path • 10GE LANPHY router blades still around $80,000 (550,000 SEK) • Vs approx $35,000 for ITU transponder 10GE LAN • Vs approx $15,000 for 10GE LAN PHY ethernet switch port • Vs approx $3,500 for 10GE LAN PHY NIC • The cost of provisioning national and international IP networks to handle a small set of applications who have a few [very] large flows is enormous • Lower layer services (Light Paths?) can support these applications • Shunting these large point to point traffic peaks to lower layers of the network will allow our existing backbone capacity to last longer and more effectively serve the vast majority of our users

20. 40 Gigabits per second • 40gbs is going to be challenging • Conventional digital encoding techniques have serious limitations at 40gbs (due to NRZ dispersion) • This means new encoding schemes are required such as QAM, DPSK, or others, and the entire optical plant must be requalified for these technologies • The optical systems for 40 Gbs will not be cheap, and you can be sure the router blades will be formidable… • - 40 Gbs will only support four (4!) of those cheap 10ge NICs I just told you about… • 100 Gbs and 160 Gbs is further out yet – attaining these speeds in upper layers (Layer 2, 3) may incorporate bonding lower layer channels together (so what will we have gained?)

21. The elusive All-Optical Network • All photonic networks – minimize regeneration and buffering • Getting closer to reality – metro/regional scale all-optical wavelength networks are feasible today • All-optical wavelength switching products are just now being introduced • These technologies can potentially reduce the cost of high speed service delivery dramatically (see DRAGON Project) • ITU grid SFPs for 1GE ~ $3000 (21,000 SEK) • ITU XFPs for 10GE switches ~$7500 (52,000 SEK) • Optical Burst Switching – • Still looking for its niche – probably 3-5 years out yet… • Could be *very* important if all-optical buffering/queuing becomes practical… work continues…

22. Light Paths • “Light Paths” are the term du jour for connection oriented services • Arose from the hope that lambdas will provide cheap plentiful capacity • However, this new term compels people to think differently about deterministic and dedicated network resources end to end… • While all-optical nirvana.net has yet to be realized, these connection oriented services…uh...”LIGHT PATHS” (sorry) over sonet, ethernet, unframed waves, and MPLS LSPs are proving quite useful and cost effective for delivering Big, Long Pipes to the Lunatic Fringe. • The interest and utility of these capabilities is forcing the discussion: • How do we architect the evolving optical R&E networks to take advantage of an integrated “light path” capability? • How (and when) to deliver such capabilities to end systems? • How do we insure it complements the existing [very successfull!] IP core?

23. Light Paths – Open Issues • Automated provisioning of Light Paths – • the R&E community is trying to create a fresh architectural model…One that moves the model from a traditional carrier centric engineering and provisioning task to one that is dynamic, user oriented, and enables application specific services. • Control Plane – what tools and protocols will work best to realize global light path services? • A Common Service Definition Paradigm that provides consistent and verifiable “light path” performance: • End-to-end, across multiple administrative domains, incorporating heterogeneous network media,… • Authentication, Authorization, and Accounting for LPs • Reservation and Scheduling of network resources. • …in conjunction with non-network resources

24. Monitoring and Security • Our networks are our strength and our weakness • We need to better understand how this amorphous internet actually functions • This is critical to being able to more effectively understanding routing events, service failures, to perform event forensics, identify weakness and threats earlier • Monitoring and recording of network information is gaining ground • But integration and synthesis is still very poor (often due to privacy issues in themselves…) • (We need another blind man’s comments in this area)

25. Projects and Initiativesa brief survey of relevant activities • National Lambda Rail • An initiative of the US Higher Education Community • A “facility” that provides fiber, waves, and other network services to the R&E community (its NOT a network) • Rolling out waves to serve networks as well as waves to serve individual research projects • High Energy Physics, CaveWave, • Pacific Wave, HOPI, DoE Ultra Science Net, Atlantic Wave* • MAN LAN – Manhattan Landing • An exchange point in New York City that interconnects multi-layer service between Europe, Canada, and US parties (Abilene, A-Wave, GLIF,…)

26. Projects and Initiativesa brief survey of relevant activities • HOPI – Hybrid Optical Packet Infrastructure • An experimental infrastructure funded by Internet2 • A series of 10GE LANPHY segments across the US (over NLR) • Design team has established an architecture and engineering plan (see www.internet2.edu/hopi) • Will provide ethernet “light paths” in 1gbs and 10gbs granularity • Will provide automated switching capabilities at the Fiber, Ethernet, MPLS (via Abilene), and Sonet…broadly construed • Incorporates Abilene and regional networks as packet switched infrastructure • Abilene will remain at 10Gbs for the next 2-3 years • Experimenting with MPLS tunnels and UCLP and similar provisioning mechanisms for setting up LSPs across the core. • Looking to study performance deltas

27. Hybrid Optical Packet Infrastructure

28. HOPI Node SEA HOPI Node NYC HOPI Node WDC HOPI Node CHI Hybrid Optical Packet Infrastructure 10 GE LANPHY Point to point ethernet segments

29. Projects and Initiativesa brief survey of relevant activities • GLIF – Global Lambda Integrated Facility • A self selecting consortium of R&E interests who have “Light Path” resources they are willing to inter-connect. • A number of international OC192 and OC48 circuits • both trans-Atlantic and trans-Pacific are involved • Regional exchange points are also important components • Amsterdam, Chicago, NYC, … • NRNs and RONs are involved as well • See www.glif.is

30. Research Activities and Testbeds(at least a few that are setting the agenda) • DRAGON – Dynamic Resource Allocation over GMPLS Optical Networks • Mid-Atlantic Crossroads, USC/ISI-East, and George Mason University • Funded by US National Science Foundation ($6.5M / four years) • Based in the Washington DC metro region • Developing dynamic all-optical GMPLS control plane • Developing inter-domain service delivery mechanisms • Service capability advertising • Routing and signaling • AAA • Heterogeneous LSPs (packet into ethernet into waves…) • Application Specific Topologies • Open Source VLSR and GMPLS protocol stacks • See dragon.maxgigapop.net

31. DRAGON Dynamic Resource Allocation over GMPLS Optical Networks • - All photonic metro/regional core • Wavelength switches • Minimize OEO & regen

32. Transport Layer Capability Set Exchange NARB NARB NARB End System End System AS 1 AS 3 AS 2 DRAGONIntra- and Inter-Domain Dynamic Optical Services • IP control plane • GMPLS protocols: OSPF-TE and RSVP-TE internal to the domain • “Network Aware Resource Broker” for inter-domain services • Open source GMPLS stacks for end systems • Virtual Label Switching Routers (VLSR) covers non-GMPLS switches, e.g. ethernet switches, fiber cross connects

33. Research Activities and Testbeds(at least a few that are setting the agenda) • CHEETAH – • University of Virginia, North Carolina State Univ., City University New York • High performance file transfer using dynamic light path allocations • Incorporates Sonet VCAT and LCAS capabilities for 50 Mbs granularity, and gigebit ethernet • See www.ece.virginia.edu/~mv/html-files/ein-home.html • OMNInet – • iCAIR (Chicago), Nortel • All optical wavelength switching trial • See www.icair.org/omninet/

34. Research Activities and Testbeds(at least a few that are setting the agenda) • NOBEL – Next generation Optical Broadband in Europe (don’t know what the “L” stands for…) • An EU initiative, large participation by the carriers and vendors • Several testbeds have been part of the program.. • Most notably today: the ACREO testbed here in Sweden • Pursuing optical services via GMPLS and ASON architectures… • A recuring theme of dynamic allocation of dedicated network resources • The ACREO National Broadband Testbed (with KTH) • MUPPET

35. Conclusions • The National Research Networks and their regional collaborators are evolving: • Becoming an integral part of the R&D process that creates the future network technologies is imperative • Developing new higher layer services supporting the R&E mission and needs of the universities will also fall to these organizations – at least in part • Becoming an integral part of the educational curricula that turns out tomorrow’s network engineers and research scientists • Fostering global collaborations on all fronts • We need to re-create the energy, aspirations, and self confidence we had in 1997 – we have a great deal to accomplish. • We can genetically engineer better elephants!

36. Tack!