1 / 23

"The OptIPuter: Enabling Campus, State, National, and Planetary-Scale LambdaGrids"

"The OptIPuter: Enabling Campus, State, National, and Planetary-Scale LambdaGrids". LONI Forum Baton Rouge, LA September 2, 2004. Dr. Larry Smarr Director, California Institute for Telecommunications and Information Technologies Harry E. Gruber Professor,

quanda
Download Presentation

"The OptIPuter: Enabling Campus, State, National, and Planetary-Scale LambdaGrids"

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. "The OptIPuter: Enabling Campus, State, National, and Planetary-Scale LambdaGrids" LONI Forum Baton Rouge, LA September 2, 2004 Dr. Larry Smarr Director, California Institute for Telecommunications and Information Technologies Harry E. Gruber Professor, Dept. of Computer Science and Engineering Jacobs School of Engineering, UCSD

  2. OptIPuter Application Motivation: Interactive Analysis of Remote Multi-Gigabyte Data Objects • Hundreds Of Megapixels 2-D Images • Satellite Imaging and Remote Sensing • Telescopes or Microscopy • GigaZone 3-D (1k x 1k x 1k) Objects • Supercomputer Simulations • Seismic or Medical Imaging • Goal: Remote Interactive Analysis and Visualization • Virtually Impossible Today • New Technology of Dedicated Lightpaths • Bring These Capabilities to Laboratory Researchers

  3. Cosmic Simulator with a Billion Zone and Gigaparticle Resolution Source: Mike Norman, UCSD • 5123 AMR or 10243 Unigrid • 8-64 Times Mass Resolution • Can Simulate First Galaxies • One Gigazone Run: • Output ~10 TeraByte • “Snapshot” is 100 GB • Must Visually Analyze 10243 Unigrid Compare with Sloan Survey SDSC Blue Horizon (2004)

  4. OptIPuter Driver: On-Line Microscopes CreatingVery Large Biological Montage Images • 2-Photon Laser Confocal Microscope • High Speed On-line Capability • Montage Image Sizes Exceed 16x Highest Resolution Monitors • ~150 Million Pixels! • Use Graphics Cluster with Multiple GigEs to Drive Tiled Displays IBM 9M Pixels Source: David Lee, NCMIR, UCSD

  5. Earth System Enterprise-Data Lives in Distributed Active Archive Centers (DAAC) NSIDC (67 TB) Cryosphere Polar Processes LPDAAC-EDC (1143 TB) Land Processes & Features ASF (256 TB) SAR Products Sea Ice Polar Processes SEDAC (0.1 TB) Human Interactions in Global Change GES DAAC-GSFC (1334 TB) Upper Atmosphere Atmospheric Dynamics, Ocean Color, Global Biosphere, Hydrology, Radiance Data ASDC-LaRC (340 TB) Radiation Budget,Clouds Aerosols, Tropospheric Chemistry ORNL (1 TB) Biogeochemical Dynamics EOS Land Validation GHRC (4TB) Global Hydrology PODAAC-JPL (6 TB) Ocean Circulation Air-Sea Interactions EOS Aura Satellite Has Been Launched Challenge is How to Evolve to New Technologies

  6. However, Average Throughput of NASA Data Products to End User is Only 50-100 Megabits/s Tested from GSFC-ICESAT August 2004 http://ensight.eos.nasa.gov/Missions/icesat/index.shtml

  7. High Resolution Aerial Photography Generates Images With 10,000 Times More Data than Landsat7 Landsat7 Imagery 100 Foot Resolution Draped on elevation data New USGS Aerial Imagery At 1-Foot Resolution ~350,000x350,000 Pixel Images of 350 US Cities 100 Billion Pixel Images! Shane DeGross, Telesis USGS

  8. States are Acquiring Their Own Dark Fiber Networks -- Illinois’s I-WIRE and Indiana’s I-LIGHT Indiana Today Nearly Two Dozen States are Creating Dark Fiber Networks—Including the Most Advanced LONI! Source: Charlie Catlett, ANL

  9. Regional Network Evolution • The Fundamental Nature of Regional Networking is Rapidly Changing • The GigaPoP Model Based on Provisioned, High-Capacity Services Steadily Is Being Replaced • On The Metro and Regional Scales • A Model of Facility-Based Networking Built with Owned Assets has Emerged • Regional Optical Networks (RONs) • Notably, this Change Increases the Importance of Regional Networks in the Traditional Three-Level Hierarchy of U.S. R&E Advanced Networking Source: Steve Corbató, Internet2

  10. Leading & Emerging Regional Optical Networks New York + New England Region (NEREN) North Carolina (NC LambdaRail) Ohio (Third Frontier Network) Oklahoma (OneNet) Oregon Pacific Northwest (Lariat – Supported by NIH) Rhode Island (OSHEAN) SURA Crossroads Southeastern U.S.) Texas (LEARN) Utah Virginia (MATP) Wisconsin (WEROC) • California (CALREN) • Colorado (FRGP/BRAN) • Connecticut (Conn. Education Network) • Florida (Florida LambdaRail) • Georgia (Southern Light Rail) • Indiana (I-LIGHT) • Illinois (I-WIRE) • Louisiana (LONI) • Maryland, D.C. & northern Virginia (MAX) • Michigan (MiLR) • Minnesota Source: Steve Corbató, Internet2

  11. NLR Will Provide an Experimental Network Infrastructure for U.S. Scientists & Researchers “National LambdaRail” Partnership Serves Very High-End Experimental and Research Applications 4 x 10Gb Wavelengths Initially Capable of 40 x 10Gb wavelengths at Buildout Starting Fall 2004

  12. Dark Fiber: Gauging Community-Wide Progress • Aggregate Dark Fiber Assets Acquired by/for U.S. R&E Optical Initiatives • CENIC (for CalREN & NLR) 6,200 (Route-miles) • FiberCo (via Level 3 for NLR & RONs) 5,650 • SURA (via AT&T) 6,000 • Plus 2,000 Route-Miles for Research • NLR Phase 2 (WilTel & Qwest) 4,000 • OARnet 1,600 • ORNL (via Qwest) 900 • Other Projects (IN,IL,MI,OR, …) 1,500+ • Total (conservative estimate) 25,850+ • Over 55% of these Assets are in Regionals • Remainder held by NLR (11,250 Route-Miles) Source: Steve Corbató, Internet2

  13. Global Lambda Integrated Facility (GLIF)Creates Metacomputers on the Scale of Planet Earth NewYork MANLAN Stockholm NorthernLight 10 Gbit/s IEEAF 10 Gbit/s 10 Gbit/s 2.5 Gbit/s 2.5 Gbit/s 10 Gbit/s SURFnet 10 Gbit/s CA*net4 Amsterdam NetherLight Dwingeloo ASTRON/JIVE Chicago StarLight Tokyo WIDE 10 Gbit/s IEEAF 10 Gbit/s DWDM SURFnet NSF 10 Gbit/s 10 Gbit/s 10 Gbit/s 2.5 Gbit/s Tokyo APAN SURFnet 10 Gbit/s 2.5 Gbit/s 10 Gbit/s London UKLight Geneva CERN Prague CzechLight www.glif.is Created in Reykjavik, Iceland 2003 Source: Kees Neggers, SURFnet

  14. The OptIPuter Project – Removing Bandwidth as an Obstacle In Data Intensive Sciences • NSF Large Information Technology Research Proposal • Cal-(IT)2 and UIC Lead Campuses—Larry Smarr PI • USC, SDSU, NW, Texas A&M, Univ. Amsterdam Partnering Campuses • Industrial Partners • IBM, Sun, Telcordia/SAIC, Chiaro Networks, Calient, Glimmerglass • $13.5 Million Over Five Years • Optical IP Streams From Lab Clusters to Large Data Objects NIH Biomedical Informatics NSF EarthScope and ORION Research Network http://ncmir.ucsd.edu/gallery.html siovizcenter.ucsd.edu/library/gallery/shoot1/index.shtml

  15. OptIPuter Middleware Architecture for Distributed Virtual Computers DVC/ Middleware High-Speed Transport Optical Signaling/Mgmt OptIPuter Applications Visualization DVC #1 DVC #2 DVC #3 Higher Level Grid Services Security Models Data Services: DWTP Real-Time Objects Layer 5: SABUL, RBUDP, Fast, GTP Grid and Web Middleware – (Globus/OGSA/WebServices/J2EE) Node Operating Systems Layer 4: XCP l-configuration, Net Management Physical Resources From Grids to LambdaGrids Source: Andrew Chien, UCSD OptIPuter Software Systems Architect

  16. Creating a Model for a Campus LambdaGrid

  17. OMNInetThe Metro Area OOO Tesbed DWDM RAM C NTONC NTON 10 Gb Lambdas

  18. 10GE OptIPuter CAVEWAVEWill Help Launch the National LambdaRail Next Step: Coupling NASA Centers to NSF OptIPuter EVL Source: Tom DeFanti, OptIPuter co-PI

  19. OptIPuter Scalable Visualization Systems:End User Clusters Connected With Gigabit Flows Earth Sciences are an OptIPuter Driver Major NASA Goddard Visit This Week 20 Megapixels 3 Megapixels Linux Cluster with Nvidia Graphics Cards

  20. High Bandwidth Optical Fibers Will Enable High Definition Global Virtual Teaming UC Irvine UC San Diego In 2005 Cal-(IT)2 will Link Its Two Buildings Creating a Collaboration Laboratory

  21. Ultra-Resolution OptIPuter Displays Utilizing Photonic Multicasting --Scaling to 100 Million Pixels UIC-EVL GlimmerglassSwitch Used to Multicast and Direct TeraVision Stream from One Tile to Another on the Geowall-2 Driven by Linux Graphics Clusters Glimmerglass Switch 30 Megapixel High-Resolution Visualizations= ~1 Gigapixel at 30fps = ~30Gb/s Bandwidth

  22. Currently Developing OptIPuter Software to Coherently Drive 100 MegaPixel Displays • 55-Panel Display • 100 Megapixel • 11x5 21” LCDs • Developed at EVL • Driven by 30 Dual-Opterons (64-bit) • 60 TB Disk • 30 10GE interfaces • Linked to OptIPuter • Complementary to NASA ARC HyperWall

  23. Special issue of Communications of the ACM (CACM):Blueprint for the Future of High-Performance Networking • Introduction • Maxine Brown (guest editor) • TransLight: A Global-scale LambdaGrid for e-Science • Tom DeFanti, Cees de Laat, Joe Mambretti, Kees Neggers, Bill St. Arnaud • Transport Protocols for High Performance • Aaron Falk, Ted Faber, Joseph Bannister, Andrew Chien, Bob Grossman, Jason Leigh • Data Integration in a Bandwidth-Rich World • Ian Foster, Robert Grossman • The OptIPuter • Larry Smarr, Andrew Chien, Tom DeFanti, Jason Leigh, Philip Papadopoulos • Data-Intensive e-Science Frontier Research • Harvey Newman, Mark Ellisman, John Orcutt www.acm.org/cacm

More Related