1 / 20

Dr. Larry Smarr, Director, California Institute for Telecommunications and Information Technology

The OptIPuter— Lambda Coupled Distributed Computing, Peer-to-Peer Storage, and Volume Visualization. Dr. Larry Smarr, Director, California Institute for Telecommunications and Information Technology Professor, Computer Science and Engineering, Jacobs School of Engineering, UCSD.

parry
Download Presentation

Dr. Larry Smarr, Director, California Institute for Telecommunications and Information Technology

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The OptIPuter—Lambda Coupled Distributed Computing, Peer-to-Peer Storage, and Volume Visualization Dr. Larry Smarr, Director, California Institute for Telecommunications and Information Technology Professor, Computer Science and Engineering, Jacobs School of Engineering, UCSD

  2. Two New Cal-(IT)2 Buildings Approved by Legislature This Week! Bioengineering • Will Create New Laboratory Facilities • Clean Rooms for Nanotech and BioMEMS • Computer Arts Virtual Reality • Wireless and Optical Networking • Interdisciplinary Teams UC Irvine UC San Diego

  3. Future Systems Will “Waste” Bandwidth and Storage, While Conserving Computing Scientific American, January 2001

  4. S-Curves of Exponential Technology Growth Lambda Grids Experimental Networks Production/ Mass Market DWDM 100% Technology Penetration Internet2 Abilene Experimental/ Early Adopters Connections Program 0% Research Gigabit Testbeds Time Technology S-Curve ~1990s 2000 2010 Networking Technology S-Curves

  5. A LambdaGrid Will Be the Backbone for an e-Science Network • Metro Area Laboratories Springing Up Worldwide • Developing GigE and 10GigE Applications and Services • Testing Optical Switches • Metro Optical Testbeds-the next GigaPOP? Apps Middleware Clusters C O N T R O L P L A N E Dynamically Allocated Lightpaths Switch Fabrics Physical Monitoring Source: Joe Mambretti, NU

  6. NSF Defines Three Classes of Networks Beyond the Commodity Internet • Production Networks (e.g. Internet2) • High-Performance Networks • Reaches All US Researchers • 24 / 7 Reliable • Experimental Networks • Trials of Cutting-Edge High-Performance Networks • Deliver Advanced Application Needs Unsupported by Production Networks • Robust Enough to Support Application-Dictated Development: • Software Application Toolkits, • Middleware, • Computing and Networking • Research Networks • Smaller-Scale Network Prototypes • Enable Basic Scientific and Engineering Network Research • Testing of Component Technologies, Protocols, Network Architectures • Not Expected to Be Persistent • Not Expected to Support Production Applications www.evl.uic.edu/activity/NSF/index.html

  7. Local and Regional Lambda Experimental Networks Are Achievable and Practical • Several GigaPOPs Are Building Multi-Lambda Metropolitan Experimental Networks by Lighting up Their Own Dark Fiber • With Hundreds of Lambdas by 2010 • LambdaGrid Software to Dynamically Couple Applications to Metro and Resources Is Possible • Metro Experimental Networks Will Be Sensitized to: • End Application Needs • Local Connectivity Issues • Current National Production Networks Have Not Begun to Satisfy • Substantial State and Local Funds Can Be Heavily Leveraged by an NSF Experimental Networks Program • Cross-country Inter-Connection • Persistent Support of Emerging Experimental Networks • First NSF Workshop UIC December 2001 • Second NSF Workshop UCI May 2002 • Expected NSF RFP by Fall 2002

  8. Cal-(IT)2 is Developing a Metro-ScaleExperimental Optical Network • Driven by Data-Intensive Applications • Real Time Seismic • Emergency Response • Medical Imaging • Linked UCSD and SDSU • Dedication March 4, 2002 Linking Control Rooms UCSD SDSU Cox, Panoram, SAIC, SGI, IBM, TeraBurst Networks SD Telecom Council 44 Miles of Cox Fiber

  9. Beginning of Cal-(IT)2Metro Optical Networking Laboratory

  10. Next Step– California Must Have a State-Wide Experimental Optical Network • The Institutes are Creating a Joint Plan • Led by Cal-(IT)2 & CITRIS • Involving QB3 and CNSI • Leveraging Today’s CENIC Investment • Provides California Internet Connectivity • K-12 and Universities • Necessary for Data-Intensive Science • Widely Available to Many Disciplines • California is Not the Leader Today! • Illinois, Indiana, and Canada Already There

  11. CENIC and CISI May Create a Dark FiberExperimental and Research Network The SoCal Component

  12. Some Scientific Applications Require Experimental Optical Networks • Challenges • Large Data Challenges in Neuro and Earth Sciences • Each Data Object is 3D and Gigabytes • Data are Generated and Stored in Distributed Archives • Research is Carried Out on Federated Repository • Requirements • Computing Requirements  PC Clusters • Communications  Dedicated Lambdas • Data  Large Peer-to-Peer Lambda Attached Storage • Visualization  Collaborative Volume Algorithms • Response • Cal-(IT)2 OptIPuter Research Project

  13. NIH is Funding a National-Scale Grid Which is an OptIPuter Application Driver Biomedical Informatics Research Network (BIRN) NIH Plans to Expand to Other Organs and Many Laboratories Part of the UCSD CRBSCenter for Research on Biological Structure National Partnership for Advanced Computational Infrastructure

  14. The OptIPuter is an Experimental Network Research Project • Multiple Lambdas Linking Clusters and Storage • Integration with InfiniBand PC Clusters • Rethink TCP/IP Protocols • Peer to Peer Storage • LambdaGrid Software Stack • Interactive Collaborative Volume Visualization • NSF Large Information Technology Research Proposal • Larry Smarr, PI • UCSD and UIC Lead Campuses • USC, UCI, SDSU, NW Partnering Campuses • Industrial Partners: IBM, Telcordia/SAIC, Chiaro Networks, CENIC • San Diego Telecom Council Letter of Support • Submitted April 4, 2002 • Seeking $15M over 5 Years

  15. The OptIPuter Research Team and ItsRegional, National, Int’l Extensions SURFnet CERN Asia Pacific CA*net4 Vancouver CA*net4 Seattle Pacific Light Rail Portland Chicago NYC UIC NU PSC San Francisco TeraGridDTFnet Asia Pacific NCSA CENIC USC UCI Los Angeles UCSD, SDSU Atlanta San Diego (SDSC) AMPATH Source: Tom DeFanti and Maxine Brown, UIC

  16. OMNInet Optical Switching TrialNow Underway in Chicago Metro Area 2x10GE • A Four-Site Network in Chicago -- The First 10GE Service Trial! • A Test Bed for All-Optical Switching and Advanced High-speed Services • Partners: SBC, Nortel, iCAIR at Northwestern, UIC, CANARIE, ANL Univ. Illinois at Chicago Northwestern Univ. 2x10GE 8x1GE 8x1GE 2x10GE Optical Switching Platform Optical Switching Platform Application Cluster Passport 8600 Application Cluster Passport 8600 OPTera Metro 5200 CA*net3/4--Chicago StarLight 8x1GE 8x1GE 2x10GE Optical Switching Platform Application Cluster Optical Switching Platform Application Cluster Passport 8600 Passport 8600

  17. Planned Chicago Metro Electronic Switching OptIPuter Laboratory Internationals: Canada, Holland, CERN, GTRN, AmPATH, Asia… Int’l GE, 10GE 16x10 GE 16x1 GE Metro GE, 10GE 16-Processor McKinley at University of Illinois at Chicago 16-Processor Montecito/Chivano at Northwestern StarLight 10x1 GE + 1x10GE Nat’l GE, 10GE Nationals: Illinois, California, Wisconsin, Indiana, Abilene, FedNets. Washington, Pennsylvania… Source: Tom DeFanti

  18. Plan for UCSD Campus-Scale OptIPuter Initially Connect: Schools of Medicine and Engineering with SIO and SDSC Philip Papadopoulos, SDSC

  19. OptIPuter Optical Research Topics • Integrate InfiniBand and IP Over Lambdas • Connectivity, Addressing, Configuration, Recovery and Resource Discovery • Design Evaluation, Monitoring and Analysis Tools for • Optimizing InfiniBand Over Dedicated Lambdas • Devise Network Control and Management (NC&M) and Traffic Engineering • Gather Cluster Demand and Signal the Reconfiguration of the Network • Extend Signaling • Permit OptIPuter Clusters to Establish and Teardown Connections to Peer Clusters Across the IP/Lambda Network • Integrate This Signaling With Network-based NC&M • Extend the IP/Lambda Network to Support Very Fast Switches • Participate in the Optical Switching of Label Switched Paths (LSPS) That Transport InfiniBand Messages • Develop Common-Control-Plane Optical Transport Architectures • With Generalized Multi-Protocol Label Switching (GMPLS) • Transport Traffic Over Multiple User Planes With Variable Switching Modes, Namely Lambda Switching • Burst Switching • Inverse Multiplexing (Enabling One Application to Use Multiple Lambdas)

  20. Other OptIPuter Research Topics • Software Architecture • LambdaGrid Middleware Architecture • Application Communication Abstractions and Cluster Bundle Composition • Real-Time, Dependable Execution Environment • Distribute Storage While Optimizing Storewidth: • Distribute Massive Pools of Physical RAM (Network Memory) • Develop Visual TeraMining Techniques to Mine Petabytes of Data • Enable Ultrafast Image Rendering • Create for Optical Storage Area Networks (OSANs) • Enhance Security Mechanisms: • End-to-End Integrity Check of Data Streams • Access Multiple Locations With Trusted Authentication Mechanisms • Use LambdaGrid Middleware for Authentication, Authorization, Validation, Encryption and Forensic Analysis of Multiple Systems and Administrative Domains

More Related