1 / 20

International Collaborations in Logistical Networking: Advancing Shared Storage Resources

This presentation discusses the concept of logistical networking and how it enables the efficient deployment and sharing of storage resources throughout the network. It explores the applications, architectures, and collaborative projects related to logistical networking.

janetkelly
Download Presentation

International Collaborations in Logistical Networking: Advancing Shared Storage Resources

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Micah Beck and Terry Moore Internet2 Member MeetingApril 10, 2003 International Collaborations in Logistical Networking

  2. LoCI Lab, including T. Moore A. Bassi Funding Dept. of Energy SciDAC National Science Foundation ANIR UT Center for Info Technology Research University of Tennessee Micah Beck James S. Plank Jack Dongarra University of California, Santa Barbara Rich Wolski Logistical Networking Research

  3. What is Logistical Networking? • A scalable mechanism for deploying shared storage resources throughout the network • An general store-and-forward overlay networking infrastructure • A way to break long transfers into segments and employ heterogeneous network technologies • P2P storage and content delivery that doesn’t use endpoint storage or bandwidth

  4. Why “Logistical Networking” • Analogy to logistics in distribution of industrial and military personnel & materiel • Fast highways alone are not enough • Goods are also stored in warehouses for transfer or local distribution • Fast networks alone are not enough • Data must be stored in buffers/files for transfer or local distribution • Conventional vs logistical networking • Datagram routers make spatial choices • Storage depots enable temporal choices

  5. Applications Logistical File System Logistical Runtime Sys L-Bone exNode IBP Local Access Physical Network Storage Stack • Model network storagearchitecture on the IP stack • Each level encapsulates details from the lower levels, while exposing details to higher levels • Layers already in place: IBP, exNode, L-Bone, Logistical Runtime System

  6. IBP: The Internet Backplane Protocol • Storage provisioned on community “depots” • Very primitive service (similar to block service, but more sharable) • Goal is to be a common platform (exposed) • Also part of end-to-end design • Best effort service – no heroic measures • Availability, reliability, security, performance • Allocations are time-limited! • Leases are respected, can be renewed • Permanent storage is to strong to share!

  7. exNode vs inode IBP Allocations • Metadata container for aggregating storage resources • Encoded as XML for network interoperability

  8. exNode + Lbone + LoRS: Download

  9. The Current L-Bone 160 depots, 16 countries, 26 US states, 10TB

  10. Multimedia Temporary storage Reducing (BWdelay) Reliable multicast Content Distribution Remote access to structured data Heterogeneous networks Managing computation state Very large data sets Bandwidth adaption Source routing Collaborative computing & visualization Logistical Networking Application Areas

  11. Planet Lab • Global overlay network for developing and accessing new network services • Seeded by Intel Research, currently 96 sites • IBP depots are running on them all and account for a lot of the use of PlantLab

  12. Lyon • Home base of LoCI Europe (Alex Bassi) • Tamanoir: HP networking project (INRIA – ENS)- • Using IBP for local data caching • IBP Service deployed node start up • Focus: Multimedia data distribution (semantic and active cache, stream adaptation, data placement) • Leaders: Laurent Lefevre (ENS) • IBCP (Institute for Biology and Protein Chemistry): Distribution of genome data

  13. E-Toile project • A computational Grid of several University, Research and Industrial sites in France • 7 participating sites, 4 of which will have IBP depot facilities at the beginning of the project • Depots are have arrived. Each will be a dedicated machine (e.g. the ENS at Lyon one will be a Sun with 1.3 Tb storage available)

  14. ARCHITECTURE PHYSIQUE DE LA GRILLE e-TOILE ID-IMAG Grenoble ID-IMAG Grenoble ID-IMAG Grenoble ID-IMAG Grenoble ID-IMAG Grenoble ID-IMAG Grenoble ID-IMAG Grenoble 12 PC bipro 1 Gb/s 250 PC en cluster IRISA Rennes IRISA Rennes VTHD 2.5Gb/s 2 Gb/s 6 PC bipro 1 Gb/s SUN Grenoble SUN Grenoble 1 Gb/s CEA Saclay CEA Saclay CEA Saclay 16 PC bipro Serveur 8 processeurs 1 Gb/s ENS Lyon ENS Lyon ENS Lyon ENS Lyon ENS Lyon ENS Lyon 1 Gb/s 1 Gb/s PRiSM Versailles PRiSM Versailles PRiSM Versailles PRiSM Versailles Clusters de PC 1,9 Ghz EDF Clamart EDF Clamart EDF Clamart EDF Clamart EDF Clamart 8 x 2 PC linked by SCI Serveur bipro MP760 IBCP 1 cluster de PC Serveur bipro 1,2 Ghz Machine SMP 16 power PC linked by Myrinet Routeur actif Routeur actif 1 cluster Myrinet de 10 PCs 1 cluster de 8 PCs Service de dépôt de données transientes IBP Serveur 3* bipro 16 Sun Cobalt E-Toile Grid Architecture

  15. CERN DataTag project • DataTag project: High performance Grid infrastructure for sustained, reliable, high performance data movement replication • Minimum of 100Mbps in the early stages • European Collaborators: CERN, INFN, INRIA, University of Amsterdam • VLAN via STARLIGHT with 10GigE interface to Abilene • Recently installed an IBP depot with 2.5Gbps connection to Chicago

  16. Italy • GARR: Italian Academic & Research Network • Collaborator in the International 6Bone project • Coming up in the next few weeks: 3 dedicated servers to be put on the PoPs (each one around 140 Gb for the beginning). • Gabriella Paolini • Universita del Piemonte Orientale • Depots at UPO, Surfnet, Tennessee • Cosimo Anglino – Data Transfer Scheduling Optimization

  17. Milano RT PoP 6net TORINO GEANT 6NET BACKBONE 6NET GARR MILANO Milano RTG PoP 6net PISA Pisa RTG GigaPoP Milano Via Lancetti Bologna RTG PoP 6net BOLOGNA Roma RTG PoP 6net ROMA GigaPoP Roma P. A.Moro Initial GARR depot locations

  18. Univ. of San Paulo: Research to enhance app performance on Brazillian research networks (Dr. Sergio Kofuji, Flavio Silva) University of Puerto Rico: Biomedical Research Infrastructure Networks in Puerto Rico (BRIN-BR) (Guy Cormier) Nanyang Technological Univ. (Singapore) Grid Replica Catalogue over IBP (Dr. Francis Lee & Ming Tang) Thailand Putchong Uthayopas of Kasetsart University Australian National U. Content distribution (Markus Buckhorn) Other projects

  19. International Project Futures Proposals in development for the European commission • Active Grid Networks (AGridNet) • Programmable Grid infrastructure focused more on research applications • Polymorphic Internet • Large public/private consortium led by Alex Gallis (University of London) • Programmable cyberinfrastructure focused on commercial applications

  20. Conclusion: Join Us! • As a user • Open infrastructure • Tools are free and easy to use • As a provider • Storage is cheap (<5k per TB) • Depots are easy to deploy • Server many application communities at once • More information and software available at http://loci.cs.utk.edu

More Related