1 / 8

Wide Area Data Resources on the Teragrid

Wide Area Data Resources on the Teragrid. Scott Michael. August 26, 2014. Outline. GPFS-WAN DC-WAN Albedo. Thanks To. Chris Jordan J. Ray Scott Steve Simms Larry Diegel. GPFS-WAN. Capacity: 700 TB Availability: SDSC, Frost NCAR Allocation: Automatic, POPS, or web form

chet
Download Presentation

Wide Area Data Resources on the Teragrid

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Wide Area Data Resources on the Teragrid Scott Michael August 26, 2014

  2. Data Resources on the TG Outline • GPFS-WAN • DC-WAN • Albedo Thanks To • Chris Jordan • J. Ray Scott • Steve Simms • Larry Diegel

  3. Data Resources on the TG GPFS-WAN • Capacity: 700 TB • Availability: SDSC, Frost NCAR • Allocation: Automatic, POPS, or web form • Brief Description: Uses IBM’s GPFS file system similar to the Lustre file system but proprietary, All hardware resides at SDSC and remote clients mount over the TG network

  4. Data Resources on the TG GPFS-WAN Pros and Cons • Pros • Large disk space • Can have clustered metadata • Can support HSM with IBM’s HPSS • Cons • Mounted at few TG sites • Requires (costly) license from IBM • Best use case: Users at SDSC or NCAR

  5. Data Resources on the TG DC-WAN • Capacity: 340 TB • Availability: BigRed IU, Lincoln NCSA, Cobalt/Ember NCSA, Longhorn TACC, Pople PSC, Dash SDSC • Allocation: POPS, request • Default allocation: 10TB • Brief Description: Uses Lustre file system, Has been tuned extensively to be optimized on TG network, All hardware resides at IU and remote clients mount over the TG network

  6. Data Resources on the TG DC-WAN Pros and Cons • Pros • Available at many TeraGrid sites • Demonstrated performance over distance (few Gbit/s) • Cons • Susceptible to network outages • Lost client mounts require remounting by remote system adminstrators

  7. Data Resources on the TG Albedo • Capacity: 500 TB total, 150 TB per site • Availability: PSC, TACC, NICS • Allocation: POPS • Brief Description: Currently in pre-production, Uses Lustre with distributed disks and a centralized metadata store at PSC, Remote clients connect to PSC for metadata and local disk for reads/writes, Users can access indivdual storage locations via the Albedo directory structure

  8. Data Resources on the TG Albedo Pros and Cons • Pros • Good performance for I/O to local storage • Available at multiple TeraGrid sites • Cons • Still in pre-production, so not all problems have been found and solved

More Related