1 / 29

University of California, San Diego February 17, 2004

BIRN Data Grid. Roman Olschanowsky. University of California, San Diego February 17, 2004. Introduction. BIRN-CC’s purpose is to provide it’s consortium of neuroscience laboratories the ability to share, compute, and collaborate.

joben
Download Presentation

University of California, San Diego February 17, 2004

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. BIRN Data Grid Roman Olschanowsky University of California, San Diego February 17, 2004

  2. Introduction BIRN-CC’s purpose is to provide it’s consortium of neuroscience laboratories the ability to share, compute, and collaborate. SRB provides the ability to transparently share data across remote sites.

  3. Outline • Introduction to SRB • Discussion on SRB basics • SRB Clients • Overview of BIRN Data Grid • Infrastructure • Topology • A BIRN site • SRB Stability • SRB Statistics

  4. Introduction to SRB • A distributed file system (Data Grid) • Client-server architecture. • Abstracts physical

  5. What we are familiar with

  6. What we are not familiar with, yet

  7. How do they differ? • Logical Abstraction • Folders are NOT physical • Files do NOT inherit physical location • Everything is potentially distributed • Access Control • Permissions are NOT rwxrwxrwx • Permissions ARE on a object by object basis • Groups and permissions ARE more similar to NTFS • Domains • Provides geographical / logical grouping of users • Provides namespace scalability: john@harvard john@mit • Also doubles as groups

  8. mySRB

  9. Common Scommands Sinit Senv Spwd Sls -l Sget Sget (Sget –m –b) Sbunload Sput Sput (Sput –m –b) Sbload Scp Smv (logical) Sphymove (physical) Srm Scd Smkdir Srmdir Serror Schmod • Don’t forget to Sexit!

  10. BIRN Portal

  11. Grid Computation • Globus • lddmm jobs: JHU utilizes both TeraGrid and BIRN for the calculation of shape differences between normal and diseased brain structures. Producing terabytes of resultant data stored back into BIRN data grid. • SRB Proxy Operations • Image conversions and manipulations via ImageMagick operations. (convert) • Analyze Volume Slice Extractor (anslex) • Data is not transferred, result of operation is streamed to user.

  12. The BIRN Data Grid

  13. The BIRN Data Grid

  14. The grid is in the details

  15. File Replication Sls /home/Demo/SRB-Tutorial/files-2: Doc.txt Sls -l /home/Demo/SRB-Tutorial/files-2: romanoly 0 z-ucsd-ncmir-nas1 15 2003-07-09-05.15 Doc.txt romanoly 1 z-jhu-cis-nas0 15 2003-07-09-05.16 Doc.txt romanoly 2 z-stanford-lucas-nas 15 2003-07-09-05.16 Doc.txt romanoly 3 z-umn-cmrr-nas0 15 2003-07-09-05.16 Doc.txt romanoly 4 z-uci-bic-nas0 15 2003-07-09-05.17 Doc.txt

  16. Components of SRB Space DR MC DR DR DR DR DR DR DR DR DR SC SC SC SC SC SRB SRB SRB SRB SRB SRB SRB SRB SRB SRB DR - Data Repository MC - Meta Catalog SC - SRB Client

  17. Physical Rack BIRN 1.0 Rack BIRN 1.5 Rack

  18. Physical Rack Layout • Gigabit/10/100 Network Switch – Cisco 4506 • Cisco 3002 3DES Encrypted VPN for secure statistics and packet tracing • Gigabit Ethernet Network Probe • DL 380 NetTools Network Statistics System • DL 380 General purpose compute platform • DL 380 Grid POP • SRB, Globus • Dual Processor Linux with1GB memory • Network Attached Storage – Gigabit Ethernet • 0.5 TB to 2 TB • UPS for Rack (208VAC/40Amp supply) • Direct Power for redundancy (208VAC/30Amp) Cisco 4506 GigE Net Probe Cisco 3002 Fiber Tap LCD PopUp Screen/Keyboard DL380 NetTools DL380 General DL380 Grid POP DL380 NAS 0.5 to 2.0 TB UPS

  19. SRB “Location” or “Slave Server” DR SRB “Physical Resources” z-jhu-cis-nas0 z-jhu-cis-nas1 “Location” z-jhu-cis-nas2 SRB “jhu-cis-nas”

  20. Logical / Compound Resources SRB “instant replication” “fast archival” “resource pooling” “My-Resource” SRB

  21. Monitoring Grid Status 0.7 TB 5.2 TB 0 TB 1.6 TB 0.8 TB 0.8 TB 3.2 TB 0.8 TB 2.4 TB 0.8 TB 0.8 TB 2.4 TB 1.6 TB 0.8 TB 5.0 TB 0.78 TB 0.08 TB

  22. Stability BIRN Grid is up and available approximately 98% Example of typical site uptime:

  23. 4.2 Million Files

  24. 6.6 Terabytes Used

  25. Logical Resources

  26. More Domains than Resources

  27. Collaboration Reports

  28. Collaboration Reports

  29. Conclusion SRB provides BIRN’s neuroscience laboratories with the ability to share and collaborate. Questions? www.nbirn.net www.sdsc.edu/srb roman2u@sdsc.edu

More Related