1 / 14

Storage Elephant

Storage Elephant. Grid Access to Mass Storage. Grid clients. Grid clients. Grid clients. Control. Info. Data Xfer. Storage Element. Overview. MSS 1. MSS 2. Control Pin/unpin files (reserve for reading) Create new file (reserve for writing) Modify metadata ... Information

javen
Download Presentation

Storage Elephant

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Storage Elephant Grid Access to Mass Storage J G Jensen, RAL/WP5

  2. Grid clients Grid clients Grid clients Control Info Data Xfer Storage Element Overview MSS1 MSS2 J G Jensen, RAL/WP5

  3. Control Pin/unpin files (reserve for reading) Create new file (reserve for writing) Modify metadata ... Information Get amount of free space Protocols supported by the SE ... Data Transfer Interfaces J G Jensen, RAL/WP5

  4. HTTPS Reliable, well known protocol Looking at extensions for Grid security Web services Rapid development Language independent (but some languages are more independent than others…) GridFTP Extension of well known FTP protocol, complicated Achieves good transfer performance LDAP WP3 information producer Currently provides only status of daemons Worker nodes can access their local SE via NFS We will investigate using SlashGrid instead – RFIO is trivial to add Protocols J G Jensen, RAL/WP5

  5. RPMs have been or can be built for RedHat 6.2, 6.3, 7.2, 7.3 Support for RedHat 8.x soon SE builds from source on both Debian x86 stable and testing and on other Linux. TODO: Test on Solaris - ? Portability J G Jensen, RAL/WP5

  6. Client App The design of the SE follows a layered model with a central core handling all paths between client and MSS. Java Clients API SE HTTP library SSL socket Tomcat Axis Apache library Axis libraries Web Services C Client Client RMANMAN Core is flexible and extensible making it easy to support new protocols, features and MSS SE core SE J G Jensen, RAL/WP5

  7. SE CORE • Rmanager (request manager) is responsible for running handlers Response returned to user RManager Handler 1 Handler 2 Handler 3 Handler 4 • Each handler performs a simple task. • The sequence of handlers depends on type of request, the type of storage system, local configurations, etc. MSS This architecture is extremely flexible and allows SE to easily meet diverse requirements for both clients and backend MSS. J G Jensen, RAL/WP5

  8. Interfaces to SE Note: diagram shows existing interfaces, not implemented functionality Control Information Data Transfer HTTPS Done Planned Done Web S. Nearly done Planned Not planned GridFTP Done Not planned Done J G Jensen, RAL/WP5

  9. SE functionality • For MSS with many features, the SE will be a thin layer • For MSS with few features (or disk), SE will provide “missing” functionality = SE Functionality CASTOR ADS Disk J G Jensen, RAL/WP5

  10. Access control: Currently uses Andrew McNab’s GACL Will also use VoMS (and CAS?) Fine grained access control – files and dirs all have ACLs but ACLs can also be shared between files/dirs GSI Will use G-HTTPS Currently uses Mike Jones’ patch to mod_ssl to accept proxies Security J G Jensen, RAL/WP5

  11. SRM – storage resource manager – collaboration with LBNL, Fermi, Jefferson, CERN Particle Physics Mass Storage System interface Mostly control interface; data transfer done through GridFTP “Old” SRM specification version 1.0 – IDL, web service..? Some naughtiness: server callbacks New specification: interoperability through WSDL Is WSDL sufficient for interoperability? Complicated space reservation semantics SRM Collaboration J G Jensen, RAL/WP5

  12. There exist people who will use Grids for non-PP, non-HEP projects! Examples from DataGrid: Earth observations (WP9): will write handlers for their own MSS. EO applications have many (106) small (2-30 kB) files Medical Imaging (WP10): will do the same but WP10 have more challenging requirements Confidentiality – data must be enciphered when it goes out to the Grid Key will be held in a Grid database on trusted site CrossGrid is / will be based on DataGrid software Non-PP Grids J G Jensen, RAL/WP5

  13. SEs are deployed and working at CERN, talking to CASTOR using RFIO RAL, talking to ADS and disks SE soon to be deployed for testing and development at INSA Lyon INSA SE will talk to DICOM (medical images) Integration with testbed in March, tested by applications in May Deployment J G Jensen, RAL/WP5

  14. Current interfaces are flexible andcan be expanded to support new commandsand features, or add support for new protocols Core is very flexible and it is easy to add new functionality We’re making the SE easier to install and maintain – will be installable by sysadmins rather than developers! Concluding Remarks J G Jensen, RAL/WP5

More Related