1 / 20

File System Access (XRootd)

File System Access (XRootd). Andrew Hanushevsky Stanford Linear Accelerator Center 13-Jan-03. Goals. High Performance File-Based Access Scalable Extensible Mass Storage integration Flexible Security Dynamically Configurable Rootd Compatable. Achieving High Performance.

sumi
Download Presentation

File System Access (XRootd)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. File System Access(XRootd) Andrew Hanushevsky Stanford Linear Accelerator Center 13-Jan-03

  2. Goals • High Performance File-Based Access • Scalable • Extensible • Mass Storage integration • Flexible Security • Dynamically Configurable • Rootd Compatable 2: XRootd

  3. Achieving High Performance • Multi-threaded server • Ability to run multiple servers on one host • MRU polling • Last active connections serviced first • Sticky sockets • Socket/thread bound for “active” connections • Using high-performance oofs layer • Unlocked I/O, sync reduction, file descriptor partitioning 3: XRootd

  4. Achieving Scalability • Socket multiplexing • One connection per client/host • Requires stream id protocol extensions • Dynamic Load Balancing • Requires client redirection protocol extensions • Using scalable oofs layer • File reuse, idle closeouts, cached file system 4: XRootd

  5. Achieving Extensibility • Opaque Information Protocol • Protocol extension allows client to relay outboard information to server • Messaging interface • Cache group • Request priority • Other things we haven’t thought of yet… 5: XRootd

  6. Achieving MSS Integration • Using oofs layer • Already integrates with almost any MSS • No brainer 6: XRootd

  7. Achieving Flexible Security • Negotiated Security Protocol Extensions • Allows client/server to agree on protocol • E.g., Kerberos, GSI, AFS Kerberos, etc. • Using oofs layer • Multi-protocol authentication support • Scalable authorization 7: XRootd

  8. Achieving Configurability • Servers must come up/down without harm • Part of dynamic load balancing • Requires client retry extension 8: XRootd

  9. Achieving Compatability • Client sensitive to server version number • Rootd servers default to base TNetFile class • XRootd servers use derived TNetFile class • Server requires client to turn on extensions • Base TNetFile class restricted to rootd ops • Derived TNetFile enables extensions • Multi-streaming, Redirection, Request Deferral, Generic Security, Opaque Information 9: XRootd

  10. security xrootd Architecture application Database Protocol Layer xrootd Filesystem Logical Layer oofs Filesystem Physical Layer ooss Filesystem Implementation xfs mss 10: XRootd

  11. Some Details • Compatible Protocol • Allows transparent use of rootd & xrootd • Multi-Streaming • Allows multiple files per connection • Cache File System • Dynamic Load Balancing • Client directed to “appropriate” server • Configurability & Recoverability 11: XRootd

  12. Compatible Protocol Application xrootd XTNetFile xrootd rootd compability TNetFile rootd 12: XRootd

  13. Multi-Streaming • Each open() assigned a stream number • Four character id that starts with 0-9 • All requests/responses identify relevant stream • Non-open file requests use arbitrary id • Any four characters not starting with 0-9 • E.g., stat(), mv(), unlink(), etc. • Parallel Multi-Streaming allowed • Client must explicitly enable it 13: XRootd

  14. Cache File System Index Area Optional data cache Default data area /databases/mydbfile symlink Naming convention allows for audit and index recovery Multiple Independent Filesystems /cache1/databases:mydbfile Data Area Any number Any Size Chosen based on free space in LRU order /cache2 /cache3 14: XRootd

  15. ufs ufs ufs Dynamic Load Balancing xrootd Dynamic Selection xrootd mss client xrootd 15: XRootd

  16. xrootd xrootd xrootd (any number) dlbd dlbd dlbd dlbd (any number) xrootd DLB Implementation I do subscribe who has the file? open again open Client wait try host:port 16: XRootd

  17. Configurability & Recoverability • Go hand in hand • Configuration changes may be disruptive • Must be able to recover from dynamic configuration changes • Dynamic Load Balancing essential part 17: XRootd

  18. Recoverability • Whenever client looses r/o connection • Back to distinguished xrootd(s) for reselection • Whenever client looses r/w connection • Limited wait/retry loop on the same server • We will be working to improve this next year! • All handled in the XTNetFile class • Disruptions merely delay the client 18: XRootd

  19. Configurability • Since we have recoverability… • Servers may come and go • New servers can be added at any time • Servers may be brought down for maintenance • Files can be moved around in real-time • Client will recover and adjust to the new configuration 19: XRootd

  20. Conclusion • Goals for rootd file access can be met • Most of this work relies on leveraging work done for the AMS • Compatability allows us to lobby the community for xrootd to become the default “rootd” 20: XRootd

More Related