1 / 14

DAV support in DPM

DAV support in DPM. Alejandro Alvarez Ayllon on behalf of the LCGDM developer team IT/SDC 13/12/2013. Overview. Current release New headers 3 rd party copies News on performance Next release. Current release. Current stable release is 0.13.0 Released the 28 th of October

shakti
Download Presentation

DAV support in DPM

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. DAV support in DPM Alejandro Alvarez Ayllonon behalf of the LCGDM developer team IT/SDC 13/12/2013

  2. Overview • Current release • New headers • 3rd party copies • News on performance • Next release DAV support in DPM

  3. Current release • Current stable release is 0.13.0 • Released the 28th of October • Available in EPEL 5 and 6 • What’s new? • Want-Digest header support • X-Auth-Ip header support • Improved HTTP 3rd party copies DAV support in DPM

  4. Want-Digest header • RFC 3230 • The client can specify an additional header to request one (or several with or without priorities) checksums • Want-Digest: md5 • Want-Digest: md5;q=0.3 sha;q=1 • RFC doesn’t specify it, but adler32 supported too • Server answers with • Digest: md5=12345== • Digest is picked from DB is available, otherwise calculated on the fly by the disk node • Can be slow! DAV support in DPM

  5. X-Auth-Ip header • Can be used by trusted DNs • NsTrustedDNS configured in zlcgdm-dav.conf • In addition to X-Auth-Dn and X-Auth-Fqan[N] • Intended to allow “portals” on top of lcgdm-dav • i.e. A nice looking web application can request a token on behalf of the user, so then the user can do the PUT to the final location DAV support in DPM

  6. X-Auth-* GET /dpm/path/file X-Auth-Ip: 137.138.100.123 GET /myapp/some/file 137.138.100.123 302 Found Location: http://disk/pfn?token=abcdef 302 Found Location: http://disk/pfn?token=abcdef GET /pfn?token=abcdef DAV support in DPM

  7. 3rd party copies • Lots of work and refactoring of the 3rd party copy logic • Following agreement with dCache • For instance • Redirect mechanism to trigger delegation if supported, failure otherwise • Delegation only if needed • GridFTP-like performance markers DAV support in DPM

  8. 3rd party copies COPY/dpm/file Destination: https://destination 1 307 Temporary Redirect Location: http://disk/pfn?token=abcdef 2 4 PUT /destination Content-Length: 123 2 3 3 COPY /pfn?token=abcdef Destination: https://destination 307 Temporary Redirect Location: /pfn?token=abcdef X-Delegate-To: /delegation/endpoint 1 3 DAV support in DPM

  9. 3rd party copies • Client-side logic implemented in Davix • Used by gfal2-plugin-http • Testing of FTS3 for HTTP 3rd party copies ongoing • First functional, then stress tests DAV support in DPM

  10. Performance More comparisons between HTTP and XROOTD have been done to check the viability of HTTP as a every-day protocol The results are encouraging! DAV support in DPM

  11. Performance – Stress Test w. staging WebDAV XRootD DAV support in DPM

  12. Next release • It will be lcgdm-dav 0.14 • Few bug-fixes • Some needed for 3rd party copies • Broke with the latest GridSite based on CANL • Bigger buffer size for reading • Should improve performance • Global Access Service will be moved into dmlite-plugin-librarian • Is anyone actually using this? DAV support in DPM

  13. Next release • Working on HTTP as the DPM replication protocol • An administrator would do a PUT with a special flag to indicate “give me a new location where to replicate this file” • Pool and filesystem could be specified • The returned URL then can be used as destination of a COPY operation • Lcgdm-dav takes care of copying the file • No delegation needed, as the destination includes a token to authorize DAV support in DPM

  14. Questions? DAV support in DPM

More Related