1 / 11

GridSite Web Servers for bulk file transfers & storage

Learn about the recent additions to the GridSite system for bulk file transfers and storage. This overview covers the security model, read/write access via HTTP(S), one-time passcodes, third party transfers, and SiteCast file location.

roblesr
Download Presentation

GridSite Web Servers for bulk file transfers & storage

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. GridSite Web Servers for bulk file transfers & storage Andrew McNab Grid Security Research Fellow University of Manchester, UK

  2. Outline • Recent “bulk file” oriented additions to the GridSite ( www.gridsite.org ) system • GridSite overview • Security model • Read/write access via HTTP(S) • Onetime passcodes • Third party transfers • SiteCast file location 15 February 2006 Andrew McNab – GridSite

  3. GridSite Overview • GridSite has evolved from the GridPP website management system • Now provides a Grid-oriented security toolkit (libgridsite) and extensions to the Apache webserver • Supports Grid/Web services on Apache using CGI • C/C++, Perl, other scripting languages • See GridSite Web Services poster for more details 15 February 2006 Andrew McNab – GridSite

  4. Design philosophy • Most Grid deployments (eg LCG + EGEE) are based on protocols and security technologies derived from the Web • So we attempt to reuse high quality implementations like Apache from the mainstream • This significantly reduces our support burden, since core Apache, mod_ssl, OpenSSL, ... is “RedHat's Problem” (or whoever does your distribution...) 15 February 2006 Andrew McNab – GridSite

  5. Security model • Authentication is done in Apache's mod_ssl using the client's X.509 certificate or GSI proxy • mod_gridsite dynamically modifies the OpenSSL callbacks to handle GSI proxies correctly • VOMS attributes are extracted if present, and the server has access to a cache of any DN-Lists which have been fetched asynchronously. • XML policy engine based on GACL or XACML languages decides whether access is permitted 15 February 2006 Andrew McNab – GridSite

  6. Read / write access • Almost all web traffic uses the GET method to fetch files, or POST to send the results of a form • But the HTTP/WebDAV RFCs also define PUT, DELETE and MOVE methods • mod_gridsite adds support for these “write” methods, subject to the policy-based access model • So HTTP(S) servers act as read/write file stores • Our htcp etc commands (cf scp) provide clients, but curl and many standard clients can be used too 15 February 2006 Andrew McNab – GridSite

  7. Onetime passcodes • For bulk files, may want an unencrypted data stream • cf GridFTP's use of an encrypted control channel and unencrypted data channel • GridSite achieves this using an HTTPS GET/PUT to establish access rights • The server then issues an HTTP redirect to an HTTP URL • A onetime passcode is returned as a cookie • This “GridHTTP” protocol works with unmodified versions of curl etc, and our htcp command 15 February 2006 Andrew McNab – GridSite

  8. Third-party transfers • WebDAV RFC defines a COPY method, which can be used for a client C to orchestrate a transfer of a file from remote server A to server B • GridSite now implements this, both in the server (gridsite-copy.cgi) and client (htcp) • We use onetime passcodes as a simple form of delegation from C to B, to give it the right to access the file • Supports both single stream and multistream HTTP 15 February 2006 Andrew McNab – GridSite

  9. Third-party transfers GET file, using passcode, over HTTP A - has file B - wants file file returned to B Onetime passcode GET - fetches passcode over HTTPS COPY - tells B to get file, gives passcode as cookie, over HTTPS C – Client/User “in charge” 15 February 2006 Andrew McNab – GridSite

  10. SiteCast • Current work is looking at how to locate local replicas of files on GridSite HTTP(S) servers • Have designed a simple replica location system for farms with many disks/hosts • Implemented in server-side (mod_gridsite) and htcp • Uses multicast of Hypertext Cache Protocol queries to find lists of replicas of a given file: looks at filesystem rather than any database • no database to keep in sync; automatically avoids replicas on dead machines; multicast can be filtered / routed by network hardware 15 February 2006 Andrew McNab – GridSite

  11. Summary • GridSite ( www.gridsite.org ) is already used for • Website/server management • Secured Web Services for grids, in C/C++/Scripts • Now also has features for bulk file transfer • Fine grained, VOMS-aware access control • Secure Read/write using HTTP or HTTPS • Third party transfers using COPY • Current work is on file location within a site • Using HTCP multicast to locate files • Very lightweight: no database needed 15 February 2006 Andrew McNab – GridSite

More Related