1 / 9

SE Workshop

SE Workshop. John Gordon Oxford Datagrid workshop 3rd July 2001. StorageElement. Is the Grid Interface to access data not to find it, but to actually get the bits Both Services and Users will be able to access data in SE RepMan only in M9 a variety of protocols possible

keene
Download Presentation

SE Workshop

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. SE Workshop John Gordon Oxford Datagrid workshop 3rd July 2001

  2. StorageElement • Is the Grid Interface to access data • not to find it, but to actually get the bits • Both Services and Users will be able to access data in SE • RepMan only in M9 • a variety of protocols possible • Deals with permanent files by default • housekeeping and garbage collection possible • Publishes metadata about SE and files

  3. SE interface • Put/Get/Delete; • open/read/write/close; • allocate/pin

  4. Metadata • SE will publish a variety of metadata about SE and files • currently asked for: • closeCEs - list of CEs defined to be ‘close’ to SE in network terms • identified by MDS DNs • protocols supported • eg gridftp • existence of file - for RepMan housekeeping and to verify contents of RC • latency of files - how long to get a named file • attribute of above

  5. Protocols and APIs • GridFTP as transfer protocol • GridFTP server as SE file access • RFIO with GSI security as client API • Other APIs and UIs which support GridFTP will be implicitly supported for data access but not necessarily for management functions

  6. How does a user find data? • User requests LFN and protocol • Resource Broker maps LFN->PFN; chooses PFN in SE which supports protocol • RB chooses CE local to SE (can it also support protocol?) • RB constructs TFN from LFN and protocol • RB passes TFN to job • job ‘opens’ TFN

  7. User Access • If user uses RFIO then data will be streamed directly from SE • If user requires unix open then: • link with rfio library to trap open • link with globus i/o library to trap open • copy file to disk outside SE which is visible from CE • using rfio or • copy file to local disk on CE node • Is this enough?

  8. Questions • Do we need explicit NFS access to SE? • Needs applications to answer - Wednesday morning • answer = yes

  9. Month9 • GDMP is Replica Manager • GDMP will move files around Grid and maintain Replica Catalog • GDMP will put files on disk local to testbed (may or may not be managed by HSM) • User will access data locally via gridftp or rfio • RepMan will not move files around in order to schedule a job. • GDMP will not access SE directly to move data • WP5 deliverables

More Related