1 / 8

S3fs in the wide area

Want to reduce wide-area network delays and enhance user experience? Learn about implementing Libasync for better bandwidth utilization and cache management. Explore optimizations for improved thread safety and data structures.

rclyburn
Download Presentation

S3fs in the wide area

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. S3fs in the wide area Alvaro Llanos

  2. Motivation • Reduce delay of the wide-are-network • Better end-user experience • Improve bandwidth use ? (deciding when to flush the cash)

  3. Design

  4. Implementation • Libasync • For handling write/read requests • Using sockets to implement • Libasync running on a different thread • Simple cash • 2 Maps, meta and data buckets • LRU replacement policy, (partial)

  5. Cache Detail Cache LRU Module S3 META DATA LibAsync Requests Cache Entry detail Char* key Void* value Bool dirty

  6. Flow of Requests Requests Is it a Read? No Send Update Request and RETURN Yes Is object in the Cache? Send Request of Read and WAIT response Return object No Yes Retrieve object from Cache

  7. Key points • Libasync • Used to queue the requests • Perform the s3 ops • Flush to S3 • Is done after sending an answer to the user • Could be done following some policy with the proper recovery and lock implementation • Cash (partial implementation) • Simple cash, key=s3Key, value=data + flags • Recovery • Not implemented

  8. Improvements to be made • No failure support • Ones it is written in the cash, it is assume it will be uploaded correctly. • Improve thread-safety • Improve data structures, flags and controls

More Related