1 / 10

NeST: Network Storage Technologies

NeST: Network Storage Technologies. Building an I/O Appliance on Commodity Systems John Bent, Remzi Arpaci-Dusseau and Miron Livny. www.nestproject.com. Problem Statement.

seanharris
Download Presentation

NeST: Network Storage Technologies

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. NeST: Network Storage Technologies Building an I/O Appliance on Commodity Systems John Bent, Remzi Arpaci-Dusseau and Miron Livny www.nestproject.com

  2. Problem Statement Appliances are attractive because they are robust, reliable, available and especially because they are easy to use. To fulfill these criteria, traditional network appliances impose policy decisions on their users and are built either as kernel modules or upon specially designed kernels. “How to build a portable, configurable I/O appliance?”

  3. Goals • Provide stable, robust and easy to use network storage on a variety of platforms • Allow configuration of server modules to fit application needs • Maintain high server throughput under heavy load • Degrade client performance gracefully as concurrency increases

  4. Nest Structure Protocols Name Space Static Configuration Administrative Interface Concurrency Architectures Runtime Adaptation Storage Management Consistency Semantics

  5. Protocols • Nest currently supports the following client requests in “nest-speak”: • Send/receive file • Send/receive partial file • Query statistics • Directory operations • Remove file • Timestamp request • Future support: ftp, http, nfs, WiND fs.

  6. Concurrency Architectures • Non-blocking single process (NOB) • Pre-allocated pool of processes (POP) • Pre-allocated pool of threads (POT) All concurrency architectures use one control thread to accept and dispatch client requests. This approach will more easily allow centralized decision making as regards admission control, authentication and quality of services guarantees.

  7. Other Modules • Static configuration • Build the most appropriate I/O appliance based on knowledge of the host system and the application needs • Namespace • Current: flat namespace • Future: hierarchical • Runtime adaptation • Bandwidth monitoring and other flows of information may prove useful in consistently maximizing system throughput

  8. Even More Modules • Consistency semantics • Well defined behavior is necessary in order to allow concurrent client access to the same file • Storage management • Current: ext2 FS on linux • Future: raw disk management • Administrative interface • An “easy to use” tool must have a nice administrative interface. This will probably be added through the http protocol.

  9. Performance Server bandwidth in each of the concurrency architectures remains constant as the number of clients increases. Client latency and bandwidth are also affected correctly as client concurrency grows.

  10. Future Work Quality of service Admission control, authentication Hybrid concurrency architectures Static configuration, runtime adaptation, namespacing, storage management, consistency semantics Future Applications Condor checkpoint server Local staging area for Condor jobs which have flocked over the WAN Storage “brick” for WiND Future Lots and lots and lots and lots more

More Related