1 / 22

NeST: Network Storage Flexible Commodity Storage Appliances

NeST: Network Storage Flexible Commodity Storage Appliances. New worlds, new promises. The grid has transformed dist comp Multiple LANS have merged Confederations of auto admin domains Possibilities for resource and data sharing abound. New worlds, new problems.

eve-whitney
Download Presentation

NeST: Network Storage Flexible Commodity Storage Appliances

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. NeST: Network StorageFlexible Commodity Storage Appliances

  2. New worlds, new promises • The grid has transformed dist comp • Multiple LANS have merged • Confederations of auto admin domains • Possibilities for resource and data sharing abound

  3. New worlds, new problems • Interoperability between domains • Management of resources more important • Discovery of resources more difficult

  4. Enter NeST • Traditional file servers have not evolved • NeST is a 2nd gen file server • Flexible storage appliance for the grid • Provides local and remote access to data • Easy management of storage resources • User level sw turns machines into storage apps • Deployable and portable

  5. Research meets production • NeST exists at an exciting intersection • Freedom to pursue academic curiosities • Opportunities to discover real user concerns

  6. Very exciting intersection

  7. Very exciting intersection

  8. Outline • Introduction • Interoperability • Resource management • Questions?

  9. Multi-protocol support • Interoperability between admin domains • NeST currently speaks • Grid FTP and FTP • HTTP • NFS (beta) • Chirp • Designed for integration of new protocols

  10. Why not JBOS? • Just a bunch of servers has limitations • NeST advantages over JBOS: • Single config and admin interface • Optimizations across multiple protocols • e.g. cache aware scheduling • Management and control of protocols • e.g. prefer local users to remote users

  11. Dangerous migrations

  12. Wasted migrations • Wasted migrations are more expensive • Distant resources are a mixed blessing • Great opportunities • Possibility for great waste

  13. Full migration support • NeST makes storage schedulable and reservable • NeST allows creation of dynamic users • Too expensive to create all accounts everywhere • Sharing is insecure and makes accounting difficult • NeST has lots • Reservations of time and space

  14. Three way matching Refers to NearestStorage. Knows where NearestStorage is. Job Ad Machine Ad Storage Ad match Job Machine NeST

  15. Type = “storage” Name = “turkey.cs.wisc.edu” HasCMSData = true CMSDataPath = /cmsdata” Storage ClassAd Three way ClassAds Type = “job” TargetType = “machine” Cmd = “sim.exe” Owner = “thain” Requirements = (OpSys==“linux”) && NearestStorage.HasCMSData Type = “machine” TargetType = “job” OpSys = “linux” Requirements = (Owner==“thain”) NearestStorage = ( Name = “turkey”) && (Type==“Storage”) Machine ClassAd Job ClassAd

  16. Conclusions • User level sw turns machines into storage appliances • Single admin interface across multiple protocols • Participates in resource discovery • Management of space and transfers

  17. For more information • http://www.cs.wisc.edu/condor/nest • Version 0.9 now available (linux only, no NFS) • Solaris and NFS coming soon • Requests welcome • People • John Bent, Nick Leroy, Alain Roy, Joseph Stanley • Miron Livny, Andrea Arpaci-Dusseau, Remzi Arpaci-Dusseau

  18. Control flow Dispatcher Datal flow Concurrencies Transfer Mgr Design structure Physical network layer Chirp FTP Grid ftp HTTP NFS Common protocol layer Storage Mgr Physical storage layer

  19. Lot management overhead

  20. NeST v JBOS, solo

  21. NeST v JBOS, team

  22. NeST proportional sharing

More Related