240 likes | 485 Views
Graybox NFS Caching Proxy. By: Paul Cychosz and Garrett Kolpin. What are Graybox Techniques?. First we need to know a little about how the “graybox” works. Inference by observation No server or client-side modifications necessary. Our Problem. Figure out what the client is caching
E N D
Graybox NFS Caching Proxy By: Paul Cychosz and Garrett Kolpin
What are Graybox Techniques? • First we need to know a little about how the “graybox” works. • Inference by observation • No server or client-side modifications necessary
Our Problem • Figure out what the client is caching • Maintain low overlap between client and proxy caches
Our Solution • Treat the client as a graybox • Monitor NFS traffic • Infer from NFS traffic the contents of client cache
Conclusions Cache overlap decreased by 5% to 50% when using graybox techniques
Outline • Approach • Results • Conclusions
Approach NFS specifics: • Implemented in kernel as another FS type • Replaces “file_operations” with its own functions for read, write, open, etc. • Uses VFS abstraction VFS RPC / XDR retrans., data wrapping NFS kernel code TCP / UDP
Approach Client cache / Replacement • Caches on page level • Not much NFS specific code – Uses global Linux page cache • Not a static size, no upper-bound • Timestamps: “jiffies”, per page.
Approach Reads typically > ~85% of traffic NFS / RPC “read” structure • RPC Packets sent out: access to dir, getattr to file, access to file, then read, read, read, … • Client use of getattr to revalidate cached copies • Exploit this, proxy explicitly look for these to determine client cache contents
Approach Proxy • Sees: • File handle • Offset • size --becomes our “cache element” • Store in hash table • Cache on reads, run our cache element replacement policy on getattr (proof of cached client copy)
Approach NFS Client Proxy Replacement (possible force-out) Read system call getattr Revalidate data hit Cache lookup OR Lookup read miss Read setup OR hit miss Return data Add Data Typical read() Forward req. to server
Viewing Client Cache Contents Test Environment: • Disable swap space • Use mincore() to view pages that client has cached • Use “balloon” program to keep client cache contents manageable
Approach Difficulties • Logistics: Initial setup, client/server/proxy all on same PC – doesn’t work • Client cache size changes -solved: compare percentages, normalize • Repeated RPC calls not idempotent • Not a significant problem, proxy efficiency degrades linearly with NFS performance if network traffic is bad
Approach Difficulties • Ambiguities: • Different FS activities DON’T generate unique RPC patterns • getattr used for a few things besides reads -- Null RPC packets Cannot tell exact blocks client caching, only which files Need second, local replacement policy on proxy!
Approach Cache Replacement Global Policies Local Policies • LRU • MRU • Forcing entire file • MRU
Outline • Approach • Results • Conclusions
What We Tested • Control Experiments • Global LRU, no local force-out • Global MRU, no local force-out • Combinations • Global LRU, whole file force-out • Global LRU, block-level local MRU • Global MRU, whole file force-out • Global MRU, block-level local MRU
How We Tested • Used Postmark benchmark • 12 files with sizes randomly selected between 4 and 50 kilobytes • Gives us more of a “real-world” workload
Outline • Approach • Results • Conclusions
Conclusions • Decrease in overlap by between 5 and 50 percent as compared to control tests • MRU global file replacement is good by itself • MRU global policy with whole-file local policy gives highest cache fill and lowest overlap
Questions? Feedback? ? ? ?