Announcements

1. Announcements • Now: focus on your Midterm 1, one week from today. • Make sure you understand the papers. This Thursday is your turn to ask questions. One week from today is my turn. • 7 Papers for Midterm 1: Beyond Rel DBs, RAID, Flash, Gamma, Bigtable, MapReduce, R-trees. • March 10th: MIDTERM 1 • March 5th: Perliminary draft of the project will be available. • March 26th: Document Design detailing how you will implement the algorithm. • April 9th: First implementation based on simple tests. • April 16th: Second implementation based on traces. • April 23rd: A competition to chose the fastest implementation. • April 30th: Midterm 2

2. Greedy Cache Management Techniques for Mobile Devices.by S. Ghandeharizadeh and S. Shayandeh.

3. Outline • Target environment: A roaming wireless device • Cache management strategies: • Offline: Simple • Online: LRU-K, GreedyDual • LRU-K does not support variable sized clips • GreedyDual does not support fix sized clips/frames • Improved variations: • Dynamic Simple, LRU-SK, IGD • Conclusions and future research

4. Today’s cell phone: Camera Camcorder Music jukebox Color video display Web browser Voice services A device is equipped with mass storage and may cache clips: Minimize the load on the wireless network, Download a clip faster and before the wireless connection is lost, Enable users to display cache resident clips when a device is not in the radio range of a base station, e.g., in an airplane. Example device: Target Environment Apple’s iPhone

5. FMC Phones • Fixed-Mobile-Convergence (FMC) phones equipped with both cellular and Wi-Fi networking cards • Wi-Fi provides limited radio range and high bandwidth. • FMC phone use the Wi-Fi to detect base stations and roam onto VoIP. Why? • Cheaper for customer • Better quality of service (higher bandwidth) • One number everywhere – on the road, at work, at home, when visiting a friend, etc. • Reliability

6. Reliability

7. Cache Management • Objective: Increase cache hit rate, the percentage of requests satisfied using the cache. A 90% cache hit rate means 9 out of 10 clips are serviced using the cache. • Based on the principle of reference locality: the user repeatedly references only a limited amount of data that is available out there. • Byte hit rate: Percentage of data that is serviced using local cache. A 90% byte hit rate means 90% of retrieved data came from the cache. • Why? • ↑cache hit rate means ↑ availability of data when a device has no base station coverage. • How to manage cache? • Offline techniques: Simple • Assumes advance knowledge of requests: Both “When?” and “what?” • Online techniques: LRU-K, GreedyDual What does “byte hit rate” optimize for?

8. Cache Management • Objective: Increase cache hit rate, the percentage of requests satisfied using the cache. A 90% cache hit rate means 9 out of 10 clips are serviced using the cache. • Based on the principle of reference locality: the user repeatedly references only a limited amount of data that is available out there. • Byte hit rate: Percentage of data that is serviced using local cache. • Why? • ↑cache hit rate means ↑ availability of data when a device has no base station coverage. • How to manage cache? • Offline techniques: Simple • Assumes advance knowledge of requests: Both “When?” and “what?” • Online techniques: LRU-K, GreedyDual Applies to techniques to manage a hierarchy of memories!

9. Simple [Ghandeharizadeh et. al. 2006] • Offline technique: assumes the entire sequence of clip references is known in advance. • Compute frequency of access to clips. • Freq-based algorithm: • Sort clips based on their frequency and assign the most frequently accessed clips to the device. • Simple algorithm: • Sort clips based on their byte-freq value and assign those with highest value to a device. Byte-freq of clip j is defined as its frequency of access divided by its size.

10. Freq-based and Simple are NOT Optimal • Freq-based and Simple are greedy heuristics.

11. Example • Assume capacity of a device is 14 bytes. • Consider the shown repository: • Simple assigns clip 2 to the device, resulting in a 10% hit ratio. • An optimal algorithm will assign clip 1 to the device, resulting in a 70% hit ratio.

12. Freq-based is not optimal • Assume capacity of each node is 14 bytes. • Consider the shown repository: • Frequency-based assigns clip 1 to a device, resulting in a 40% hit ratio. • Optimal assigns clips 2 and 3 to the device, resulting in a 60% hit ratio.

13. LRU [Denning 68] • Online technique. • Victim is the least recently accessed clip. Pseudo-code for LRU (page p) If p is in the buffer then LAST(p) = current time; Else i) Min = current time + 1; ii) For all pages q in the buffer do a) If (LAST(q) < min) victim = q Min = LAST(q) iii) If victim is dirty then flush it to disk iv) Fetch p into the buffer frame held by victim v) LAST(p) = current time

14. LRU-K [O’Neils, Weikum 93] • The victim page (page to be dropped) is the one whose backward K-distance is the maximum of all pages in buffer. • Definition of Backward K-distance bt(p,K): Given a reference string known up to time t (r1, r2, …,rt), the backward distance bt(p,K) is the distance backward to the Kth most recent reference to page p.

15. Limitation of LRU-K • Assumes equi-sized clips/pages. • Provides no support for variable sized clips.

16. GreedyDual [Cao, Irani 97] • Online technique • Conceptually: • Assigns a priority Q to each clip. • When clip j is cached, Q is set to 1/Size(j). • When there is insufficient free space: • Victim is the clip with the lowest Q value (Qmin) • For each cached clip j, reduce its Qj value by Qmin. • If a clip k is referenced and observes a cache hit, restore its Q value to 1/Size(k)

17. GreedyDual • Clever technique of [Cao, Irani 97] is simple and efficient.

18. A Comparison: Cache Hit Rate Variable sized objects. Larger cache

19. Key Findings • Simple and GreedyDual outperform LRU-K because they consider the size of clips when choosing a victim. • While memory is inexpensive, cache hit rate is a log function of memory size. • One may have to increase memory size several folds in order to increase cache hit rate by a few percentage. • GreedyDual is not a replacement for LRU-K with equi-sized pages. • GreedyDual does not consider how recently clips were referenced. • E.g., buffer pool of 2 clips and the reference string: o1, o2, o1, o3, o1, o2, o1, o3,…

20. A Comparison: Byte Hit Rate Variable sized objects. Larger cache

21. A Comparison: Equi-Sized Objects Equi sized objects. Larger cache

22. Enhanced Online Techniques • Is it possible to improve existing online caching techniques?

23. Enhanced Online Techniques • Is it possible to improve existing online caching techniques? • Of course, the answer is YES. Otherwise, I would not be assigning this paper for you to read.

24. LRU-S2 • Observation: LRU-K swaps out the clip with minimum value where Ik is the interval of time from current time to the last Kth reference. • LRU-SK swaps out clip with minimum value for:

25. Improvement Larger cache

26. Dynamic Simple (DYNSimple) • Online version of Simple. Key challenge: Compute the frequency of access to different clips online. • Address the challenge using the history of references maintained for each clip (LRU-K). • Conceptually: • Compute the arrival rate of requests for each clip i: • Frequency of access for clip j is its arrival rate divided by arrival rate of all other clips:

27. DYNSimple (Cont…) • A larger value for K: • Enhances estimates of frequency of access for a clip. • Makes it harder for DYNSimple to forget the past. DYNSimple won’t be able to react quickly to changing patterns of clip references. • A value of 2 for K should be sufficient for most cases.

28. Interval based GreedyDual (IGD) • Extend IGD to consider interval of time since last reference. • Conceptually: • Similar to LRU-K, maintain history of K references for each clip. • Let IK denote the interval of time for the past K references. • Let Ref(x) denote the number of references for clip x once it is cached, independent of its history. • Priority of clip x is set to:

29. Equi-sized clips

30. Zipfian distribution of access • Visit http://dblab.usc.edu for code

31. Varying Access Frequencies • 10,000 requests are issued after each shift. ClipID Clip Req Generator

32. Performance results

33. Adaptation

34. Adaptation (Cont…)

35. Contributions • Identified limitation of today’s cache management techniques by comparing them with one another. • Three novel (online) techniques to enhance cache hit rate of a device: • DYNSimple • LRU-SK • Interval based GreedyDual (IGD) • With K=2, DYNSimple provides a higher cache hite rate and adapts faster.

36. Future Research • Cooperative caching with • Stationary devices • Mobile devices

37. Stationary Devices • A home network may consist of several wireless devices that communicate with one another. • Example: An average household of 3 TVs uses one AppleTV to manage each TV. One or two PCs might also be deployed in the same household.

38. Stationary Devices 0 5 7 6 3 4 2 1

39. Stationary Devices • Should multiple devices cooperate and coordinate the state of their caches? • Why cooperate? • Increase availability of data in the home • Minimize average startup latency, defined as the amount of time elapsed from when a user references a clip to the onset of display. • Key design questions: • How does a device decide to join (leave) a cooperative group? • How is the autonomy of each device preserved when synchronizing its cache with other devices? • How is the placement of clips across the different caches controlled?

40. Cooperative Caching • Consider multiple devices: • Outside the coverage of the base station • Close enough to form an ad-hoc network • Should these devices cooperate with one another by making their caches dependent on one another?

41. Final Thoughts • Delivery of continuous media in wireless context is an exciting area of research with many challenging design decisions. • Requirements of an application will dictate design decisions. • Develop parameterized algorithms whose settings can support diverse applications.