Temporal and Spatial Locality: A Time and a Place for Everything

1. Temporal and Spatial Locality:A Time and a Place for Everything Rick Bunt University of Saskatchewan Carey Williamson University of Calgary

2. Roberta Bodnarchuk Milind Deshpande Kevin Froese Neeraj Gulati Sung Hung Ling Shikharesh Majumdar Shane McDonald Jennifer Murphy Adeniyi Oke Judy Peachey Roll the credits …

3. Outline • What is locality • Elements of locality • Types of locality • Applications of locality • Changing locality • Concluding remarks

4. What is Locality An empirically observed phenomenon that has substantial intuitive appeal and numerous practical implications • Parachor Curve • “during any interval of execution, a program favors a subset of its pages, and this set of favored pages changes slowly”[Denning 1970]

5. Impact of Locality • Acceptable page fault rates can be achieved even when the memory allocated to a program is much less than that required to store all of its pages • Internet routers can make high speed routing decisions with very modest forwarding caches • Mobile users can work with remotely stored files even though they are located far from the file server

6. Known Aliases • The law of scattering • The principle of least effort • The 80-20 rule • Concentration of productivity • The law of diminishing returns

7. The Underlying Concept • There is a very large population of items, many more than we can manage • There is a small core of relevant items on which we can productively focus our attention • This core will continue to be relevant long enough to justify our attention

8. Locality Through the Ages • Bradford’s Law of Scattering [1934] • Zipf’s Principle of Least Effort [1949] • Many applications before we discovered it • population distribution, distribution of wealth, distribution of biological species, article distribution in journals, and word usage in natural language. • has been used to plan the location of libraries and other facilities, to model the popularity of television programs, and toorder search keys in hashing tables

9. [Bunt and Murphy 1984] [Arlitt and Williamson 1997] Elements of Locality • Concentration: • Persistence:

10. Types of Locality • Temporal Locality: • Clustering in time: items referenced in the immediate past have a high probability of being re-referenced in the immediate future • Spatial Locality: • Clustering in space: items located physically near an item referenced in the immediate past have a high probability of being re-referenced in the immediate future

11. Visualizing Locality • The memory map[Hatfield and Gerald 1971]

12. Visualizing Locality • Locality curves:An intrinsic measure[Bunt and Murphy 1984]

13. Visualizing Locality • The Zipf Distribution[Oke and Bunt 2002]

14. Applications of Locality • Memory management • File systems • Networks • Web applications

15. Applications in Memory Management • Various approaches to memory management address the principles of locality in different ways: • Recency-based replacement policies such as LRU appeal to the presence of temporal locality in a memory reference pattern. • Fetch policies that bring in adjacent items along with the item requested appeal to spatial locality. • Hit ratios are high when good choices are made.

16. Program Behaviour • We know how our programs are structured – program behaviour tells us how they execute • Considerable interest throughout the 1960’s, 70’s and 80’s • in measuring the referencing behaviour of executing programs • in developing models of this behaviour • in informing the design of approaches to manage resources

17. Applications in File Systems • Some examples: • Rodriguez-Rosell [1976] – an early empirical study of locality properties in database references • Smith [1981] – characterized long-term file reference behaviour with a focus on file migration • Williamson and Bunt [1986] – characterized short-term file reference behaviour with a focus on file system management • Majumdar and Bunt [1986 ] – evidence of phase behaviour in file reference traces • Smith [1978], Kearns and DeFazio [1983, 1989] and Verkamo [1985] – locality properties in database references

18. Applications to File Caching • Locality properties have been exploited successfully in the development of the caching techniques that are common in today’s distributed file systems. • Applications to mobile computing • Froese and Bunt [1999] explored the application of temporal locality in the design of optimistic caching techniques to support the file system activities of mobile users • Lindsey et al. [2003] reported evidence of spatial locality in the usage patterns on a campus wireless network

19. Applications in Networks • Even on early networks, measurements showed that packet traffic patterns are far from random • structural properties in the network packet behaviour reflect the particular applications, protocols, and network technologies being used • temporal correlations in the timing structure of the packet arrival process • spatial correlations in the destinations seen in consecutive (or nearly consecutive) packets on the network • These structural characteristics arise from large application-layer messages (such as those that result from file transfers), that require multiple network-layer packets for transport on the network, and from the bidirectional exchanges of data packets and acknowledgement packets typical in reliable data transfer protocols such as TCP

20. Train Spotting • Jain and Routhier [1986] proposed the packet train model for network traffic characterization based on observations of correlated arrivals • provided both a descriptive mechanism for understanding network traffic patterns and a generative mechanism for network traffic modelling • A packet train is a sequence of related network packets that traverse the network like train cars behind a steaming locomotive • characterized by short (often deterministic) inter-arrival times between the consecutive packets of a train, with slightly longer (and more random) inter-arrival times between trains

21. Locality in Web Traffic [Arlitt and Williamson 1997]

22. Application to Document Caching • Effective caching of Web documents can significantly lower the number of requests processed by a Web server, reduce the packet traffic volume on the Internet backbone, and improve the user-perceived Web response time. • Locality is the key to success • Temporal locality in Web accesses implies that Web objects accessed in the recent past are likely to be accessed again (perhaps by some other client) in the near future. • Spatial locality in Web accesses reflects Web page structure (i.e., multiple related Web objects) or geographic correlations in the surfing behaviour of Web users. • Caching can be applied at the servers, at the clients, or at proxies

23. Exploiting Locality in Web Requests • Temporal locality • To the extent that temporal locality is present, recency-based policies such as LRU can be effective for a Web cache. • Spatial locality • Pre-fetching can be successful if spatial locality is present. The persistent connection feature of HTTP exploits the fact that multiple Web objects are often requested consecutively from the same Web server. • Spatial locality can also reflect geographic correlations in the surfing behaviours of Web users, and can make geographic caching and content distribution networks effective.

24. Changing Locality Characteristics • Intentionally: Program restructuring • Inadvertently: Filtering locality

25. Program Restructuring • The basic premise • improving the locality of a program will improve its performance • A variety of methods were proposed to do this • (Replacement) strategy-based methods • Strategy-independent methods • The basic idea • divide the program’s address space into small blocks • cluster the blocks onto pages according to the dynamic reference characteristics (via restructuring graph).

26. [Bunt and Murphy 1984] [Peachey, Bunt and Colbourn 1985] Effects of Restructuring Restructuring improves locality: Restructuring improves performance:

27. Damaging Locality • The locality characteristics of an arriving request stream can be damaged in two ways: • The interleaving of independent request streams (the aggregration effect) • The removal of “hits” by upstream caches (the filtering effect) • This presents problems for traditional approaches to cache management

28. The Cache Filter Effect [Froese and Bunt 1996]

29. Concluding Remarks • Locality has returned great value • helps us to understand why certain approaches to resource management are effective (or ineffective) • inspires new approaches to managing resources • Productive application across a wide range of domains reflects that locality speaks to a basic behavioural property (“concentration of productivity”) • Should be, and will continue to be, an important research topic