Distributed Shared Memory in Operating Systems: Advantages and Implementation Issues

1. Outline • Announcement • Distributed file systems – continued • Distributed shared memory

2. Announcement • Programming project #1 demonstration • You can come on April 1 or April 3 from 9:30-11:00 in my office to do the demonstration • Or you can set up an appointment with me • You need to have the demonstration before April 17 • What you need to prepare • I may ask you some questions based on your report and your implementation • I will ask you to demonstrate at least one configuration with and without mutual exclusion COP 5611 - Operating Systems

3. Distributed Shared Memory • Distributed computing is mainly based on the message passing model • Client/server model • Remote procedure calls • Distributed shared memory • is a resource management component that implements the shared memory model in distributed systems, where there is no physically shared memory COP 5611 - Operating Systems

4. Memory Hierarchies COP 5611 - Operating Systems

5. Virtual memory COP 5611 - Operating Systems

6. Page Table Mapping COP 5611 - Operating Systems

7. Distributed Shared Memory – cont. • This is a further extension of the virtual memory management on a single computer • When a process accesses data in the shared address space, a mapping manager maps the shared memory address to the physical memory, which can be local or remote COP 5611 - Operating Systems

8. Distributed Shared Memory – cont. COP 5611 - Operating Systems

9. Distributed Shared Memory – cont. A mapping manager is a layer of software to map addresses in user space into the physical memory addresses COP 5611 - Operating Systems

10. Distributed Shared Memory – cont. • With DSM, application programs can access data in the shared space as they access data in traditional virtual memory • Mapping manager can move data among the local main memory, local disk, and another nodes COP 5611 - Operating Systems

11. Distributed Shared Memory – cont. • Advantages of DSM • It is easier to design and develop algorithms with DSM than message passing models • DSM allows complex structures to be passed by reference, simplifying the distributed application development • DSM can cut down the overhead of communication by exploiting locality in programs • DSM can overcome some the architectural limitations of shared memory machines COP 5611 - Operating Systems

12. Distributed Shared Memory – cont. • Central implementation issues in DSM • How to keep track of the location of remote data • How to overcome the communication delays and high overhead associated with communication protocols • How to improve the system performance COP 5611 - Operating Systems

13. The Central-Server Algorithm • A central server maintains all the shared data • It serves the read requests from other nodes or clients by returning the data items to them • It updates the data on write requests by clients COP 5611 - Operating Systems

14. The Central-Server Algorithm– cont. COP 5611 - Operating Systems

15. The Migration Algorithm • In contrast to the central-server algorithm, data in the migration algorithm is shipped to the location of data access request • Subsequent accesses can then be performed locally • Thrashing can be a problem COP 5611 - Operating Systems

16. The Migration Algorithm – cont. COP 5611 - Operating Systems

17. The Migration Algorithm – cont. COP 5611 - Operating Systems

18. The Read-Replication Algorithm • The read-replication algorithm allows multiple node to have read access or one node to have read-write access COP 5611 - Operating Systems

19. The Read-Replication Algorithm– cont. COP 5611 - Operating Systems

20. The Read-Replication Algorithm– cont. COP 5611 - Operating Systems

21. The Full-Replication Algorithm • This is a further extension of the read-replication algorithm • It allows multiple nodes to have both read and write access to shared data blocks • Data consistency issue COP 5611 - Operating Systems

22. The Full-Replication Algorithm– cont. COP 5611 - Operating Systems

23. Memory Coherence • Memory consistency models • Many consistency models have been proposed • These models differ in • Restrictiveness • Implementation complexity • Ease of programming • Performance COP 5611 - Operating Systems

24. Memory Coherence – cont. • Strict consistency • Any read on a data item x returns a value corresponding to the result of the most recent write on x • Sequential consistency • The result of any execution is the same as if the operations by all processes on the data store were executed in some sequential order and the operations of each individual process appear in this sequence in the order specified by its program COP 5611 - Operating Systems

25. Memory Coherence – cont. COP 5611 - Operating Systems

26. Memory Coherence – cont. COP 5611 - Operating Systems

27. Memory Coherence – cont. COP 5611 - Operating Systems

28. Memory Coherence – cont. • General consistency • All the copies of a memory location eventually contain the same data when all the writes issued by every process have completed • Causal consistency • Writes that are causally related must be seen by all the processes in the same order; concurrent writes may be seen in a different order on different machines • Processor consistency is related to causal consistency but with one more relaxation COP 5611 - Operating Systems

29. Memory Coherence – cont. • Weak consistency • Accesses to synchronization variables are sequentially consistent • Before a synchronization access, all previous regular data accesses must be completed • Before a regular data access, all previous synchronization accesses must be completed COP 5611 - Operating Systems

30. Memory Coherence – cont. COP 5611 - Operating Systems

31. Memory Coherence – cont. • Release consistency • Synchronization operations are broken down into acquire and release operations • Before a read or write operation on shared data is performed, all previous acquires done by the process must have completed successfully • Before a release is allowed, all previous reads and write done by the process must have been completed COP 5611 - Operating Systems

32. Memory Coherence – cont. • Coherence protocols • A protocol to keep replicas coherent • Write-invalidate protocol • A write to a shared data causes the invalidation of all copies except the one where the write occurs • Write-update protocol • A write to a shared data causes all copies of that data to be updated COP 5611 - Operating Systems

33. Memory Coherence – cont. COP 5611 - Operating Systems

34. Memory Coherence – cont. • Type-specific memory coherence • Exploiting application specific semantics information • The system uses different kinds of coherence mechanisms for different kinds of classes • Write-one objects • Write-many objects • Read-mostly objects COP 5611 - Operating Systems

35. Design Issues • Granularity • The size of the shared memory unit • Advantages of large sizes • A page size that is a multiple of the size provided by the underlying memory management system allows for the integration of DSM and the memory management system • Better utilization of locality of reference • Disadvantages • The greater the chance for contention • False sharing COP 5611 - Operating Systems

36. False Sharing COP 5611 - Operating Systems

37. Design Issues – cont. • Page replacement • Traditional methods such as LRU can not be used directly • Page access modes must be taken into consideration • For example, private pages may be replaced before shared pages COP 5611 - Operating Systems

38. Case Studies • IVY • Integrated Shared Virtual Memory at Yale • The granularity is a page • The address space is divided into shared virtual memory address space and private space • The coherence protocol • Write fault handling • Read fault handing COP 5611 - Operating Systems

39. IVY – cont. • Coherence protocol • The centralized manager scheme • Fixed distributed manager scheme • Dynamic distributed manager scheme COP 5611 - Operating Systems

40. IVY – cont. COP 5611 - Operating Systems

41. IVY – cont. • Double fault • Memory allocation • Process synchronization COP 5611 - Operating Systems

42. Case Studies – cont. • Mirage • Thrashing control • Clouds • The RA kernel • Distributed shared memory controller COP 5611 - Operating Systems

43. Summary • Distributed shared memory tries to provide an easy-to-use interface for distributed applications • Distributed programs access data in the shared address space as they access data in local memory • However, the performance is a critical issue • Replication is used to reduce the high cost of communications • Memory coherence is however difficult and expensive to achieve COP 5611 - Operating Systems