Code and Process Migration

1. Code and Process Migration • Motivation • How does migration occur? • Resource migration • Agent-based system • Details of process migration CS677: Distributed OS

2. Motivation • Key reasons: performance and flexibility • Process migration (aka strong mobility) • Improved system-wide performance – better utilization of system-wide resources • Examples: Condor, DQS • Code migration (aka weak mobility) • Shipment of server code to client – filling forms (reduce communication, no need to pre-link stubs with client) • Ship parts of client application to server instead of data from server to client (e.g., databases) • Improve parallelism – agent-based web searches CS677: Distributed OS

3. Motivation • Flexibility • Dynamic configuration of distributed system • Clients don’t need preinstalled software – download on demand CS677: Distributed OS

4. Migration models • Process = code seg + resource seg + execution seg • Weak versus strong mobility • Weak => transferred program starts from initial state • Sender-initiated versus receiver-initiated • Sender-initiated (code is with sender) • Client sending a query to database server • Client should be pre-registered • Receiver-initiated • Java applets • Receiver can be anonymous CS677: Distributed OS

5. Who executed migrated entity? • Code migration: • Execute in a separate process • [Applets] Execute in target process • Process migration • Remote cloning • Migrate the process CS677: Distributed OS

6. Models for Code Migration • Alternatives for code migration. CS677: Distributed OS

7. Do Resources Migrate? • Depends on resource to process binding • By identifier: specific web site, ftp server • By value: Java libraries • By type: printers, local devices • Depends on type of “attachments” • Unattached to any node: data files • Fastened resources (can be moved only at high cost) • Database, web sites • Fixed resources • Local devices, communication end points CS677: Distributed OS

8. Resource Migration Actions Resource-to machine binding • Actions to be taken with respect to the references to local resources when migrating code to another machine. • GR: establish global system-wide reference • MV: move the resources • CP: copy the resource • RB: rebind process to locally available resource Process-to-resource binding CS677: Distributed OS

9. Agents • Software agents • Autonomous process capable of reacting to, and initiating changes in its environment, possibly in collaboration • More than a “process” – can act on its own • Mobile agent • Capability to move between machines • Needs support for strong mobility • Example: D’Agents (aka Agent TCL) • Support for heterogeneous systems, uses interpreted languages CS677: Distributed OS

10. Software Agents in Distributed Systems • Some important properties by which different types of agents can be distinguished. CS677: Distributed OS

11. Distributed Scheduling: Motivation • Distributed system with N workstations • Model each w/s as identical, independent M/M/1 systems • Utilization u, P(system idle)=1-u • What is the probability that at least one system is idle and one job is waiting? CS677: Distributed OS

12. Implications • Probability high for moderate system utilization • Potential for performance improvement via load distribution • High utilization => little benefit • Low utilization => rarely job waiting • Distributed scheduling (aka load balancing) potentially useful • What is the performance metric? • Mean response time • What is the measure of load? • Must be easy to measure • Must reflect performance improvement CS677: Distributed OS

13. Design Issues • Measure of load • Queue lengths at CPU, CPU utilization • Types of policies • Static: decisions hardwired into system • Dynamic: uses load information • Adaptive: policy varies according to load • Preemptive versus non-preemptive • Centralized versus decentralized • Stability: l>m => instability, l1+l2<m1+m2=>load balance • Job floats around and load oscillates CS677: Distributed OS

14. Components • Transfer policy: when to transfer a process? • Threshold-based policies are common and easy • Selection policy: which process to transfer? • Prefer new processes • Transfer cost should be small compared to execution cost • Select processes with long execution times • Location policy: where to transfer the process? • Polling, random, nearest neighbor • Information policy: when and from where? • Demand driven [only if sender/receiver], time-driven [periodic], state-change-driven [send update if load changes] CS677: Distributed OS

15. Sender-initiated Policy • Transfer policy • Selection policy: newly arrived process • Location policy: three variations • Random: may generate lots of transfers => limit max transfers • Threshold: probe n nodes sequentially • Transfer to first node below threshold, if none, keep job • Shortest: poll Np nodes in parallel • Choose least loaded node below T CS677: Distributed OS

16. Receiver-initiated Policy • Transfer policy: If departing process causes load < T, find a process from elsewhere • Selection policy: newly arrived or partially executed process • Location policy: • Threshold: probe up to Np other nodes sequentially • Transfer from first one above threshold, if none, do nothing • Shortest: poll n nodes in parallel, choose node with heaviest load above T CS677: Distributed OS

17. Symmetric Policies • Nodes act as both senders and receivers: combine previous two policies without change • Use average load as threshold • Improved symmetric policy: exploit polling information • Two thresholds: LT, UT, LT <= UT • Maintain sender, receiver and OK nodes using polling info • Sender: poll first node on receiver list … • Receiver: poll first node on sender list … CS677: Distributed OS

18. Case Study: V-System (Stanford) • State-change driven information policy • Significant change in CPU/memory utilization is broadcast to all other nodes • M least loaded nodes are receivers, others are senders • Sender-initiated with new job selection policy • Location policy: probe random receiver, if still receiver, transfer job, else try another CS677: Distributed OS

19. Sprite (Berkeley) • Workstation environment => owner is king! • Centralized information policy: coordinator keeps info • State-change driven information policy • Receiver: workstation with no keyboard/mouse activity for 30 seconds and # active processes < number of processors • Selection policy: manually done by user => workstation becomes sender • Location policy: sender queries coordinator • WS with foreign process becomes sender if user becomes active: selection policy=> home workstation CS677: Distributed OS

20. Sprite (contd) • Sprite process migration • Facilitated by the Sprite file system • State transfer • Swap everything out • Send page tables and file descriptors to receiver • Demand page process in • Only dependencies are communication-related • Redirect communication from home WS to receiver CS677: Distributed OS

21. Today: Naming • Names are used to share resources, uniquely identify entities and refer to locations • Need to map from name to the entity it refers to • E.g., Browser access to www.cnn.com • Use name resolution • Differences in naming in distributed and non-distributed systems • Distributed systems: naming systems is itself distributed • How to name mobile entities? CS677: Distributed OS

22. Example: File Names • Hierarchical directory structure (DAG) • Each file name is a unique path in the DAG • Resolution of /home/steen/mbox a traversal of the DAG • File names are human-friendly CS677: Distributed OS

23. Resolving File Names across Machines • Remote files are accessed using a node name, path name • NFS mount protocol: map a remote node onto local DAG • Remote files are accessed using local names! (location independence) • OS maintains a mount table with the mappings CS677: Distributed OS

24. Name Space Distribution • Naming in large distributed systems • System may be global in scope (e.g., Internet, WWW) • Name space is organized hierarchically • Single root node (like naming files) • Name space is distributed and has three logical layers • Global layer: highest level nodes (root and a few children) • Represent groups of organizations, rare changes • Administrational layer: nodes managed by a single organization • Typically one node per department, infrequent changes • Managerial layer: actual nodes • Frequent changes • Zone: part of the name space managed by a separate name server CS677: Distributed OS

25. Name Space Distribution Example • An example partitioning of the DNS name space, including Internet-accessible files, into three layers. CS677: Distributed OS

26. Name Space Distribution • A comparison between name servers for implementing nodes from a large-scale name space partitioned into a global layer, as an administrational layer, and a managerial layer. • The more stable a layer, the longer are the lookups valid (and can be cached longer) CS677: Distributed OS

27. Implementing Name Resolution • Iterative name resolution • Start with the root • Each layer resolves as much as it can and returns address of next name server CS677: Distributed OS

28. Recursive Name Resolution • Recursive name resolution • Start at the root • Each layer resolves as much as it can and hands the rest to the next layer CS677: Distributed OS

29. Which is better? • Recursive name resolution puts heavy burden on gobal layer nodes • Burden is heavy => typically support only iterative resolution • Advantages of recursive name resolution • Caching possible at name servers (gradually learn about others) • Caching improves performance • Use time-to-live values to impose limits on caching duration • Results from higher layers can be cached for longer periods • Iterative: only caching at client possible CS677: Distributed OS

30. Communication costs • The comparison between recursive and iterative name resolution with respect to communication costs • Recursive may be cheaper CS677: Distributed OS

31. The DNS Name Space • The most important types of resource records forming the contents of nodes in the DNS name space. CS677: Distributed OS

32. DNS Implementation • An excerpt from the DNS database for the zone cs.vu.nl. CS677: Distributed OS

33. X.500 Directory Service • OSI Standard • Directory service: special kind of naming service where: • Clients can lookup entities based on attributes instead of full name • Real-world example: Yellow pages: look for a plumber CS677: Distributed OS

34. The X.500 Name Space (1) • A simple example of a X.500 directory entry using X.500 naming conventions. CS677: Distributed OS

35. The X.500 Name Space (2) • Part of the directory information tree. CS677: Distributed OS

36. LDAP • Lightweight Directory Access Protocol (LDAP) • X.500 too complex for many applications • LDAP: Simplified version of X.500 • Widely used for Internet services • Application-level protocol, uses TCP • Lookups and updates can use strings instead of OSI encoding • Use master servers and replicas servers for performance improvements • Example LDAP implementations: • Active Directory (Windows 2000) • Novell Directory services • iPlanet directory services (Netscape) • Typical uses: user profiles, access privileges, network resources CS677: Distributed OS