LINF2345: Languages and Algorithms for Distributed Applications

1. LINF2345: Languages and Algorithms for Distributed Applications Seif Haridi Peter Van Roy S. Haridi and P. Van Roy, LINF2345

2. Overview • Course organization and objectives • Course overview • General concepts • Language-based distribution • Some advanced concepts • Motivation for distributed systems • Introduction to distributed systems • Distributed algorithms • Basic underlying concepts • Programming language, operating system, and network basics • Addressing: Internet hosts, process on a host, Web document S. Haridi and P. Van Roy, LINF2345

3. Course organizationand objectives S. Haridi and P. Van Roy, LINF2345

4. Objectives • Understand some of the fundamental aspects of distributed systems • Course in three parts: • General Concepts • Language-Based Distribution • Advanced Concepts S. Haridi and P. Van Roy, LINF2345

5. LINF2345 organization • Evaluation • Test 25% (dispensatory) around the seventh week • Final exam 50% or 75% (if redo test) • Project 25% • Web page • http://www.info.ucl.ac.be/notes_de_cours/LINF2345 • Most material will be put there • People • Teacher: Peter Van Roy (pvr@info.ucl.ac.be) • Assistant: Yves Jaradin (yjaradin@info.ucl.ac.be) S. Haridi and P. Van Roy, LINF2345

6. Project • There will be one project, to be done in groups of two, around the 10th-12th week • Practical sessions will build toward the project • Exercises to review Oz • Exercises to introduce Distributed Oz • What will the project be? • Maybe a peer-to-peer application, using our peer-to-peer middleware? S. Haridi and P. Van Roy, LINF2345

7. Lecture structure • Reminder of last lecture • Overview • Content • Summary • Reading suggestions S. Haridi and P. Van Roy, LINF2345

8. Material • Lectures are based on mainly the following • Andrew S. Tanenbaum, Maarten van Steen, Distributed systems: Principles and Paradigms, Prentice-Hall 2002. • Randy Chow and Theodore Johnson, Distributed Operating Systems & Algorithms, Addison Wesley 1997, ISBN 0-201-49838-3. • Peter Van Roy and Seif Haridi, Concepts, Techniques, and Models of Computer Programming, chapter 11, MIT Press, 2004 • Some research papers! • The books are available in the INGI library • Research papers will be hand-outs S. Haridi and P. Van Roy, LINF2345

9. Other recommended material • Coulouris, Dollimore, Kindberg, “Distributed Systems: Concepts and Design”, Addison-Wesley (3rd Edition) • M.L. Liu, Distributed Computing, Principles and Applications, Addison Wesley • Nancy Lynch, Distributed Algorithms S. Haridi and P. Van Roy, LINF2345

10. Questions and using brakes! • Please do ask questions during the lectures • repeat an explanation • give better explanation • for an example? • Please say when things go too fast! • Please say when things go too slow! S. Haridi and P. Van Roy, LINF2345

11. Background knowledge • I assume some basic knowledge about • Programming languages knowledge: C/Java • Operating systems knowledge: basic concepts • Networking: basic concepts • Algorithms and data structures • I will try to be as elementary as possible • Ask me to explain if I assume something you don’t know S. Haridi and P. Van Roy, LINF2345

12. Course overview S. Haridi and P. Van Roy, LINF2345

13. Languages and algorithms for distributed applications (1) • Part 1: general concepts of distributed systems • Inter-process communication • Processes, threads, client/servers, code migration, software agents • Naming services • Clocks and synchronization S. Haridi and P. Van Roy, LINF2345

14. Languages and algorithms for distributed applications (2) • Part 2: language-based distribution • Network-transparent distribution • Review of Oz language • Practical introduction to Distributed Oz • Open distribution • Connecting independent computations • Foundations of Distributed Oz • Language entities and their protocols • Fault tolerance S. Haridi and P. Van Roy, LINF2345

15. Languages and algorithms for distributed applications (3) • Part 3: some advanced concepts • Decentralized (peer-to-peer) systems • As contrasted with centralized (client/server) systems • Peer-to-peer library in Mozart • Introduction to the programming project • Introduction to distributed algorithms • Introduction to distributed transactions • Some systems • Grid, CORBA, Web Services, Erlang, etc. S. Haridi and P. Van Roy, LINF2345

16. Language-based distribution • How can we make distributed programming really simple? • First approximation: network transparency • Second approximation: extend with control over network communication (network awareness) • Third approximation: add failure detection ability and connection ability (fault tolerance and openness) • (Security is also important, but is outside the scope of this course) S. Haridi and P. Van Roy, LINF2345

17. Distributed Oz • Keep same language semantics while executing over a network • Can this work? Waldo et al say it can’t! • It depends on the language design • The main problems are state and concurrency • Global state is very expensive in a distributed system • A distributed system is naturally concurrent • The language has to do two things: • Make it easy to program without global state • Make concurrency easy and efficient • Both of these are hard in Java but easy in Oz • You will see how simple it really is! S. Haridi and P. Van Roy, LINF2345

18. Some advanced concepts • Distributed programs have two extremes • Centralized (client/server) • Decentralized (peer-to-peer) • We will explore this spectrum! • We will use the peer-to-peer library of Mozart • Distributed algorithms • Leader election, mutual exclusion, snapshots • Can be very subtle in a distributed setting! • Some systems and middleware • Grid, CORBA, Web Services, Erlang, etc. S. Haridi and P. Van Roy, LINF2345

19. Examples of middleware • Distributed systems • Grid: Globus • CORBA • Distributed COM • Web Services • GLOBE • Erlang • Distributed coordination-based systems • JavaSpaces • Security • E S. Haridi and P. Van Roy, LINF2345

20. Distributed algorithms • Model of distributed computations • Techniques for coordination of processes • Techniques for high availability • Fault tolerance • Reliable group communication • Distributed agreement • Techniques for scalability • Consistency models • Replication techniques S. Haridi and P. Van Roy, LINF2345

21. Motivation fordistributed systems S. Haridi and P. Van Roy, LINF2345

22. Distributed system and distributed computing • Early computing was performed on a single processor. Uni-processor computing can be called centralized computing. • A distributed system is a collection of independent computers, interconnected via a network, capable of collaborating on a task. • Distributed computing is computing performed in a distributed system. S. Haridi and P. Van Roy, LINF2345

23. Distributed systems S. Haridi and P. Van Roy, LINF2345

24. Examples of distributed systems • Network of workstations (NOW): a group of networked personal workstations connected to one or more server machines. • The Internet • An intranet: a network of computers and workstations within an organization, segregated from the Internet via a protective device (a firewall). S. Haridi and P. Van Roy, LINF2345

25. Computers in a distributed system • Workstations: computers used by end-users to perform computing (desktops or laptops) • Server machines: computers which provide resources and services • Personal Digital Assistants (PDAs): handheld computers connected to the system via a wireless communication link. S. Haridi and P. Van Roy, LINF2345

26. Centralized vs. distributed computing S. Haridi and P. Van Roy, LINF2345

27. Evolution of paradigms • Client-server: Socket API, remote method invocation • Distributed objects • Object broker: CORBA • Network service: Jini • Object space: JavaSpaces • Mobile agents • Message oriented middleware (MOM): Java Message Service • Collaborative applications S. Haridi and P. Van Roy, LINF2345

28. Cooperative distributed computing projects Cooperative distributed computing projects (also called distributed computing in some literature): these are projects that parcel out large-scale computing to workstations, often making use of surplus CPU cycles. Example: seti@home: project to scan data retrieved by a radio telescope to search for radio signals from another world. S. Haridi and P. Van Roy, LINF2345

29. Why distributed computing? • Economics: Distributed systems allow the pooling of resources, including CPU cycles, data storage, input/output devices, and services • Reliability: Distributed systems allow replication of resources and/or services, thus reducing service outage due to failures • Universality: The Internet has become a universal platform for distributed computing, i.e., it is available everywhere with substantially identical standards S. Haridi and P. Van Roy, LINF2345

30. The strengths and weaknesses of distributed computing In any form of computing, there is always a tradeoff in advantages and disadvantages Some of the reasons for the popularity of distributed computing : • The affordability of computers and availability of network access • Resource sharing • Scalability • Fault tolerance S. Haridi and P. Van Roy, LINF2345

31. The strengths and weaknesses of distributed computing The disadvantages of distributed computing: • Multiple Points of Failures: the failure of one or more participating computers, or one or more network links, can spell trouble. • Security Concerns: In a distributed system, there are more opportunities for unauthorized attack. • Programming Difficulty: Complex APIs, many issues that have to be handled at the same time S. Haridi and P. Van Roy, LINF2345

32. Introduction to distributed systems S. Haridi and P. Van Roy, LINF2345

33. What is a distributed system? S. Haridi and P. Van Roy, LINF2345

34. A distributed system • A simplified view Processor Communication Medium Process Thread Communication channel Node: processor/process S. Haridi and P. Van Roy, LINF2345

35. A distributed system • Set of computing nodes that cooperate in order to achieve a well defined goal • Nodes cooperate through communication • Communication is by message passing at the fundamental level S. Haridi and P. Van Roy, LINF2345

36. A distributed system • A distributed system is one/more applications running on a collection of independent computers that appears to its users as a single coherent system S. Haridi and P. Van Roy, LINF2345

37. What is a distributed system? • Hardware is distributed • n processing elements (processor + memory), PE • Interconnected by some network • No shared memory • Software is distributed • No centralized OS, each PE has its own copy of OS • No physically centralized file system • Means for inter-process communication is message passing at the lowest level S. Haridi and P. Van Roy, LINF2345

38. Why distributed systems? • Information exchange (collaborative work) • Resource sharing (e.g. printer, backup storage, disk units, etc.) • Resource sharing (applications, information, media, services) • Cost reduction • Increase of availability (partial failure) • Increase of performance through parallelism, ... S. Haridi and P. Van Roy, LINF2345

39. Main characteristics • No shared memory between nodes • Each node has its memory • Communication by message passing • No global clock • Each node has its own clock • Impossible for a node to obtain an instantaneous global state of the system S. Haridi and P. Van Roy, LINF2345

40. Examples ofdistributed systems • Airline reservation system • Bank automated teller machine network • CSCW (Computer Supported Cooperative Work) • Intranet • Internet: TCP/IP-based communications infrastructure • Mobile computing S. Haridi and P. Van Roy, LINF2345

41. intranet ISP backbone satellite link desktop computer: A typical portion of the Internet server: network link: S. Haridi and P. Van Roy, LINF2345

42. Desktop email server computers print and other servers Local area Web server network email server print File server other servers the rest of the Internet router/firewall A typical intranet S. Haridi and P. Van Roy, LINF2345

43. How are distributed systems built? • A number of computers connected by a network • Distribution middleware services layer that gives a uniform view of the nodes, and hides some of the network and distribution aspects • Applications on top of the middleware service layer (using a programming system or combination of programming systems) S. Haridi and P. Van Roy, LINF2345

44. Middleware view S. Haridi and P. Van Roy, LINF2345

45. Middleware view • A distributed system is often organized as a layer on the top of local operating systems S. Haridi and P. Van Roy, LINF2345

46. Goals of a distributed system • Transparency • Hide the fact the processes are resources are physically distributed • Scalability • Distributed systems should be easy to expand • Availability • Distributed systems should be continuously available • Openness • Adding new users/components into the system • Adding new functionality, incrementally and independently by independent developer teams S. Haridi and P. Van Roy, LINF2345

47. Transparency • Ideally a distributed application (system) should look like a conventional centralized system, with no distinction between local and remote resources • This is the user view • The developer view is different • Network aware, knows the cost of distribution of programming entities (e.g. objects) • Have means to control the distribution behavior S. Haridi and P. Van Roy, LINF2345

48. Transparency • Access Transparency • Hide differences in data representation and how a resource is accessed • Hides heterogeneity of underlying nodes • Location Transparency • Hide where a resource/service is located • Migration Transparency • Hides that resources/services may be moved to another location without affecting how they are accessed S. Haridi and P. Van Roy, LINF2345

49. Transparency • Relocation Transparency • Hides that a resource may be moved to another location while inuse • Failure Transparency • Hide the failure and recovery of a resource • Concurrency Transparency • Hides that a resources may be shared by a number of competitive uses/processes S. Haridi and P. Van Roy, LINF2345

50. Transparency S. Haridi and P. Van Roy, LINF2345