Communication I

1. Communication I CSE 5306 Lecture Quiz due 16 June 2014 at 5 PM

2. Communication • Nondistributed systems’ processes exchange low-level messages via shared memory. • In general the 7-layer Open Systems Interface (OSI) network architecture is “too difficult” (i.e., low performance, not transparent) to use for distributed systems’ low-level message passing. • But three protocols help: remote procedure calls (RPC) for client-server apps, email-like message-oriented middleware (MOM) and streaming for continuously flowing communications.

3. Layered Protocols • To talk to another distributed system via the OSI-model, you would have to obey its protocols at every level: application, presentation, session, transport, network, data link and physical. • Telephone-like connection-oriented protocols require making a connection before and breaking it after the message exchange. Snail mail-like connectionless protocols don’t require that. • Your standard top-level message with its header gets embedded in standard lower-level messages with their headers. As these ascend toward your friend’s top level, the headers get stripped off.

4. Lower Level Protocols • The physical layer describes computer connectors and the voltage and frequency of bits. • The data link layer gathers bits into a frame, with start and stop bits, a sequence number (in case the frames pursue multiple routes and arrive out of order) and a error detecting checksum. • The network layer generally uses the connectionless Internet Protocol (IP), which deals in packets.

5. Transport Protocols 4. The transport layer generally uses the connection-oriented Transmission Control Protocol (TCP), which ensures end-to-end reliable packet deliveries. • It puts all arriving packets in proper sequence, and even asks that damaged or missing ones be sent again. If you don’t need a connection, use the simpler Universal Datagram Protocol (UDP). • A Real-Time Transport Protocol (RTP) also has been proposed.

6. Higher Level Protocols 7. Your process lives in the application layer. • An example is HyperText Transfer Protocol (HTTP), by which browsers and Web servers handle Web page transfers. 5-6. The session (who is speaking) and representation (ASCII vs. EBCDIC characters) layers are never used. • Middleware can replace these nicely.

7. R U O K ? • If distributed systems can’t share memory, and the OSI network architecture is “too difficult,” how candistributed systems communicate? • Remote procedure calls for client-server apps. • Email-like message-oriented middleware. • Streaming a continuous flow of communications. • All of the above. • None of the above.

8. R U O K ? 2. What is so “difficult” about using the OSI model for distributed systems’ communications? • Obeying its protocols at all 7 levels. • Unlike snail mail connectionless protocols, the transport layer’s telephone-like connection-oriented protocol would require making a connection before and breaking it after every message exchange. • On its way down to the network, your application layer message with its header would be inserted into all 6 lower-level messages with their headers, and all of those headers would get stripped off as the message ascends from the network to your friend’s application level. • All of the above. • None of the above.

9. R U O K ? 3. Which two OSI layers are seldom used, and why? • OSI’s transport layer assurance of end-to-end packet deliveries is unnecessary. And RPC’s IDL does a better job of specifying message formats than OSI’s representation layer ever did. • Specifications of bits and wires in OSI’s physical layer are irrelevant to modern distributed systems’ low-level communications. And OSI’s session layer does not need to tell whose turn it is to talk, in the course of an RPC. • RPC’s IDL does a better job of specifying message formats than the OSI’s representation layer ever did. And OSI’s session layer does not need to tell whose turn it is to talk, in the course of an RPC. • OSI’s transport layer assurance of end-to-end packet deliveries is unnecessary. And specifications of bits and wires in OSI’s physical layer are irrelevant to modern distributed systems’ low-level communications. • Specifications of bits and wires in OSI’s physical layer are irrelevant to modern distributed systems’ low-level communications. And RPC’s IDL does a better job of specifying message formats than the OSI’s representation layer ever did.

10. R U O K ? Match the following OSI network layers with their definitions below. 4. Application __ 5. Transport __ 6. Network __ 7. Data link __ 8. Physical __ • Uses connection-oriented TCP to ensure end-to-end reliable packet deliveries. • Describes computer connectors and the voltage and frequency of bits. • Gathers bits into a frame, with start and stop bits, a sequence number and an error detecting checksum. • Uses connectionless IP to deal with packets. • The users’ processes.

11. Middleware Protocols • Middleware can authenticate (potential) users and processes. • It can enforce mutual exclusion (mutex) of non-reentrant resources. • Middleware can transparently call remote procedures (RPC) or stream your real-time media. • Middleware’s reliable multicasting service can guarantee scalability, if it knows the application’s requirements. • Fault-tolerant transaction apps can rely on middleware to ensure that a task is cancelled, if any one of its essential component processes do not finish (see “atomic,” p.21).

12. Types of Communication • When you open a Gmail or Yahoo account, the server sends your client terminal a user agent to help you compose, send and receive emails. • You send a discrete email and logout, knowing your email will linger (persistent) at the server until the addressee picks it up. • In phone-like transient communications, the transportlayer drops all stored materials, when you hang up. Streaming is like this. • Mail is asynchronous. Synchronous communications put you on hold (I/O process blocking) till: 1) your friend answers, 2) she receives your voicemail or 3) she calls you back.

13. Remote Procedure Call • When distributed systems’ processes use send and receive procedures to explicitly exchange messages, their communications are not concealed, a clear violation of access transparency. • When process A calls procedure B on another machine, parameters pass transparently both ways. • The other machine crashing and differences in the machines’ address spaces or OSes can cause problems.

14. Conventional Procedure Call • In a single machine, a procedure call is simple: count =read( filename, buffer, nbytes ); • The call pushes all three arguments onto the stack after the function’s local variables and the caller’s return address. • filename and nbytes are passed by value, but buffer (a local array pointer) is passed by reference. • read is an interface between the calling program and OS.

15. Client and Server Stubs • The client procedure calls the client stub in the normal way. • The client stub builds a message and calls the local operating system. • The client’s OS sends the message to the remote OS. • The remote OS gives the message to the server stub. • The server stub unpacks the parameters and calls the server. • The server does the work and returns the result to the stub. • The server stub packs it in a message and calls its local OS. • The server’s OS sends the message to the client’s OS. • The client’s OS gives the message to the client stub. • The stub unpacks the result and returns to the client.

16. R U O K ? 9. Which of the following communication protocols does middleware provide for distributed systems? • It authenticates users and processes, and it enforces mutual exclusion in non-reentrant resources. • It can transparently call remote procedures (RPC) or stream your real-time media. • Its reliable multicasting service can guarantee scalability, if it knows the application’s requirements. • Fault-tolerant transaction apps can rely on middleware to ensure that a task is cancelled, if any one of its “atomic” processes does not finish. • All of the above.

17. R U O K ? 10. Which of the following accurately describe middleware communications in behalf of distribute systems? • When you open a Gmail or Yahoo account, the server sends your client terminal a user agent, who helps you compose, send and receive emails. • You send an email and logout, fully confident that your email will stay at the server to be picked up later by the addressee. • The transport layer drops all stored materials, after middleware completes its streaming protocol. • If desired, middleware can use I/O process blocking to put you on hold until: 1) your friend answers, 2) she receives your voicemail or 3) she calls you back. • All of the above.

18. R U O K ? 11. Which of the problems below does RPC solve? • An app’s procedure call does not bind, if the procedure name is misspelled. • When distributed systems’ processes use send and receive procedures to explicitly exchange messages, their communications are not concealed, which violates access transparency. • The other machine may crash. • All of the above. • None of the above.

19. R U O K ? 12. Which of the following accurately characterize a local procedure call? • In the application layer, it looks just like a remote procedure call; e.g., count = read( filename, buffer, nbytes ); • The call pushes all three arguments onto the stack, after the function’s local variables and the caller’s return address. • The parameters filename and nbytes are passed by value, and buffer (a local array pointer) is passed by reference. • The function name “read” is an interface between the calling program and OS. • All of the above.

20. R U O K ? Arrange the following steps of a remote procedure call in the proper order. 13. The server stub unpacks the parameters and calls the server, which does the work, returning its results to the stub. __ 14. The client procedure calls the client stub, which builds a message and calls the local operating system. __ 15. The server stub packs it in a message and calls its local OS, which sends the message to the client’s OS. __ 16. The client’s OS gives the message to the client stub, which unpacks the result and returns it to the client. __ 17. The client’s OS sends the message to the remote OS, which gives the message to the server stub. __

21. Passing Parameters by Value • The IBM PCs and IBM mainframes represent characters differently; i.e., ASCII and EBCDIC. • Intel and SPARC order data bytes differently; i.e., little- (lsb right) and big-endian (msbright). (Strings in both read left to right.) • RPC’s client stub and server stub must translate.

22. Passing Parameters by Reference • The client’s pointer is meaningless in the server’s address space. • Instead of passing a pointer, the client stub passes the actual buffer full of data. • Or the client stub is passes the size of the buffer, if it wants the server to return data.

23. Parameter Specification and Stub Generation • An Interface Definition Language (IDL) fully describes the client stub’s represen-tations for the server stub: • Character code used in procedure name; e.g., ASCII. • Types of all parameters; e.g., INT32. • Floating point representation; e.g., IEEE Std 754. • Data byte sequence; e.g., little endian. • Network; e.g., connection-oriented TCP/IP. foobar( char x; float y; int z[5] ) { … };

24. Asynchronous RPC • No server response is required when transferring money, adding to a database, starting a remote service or batch processing. • In an asynchronous RPC, the server stub acknowledges the client stub’s request and releases the client to resume execution. • In a deferred synchronous RPC, the server also calls the client back later with results. • In a one-way RPC, the client does not even wait for the server’s acknowledgement, but what if the client’s request was not received?

25. R U O K ? 18. Which of the following problems arise in RPCs’ passing parameters by value? • Characters may be represented differently. • A client’s array pointer is meaningless in the remote server’s address space. • Data bytes may be arranged in different orders. • All of the above. • Both a and c above.

26. R U O K ? 19. Which of the following problems arise in RPCs’ passing parameters by reference? • Characters may be represented differently. • A client’s array pointer is meaningless in the remote server’s address space. • Data bytes may be arranged in different orders. • All of the above. • Both a and c above.

27. R U O K ? 20. Which of the following is specified by the middleware’s Interface Definition Language? • The character code used in all names. • Types of all parametersand the floating point representation standard. • Data byte sequence. • Network protocols. • All of the above.

28. R U O K ? 21. Which of the following accurately characterizes RPC dialogs? • In an asynchronous RPC, the server stub immediately acknowledges the client stub’s request and releases the client to resume execution. • In a deferred synchronous RPC, the server also calls back the previously released client with results later on. • In a one-way RPC, the client does not even wait for the server’s acknowledgement, fully confident that the client’s request (e.g., money transfer, database entry, starting a remote service, batch processing) was received. • All of the above. • None of the above.

29. Introduction to DCE • The Distributed Computing Environment (DCE) is a middleware that runs between distributed applications and a network operating system. • All client-server communications are via RPCs. • Many servers are within DCE: • Distributed file service accesses files worldwide. • Directory service accesses global printers, etc. • Security service authorizes resource users. • Distributed time service synchronizes all clocks.

30. Goals of DCE RPC • Give simple, legacy, local procedures global reach. • Hide all bindings and communications details. • Client and server can be… • Written in different languages; e.g., Java, C. • Run on different hardware; e.g., Intel, IBM. • Run with different OSes; e.g., UNIX, VMS, Windows. • Using different data representations; e.g., ASCII. • Running different network protocols; e.g., TCP/IP. • …entirely without user’s intervention/knowledge.

31. Writing a Client and Server • Write IDL function declarations (like C function prototypes). • Marshall parameters;i.e., pack them into messages. • Ask uuidgen for globally unique interface names (32-hex digits). • Add to client stub the names of all remote procedures to be called. • Add to server stub all required runtime system procedure names. • IDL compiler generates header file and client and server stubs. • Write the client and server apps.

32. Binding a Client to a Server • A server must be registered before clients call. • DCE daemons have (server, end point) tables. • A new server provides lists of its end points, protocols and services to its DCE daemons. • It registers network addresses of its machine and itself with DCE’s directory service. • Then server and client may bind.

33. Performing an RPC • App does RPC. • Client stub marshals parameters. • Runtime library sends message ro server stub. • Server stub unmarshals parameters and calls server. • Reply returns via reverse route. • If remote procedure is marked “idempotent,” process can repeat in case server crashes.

34. R U O K ? 22. Which of the following servers already are available within the Distributed Computing Environment (DCE) middleware? • A worldwide distributed file service. • A worldwide directory service for printer access. • A security service to authorize resource users. • A distributed time service to synchronize the clocks of every DCE-administered middleware. • All of the above.

35. R U O K ? 23. Which of the following were goals of those who created DCE’s RPC service? • Give global reach to simple, legacy, local procedures. • Make all bindings and communications details transparent. • Minimize users’ interventions and required knowledge. • All of the above. • None of the above.

36. R U O K ? Arrange the following steps for writing a DCE RPC client and server in their proper order. 24. Write all IDL function declarations. __ 25. After the IDL compiler generates header file and client and server stubs, write all client and server apps. __ 26. Marshall parameters; i.e., pack them into messages. __ 27. Add to client stub the names of all remote procedures to be called, and add to server stub all required runtime system procedure names. __ 28. Get globally unique interface names (32-hex digits) from uuidgen. __

37. R U O K ? Arrange in proper order the following steps for binding a DCE RPC client to its server. 29. Client tests all of the above by doing an RPC. __ 30. Register service with remote directory server. __ 31. Client asks DCE daemon for server’s end point. __ 32. Register server end point with local DCE daemon. __ 33. Client looks up server on directory server. __

38. R U O K ? Arrange in proper order the following steps for performing an RPC. 34. Client stub marshals parameters. __ 35. App issues RPC to its local OS. __ 36. Server’s reply returns to app via reverse route. __ 37. Runtime library sends message to server stub. __ 38. Server stub un-marshals parameters and calls server. __

39. Message-Oriented Communication • Remote procedure calls and remote object invocations enhance access transparency by hiding distributed systems’ communications. • But sometimes your friend is not available when you call. • Blocking your progress till she responds would reduce your performance. • RPCs and ROIs are synchronous—sometimes we need the asynchrony of emails. • Plugging directly into the transport layer’s socket enables a distributed system to do message-oriented transient communications.

40. Berkeley Sockets • A “socket” is an (abstract) communications end point, where an application can directly write to and read from another networked application. • Berkeley UNIX offers a set of primitives to make use of the TCP/IP’s messaging protocols.

41. TCP/IP vs. MPI • Sockets reduce multicomputer performance: • Simple send and receive lack access transparency. • General purpose TCP/IP protocols are too slow. • MPI handles advanced multicomputer features: • Suited for parallel apps’ transient* communications. • Small group and process ID numbers for processes. • Flow-control buffering and precise synchronization. • Does not automatically recover from process crashes. • Many efficient application-level primitives. • Makes the many proprietary libraries compatible. • All proprietary library features—bloated or highly optimal performance? *Message stored only while sender and receiver are both running.

42. The Message-Passing Interface (MPI) • MPI_bsend enables transient asynchronous communications by asking OS to hold message till receiver sends MPI_recv. (Can ask if done.) • MPI_send blocks sender till receiver calls. • MPI_sendrecv is a normal RPC. • MPI_irecv says asynchronous receiver welcomes messages.

43. R U O K ? 39. Why NOT use remote procedure calls and remote object invocations for transparent message-oriented transient communications among distributed systems? • Sometimes your friend is not available when you call. • Blocking your progress till she responds would reduce your performance. • RPCs and ROIs are synchronous—sometimes we need the asynchrony of emails. • All of the above. • None of the above.

44. R U O K ? 40. Why NOT use TCP/IP for transparent message-oriented transient communications among distributed systems? • Sockets (i.e., communications end points) compromise highly optimized multicomputer performance. • Simple send and receive primitives lack access transparency. • General purpose TCP/IP protocols are too slow. • All of the above. • None of the above.

45. R U O K ? 41. Why does MPI serve the high-performance multicomputer best? • Its efficient primitives are perfect for parallel apps’ transient communications. • Its group and process IDs are conveniently limited in scope. • It enables flow-control buffering and precise synchronization. • It squeezes all highly-optimized proprietary libraries into one portable protocol. • All of the above.

46. R U O K ? Match the following primitives with their definitions below. 42. Bind __ 43. MPI_bsend __ 44. MPI_irecv __ 45. MPI_send __ 46. MPI_sendrecv__ • Enable transient asynchronous communications by asking OS to hold message till receiver sends MPI_recv. • Block sender till receiver calls. • Like a normal RPC. • An asynchronous receiver inviting other’s messages. • Attach a local address to a TCP/IP socket.

47. Message-Oriented Persistent Communication • Email-like message queuing systems may store messages for several days. • They may transmit those messages, even while sender and receiver are both inactive. • Message-oriented middleware (MOM) is best qualified to support persistent asynchronous communications. • MOM calls upon the message-passing interface (MPI) to complete the miracle.

48. Message-Queuing Model • Given proper addresses of the sender’s queue and the receiver’s queue, a message may be routed through any number of intermediate server queues on its way to the receiver, perhaps thousands of miles away. • There is no guarantee when or even if the message will be delivered. • The fact that fragments of the message may travel different routes is hidden from the users’ applications. • MPI’s put, get, poll and notify primitives handle the process completely.

49. General Architecture of a Message-Queuing System • The sender’s source queue is on her machine—or on her LAN’s message server, sent there via RPC. • Likewise, her destination queue is on her machine—or on her LAN’s message server, received from there via RPC. • Every queued message contains its ultimate destination address. • With overlay network databases of all interconnected queues (including Internet DNS names), queue managers collectively relay messages among their queues. • Each router knows all regional queues, but each queue manager only knows her nearest router. • “Gateways” relays reformat messages for their users; others multicast to all.

50. Message Brokers • Analogous to the gateway router, a “message broker” is an application (i.e., not an integral part of the queuing system) that translates a received message of type T1 to type T2, which its user understands. • The user’s organization uses sophisticated tools (e.g., conversion rules and programs) to program its own message broker. • T1 and T2 may be two different database or word processor formats; e.g., MATLAB “imports” Excel. • A publish/subscribe message broker delivers only messages of interest to the subscribing receiver.