Understanding XDR Data Representation: A Comprehensive Guide

1. XDRExternal Data Representation Process A Process B XDR Encode/Decode XDR Encode/Decode Transport Transport

2. XDR as a case study • Sun RPC uses XDR. • A good example of a “layer”. • Interesting API. • Powerful paradigm for creation of complex data structures.

3. XDR • XDR provides a service associated with the OSI Presentation Layer. • Common data representation • Library (not part of the O.S.). • Easy to port to new architectures. • Independence from transport layer.

4. Data Conversion • Asymmetric Data Conversion • client always converts to the server’s data representation. • Symmetric Data Conversion • both client and server convert to some standard representation.

5. XDR Data Types • boolean • char • short • int • long • float • double • enumeration • structure • string • fixed length array (1-D) • variable length array (1-D) • union • opaque data

6. Implicit vs. Explicit Typing • Explicit Typing means that each piece of data includes information about the type. • Implicit typing means that the sender and receiver must agree on the order and type of all data. • XDR uses Implicit Typing

7. XDR Programming • Most XDR implementations are based on a buffer paradigm. • A buffer is allocated that is large enough to hold an entire message. • Individual data items are added to the buffer one at a time. • XDR buffers can be attached to a file, pipe, socket or memory.

8. Conversion Terminology • Converting from local representation to XDR representation is called Encoding. • Converting from XDR representation to local representation is called Decoding. Sender Receiver ENCODE DECODE XDR

9. XDR Buffer Creation • There are a number of buffer creation functions in the XDR library. • xdrmem_create • destination for encoding -or- source for decoding is a chunk of memory. • xdrstdio_create • destination for encoding -or- source for decoding is a file descriptor.

10. XDR filters • The XDR library includes an extensive set of filters that perform encoding/decoding operations. • Each XDR stream includes an attribute that determines the specific operation that will be performed by a filter (encoding or decoding).

11. XDR Filters • The filter to encode/decode an integer is called xdr_int: bool_t xdr_int( XDR *xdrs, int *ip); • the return value indicates success or failure. • xdrs is a pointer to an XDR stream • ip is a pointer to an integer.

12. xdr_int() • If the XDR stream operation is XDR_ENCODE int count; XDR *xstream; xdr_int(xstream, &count); will convert (encode) the value of count to the integer representation used by XDR (big-endian) and put the result on the XDR stream.

13. xdr_int() • If the XDR stream operation is XDR_DECODE int count; XDR *xstream; xdr_int(xstream, &count); will get an XDR integer from the stream, convert (decode) the value to local integer representation and put the result in count.

14. xdr_int() Source Encodes int count; xdr_int(xstream,&count); Destintation decodes int count; xdr_int(xstream,&count);

15. Complex Data Filters • The XDR library includes a set of filters designed to translate complex C data structures to and from XDR representation. • Many of these filters make use of the simpler filters to convert individual components.

16. xdr_array() • The xdr_array() filter provides support for encoding/decoding a variable length array. bool_t xdr_array( XDR *xdrs, char *arrp, u_int *sizep, u_int maxsize, u_int elsize, xdrproc_t elproc); sizep is a pointer to the size of the array. elsize is the size of each element (in bytes). elproc is a pointer to a function that can encode/decode individual array elements.

17. xdr_array() Source Array -> 0 1 2 3 elproc() Destination Array -> 0 1 2 3

18. Inside xdr_array() encode/decode the number of elements in the array xdr_int(xdrs,&sizep); for (i=0;i<sizep;i++) elproc(xdrs,arrp+elsize*i); encode/decode each array element.

19. xdr_string() • the string conversion filter is a little different since XDR strings have a maximum size. bool_t xdr_string( XDR *xdrs, char *string, u_int maxsize);

20. Problem!! • We want to send an array of strings between processes. • What is the problem (using xdr_array)? • What is a possible solution?

21. Distributed Program Design Typical Sockets Approach • Communication-Oriented Design • Design protocol first. • Build programs that adhere to the protocol. • Application-Oriented Design • Build application(s). • Divide programs up and add communication protocols. RPC

22. RPCRemote Procedure Call • Call a procedure (subroutine) that is running on another machine. • Issues: • identifying and accessing the remote procedure • parameters • return value

23. Server Client blah, blah, blah bar = foo(a,b); blah, blah, blah int foo(int x, int y ) { if (x>100) return(y-2); else if (x>10) return(y-x); else return(x+y); } protocol

24. Sun RPC • There are a number of popular RPC specifications. • Sun RPC (ONC RPC) is widely used. • NFS (Network File System) is RPC based. • Rich set of support tools.

25. Sun RPC Organization Remote Program Shared Global Data Procedure 1 Procedure 2 Procedure 3

26. Procedure Arguments • To reduce the complexity of interface specification Sun RPC includes support for a single argument to a remote procedure.* • Typically the single argument is a structure that contains a number of values. • * Newer versions can handle multiple args.

27. Procedure Identification • Each procedure is identified by: • Hostname (IP Address) • Program identifier (32 bit integer) • Procedure identifier (32 bit integer)

28. Procedure Identification • Each procedure is identified by: • Hostname (IP Address) • Program identifier (32 bit integer) • Procedure identifier (32 bit integer) • Program Version identifier • for testing and migration.

29. Program Identifiers • Each remote program has a unique ID. • Sun divided up the IDs: 0x00000000 - 0x1fffffff 0x20000000 - 0x3fffffff 0x40000000 - 0x5fffffff 0x60000000 - 0xffffffff Sun Sys Admin Transient Reserved

30. Procedure Identifiers &Program Version Numbers • Procedure Identifiers usually start at 1 and are numbered sequentially • Version Numbers typically start at 1 and are numbered sequentially.

31. Iterative Server • Sun RPC specifies that at most one remote procedure within a program can be invoked at any given time. • If a 2nd procedure is called the caller blocks until the 1st procedure has completed. • This is useful for applications that may share data among procedures. • Example: database - to avoid insert/delete/modify collisions.

32. Communication Semantics • To act like a local procedure (exactly one invocation per call) - a reliable transport (TCP) is necessary. • Sun RPC does not support reliable call semantics. • At Least Once Semantics • Zero or More Semantics If the procedure returns No reply

33. Dynamic Port Mapping • Servers typically do not use well known protocol ports. • Clients known the Program ID (and host). • A port lookup service runs on each host that contains RPC servers. • RPC servers register themselves with the port mapper

34. The portmapper • Each system which will support RPC servers runs a port mapper server that provides a central registry for RPC services. • Servers tell the port mapper what services they offer. • Clients ask a remote port mapper for the port number corresponsing to Remote Program ID.

35. More on the portmapper • The portmapper is itself an RPC server! • The portmapper is available on a well-known port (111).

36. Sun RPC Programming • The RPC library is a collection of tools for automating the creation of RPC clients and servers. • RPC clients are processes that call remote procedures. • RPC servers are processes that include procedure(s) that can be called by clients.

37. RPC Programming • RPC library • XDR routines • RPC run time library • call rpc service • register with portmapper • dispatch incoming request to correct procedure • Program Generator

38. RPC Run-time Library • High- and Low-level functions that can be used by clients and servers. • High-level functions provide simple access to RPC services.

39. High-level Client Library int callrpc( char *host, u_long prognum, u_long versnum, u_long procnum, xdrproc_t inproc, char *in, xdrproc_t outproc, char *out);

40. High-Level Server Library int registerrpc( u_long prognum, u_long versnum, u_long procnum, char *(*procname)() xdrproc_t inproc, xdrproc_t outproc);

41. High-Level Server Library (cont.) void svc_run(); • svc_run() is a dispatcher. • A dispatcher waits for incoming connections and invokes the appropriate function to handle each incoming request.

42. High-Level Library Limitation • The High-Level RPC library calls support UDP only (no TCP). • You must use lower-level RPC library functions to use TCP. • The High-Level library calls do not support any kind of authentication.

43. Low-level RPC Library • Full control over all IPC options • TCP & UDP • Timeout values • Asynchronous procedure calls • Multi-tasking Servers • Broadcasting

44. RPCGEN • There is a tool for automating the creation of RPC clients and servers. • The program rpcgen does most of the work for you. • The input to rpcgen is a protocol definition in the form of a list of remote procedures and parameter types.

45. RPCGEN Protocol Description Input File rpcgen Client Stubs XDR filters header file Server skeleton C Source Code

46. rpcgen Output Files > rpcgen foo.x foo_clnt.c (client stubs) foo_svc.c (server main) foo_xdr.c (xdr filters) foo.h (shared header file)

47. Client Creation > gcc -o fooclient foomain.c foo_clnt.c foo_xdr.c • foomain.c is the client main() (and possibly other function) that call rpc services via the client stub functions in foo_clnt.c • The client stubs use the xdr functions.

48. Server Creation gcc -o fooserver fooservices.c foo_svc.c foo_xdr.c • fooservices.c contains the definitions of the actual remote procedures.

49. Example Protocol Definition struct twonums { int a; int b; }; program UIDPROG { version UIDVERS { int RGETUID(string<20>) = 1; string RGETLOGIN( int ) = 2; int RADD(twonums) = 3; } = 1; } = 0x20000001;