CS6223: Distributed Systems

1. CS6223: Distributed Systems Advanced RPC

2. Challenges for RPC implementation • Locating server process • Incompatible data representation • Failure of remote procedure call • Security • ……

3. Locating services • A bind-server maintains a DB of locally provided services • Each service is identified by its program no. and version no. • Each service is associated with its transport address • Service registration • When an RPC server is started, it registers itself to the bind-server with the transport address at which it is listening • Service query • When a client makes an RPC call to a server, it first inquires the bind-server of the transport address of the server

4. rpcbind in SUN OS • “rpcbind” should be started before any other RPC service. • can only be started by the super-user • When rpcbind is started, it checks if the name-to-address translation functions correctly. • If it fails, the network configuration databases may be corrupted. The rpcbind reports the error and terminates.

5. Report RPC information: rpcinfo • rpcinfo [host] • lists all registered RPC services with rpcbind on host in format of: program# ver# netid service owner • If host is not specified, the local host is the default. • Other options: rpcinfo [-T transport] [host] [prognum] [versnum]

6. RPC failures • Local procedure calls do not fail • If there is a core dump, the entire process dies • More error-prone with RPC: • Server failure • Network failure • Client failure (while server is still executing code for it) • Transparency breaks here • Applications should be prepared to deal with RPC failures

7. RPC semantics • Local procedure call: exactly once • Exactly once is difficult to achieve with RPC. A remote procedure may be executed: • 0 time: server crashed or RPC request is lost • 1 time: everything worked well • 1 or more: client retransmits the request after timeout while the server already executed the previous request

8. RPC semantics • Most RPC systems offer either • at least oncesemantics, or • at most oncesemantics • Try to design RPC functions idempotent: • idempotent functions: executing multiple times produces the same result as executing once • non-idempotent functions: side-effects

9. More issues Performance • RPC is slower … a lot slower Security • Messages visible over network • Authenticate client • Authenticate server

10. RPC protocol compiler: rpcgen • Generate stub routines from the interface definition • rpcgen –a name.x • name.h: header • name_xdr.c: XDR conversion routines • name_svc.c: server skeleton • name_server.c: service routines (user program) • name_clnt.c: client stub • name_client.c: client program

11. Other options to rpcgen • –Tproto_tbl.i: RPC dispatch table • –Scproto_client.c: sample client file • –Ssproto_server.c: sample server file • –Smmakefile.proto: sample makefile • –a all above files • –C: ANSI C code • –A: code supporting Auto MT (multi-thread) mode • –N: procedure can have multiple arguments

12. RPC Function Names and Parameters • Naming: lowercase, suffixed with an “_” and version no. • Returning parameter: pointer to the defined result • Two arguments in RPC calls: • a pointer to the defined argument • a pointer to a client handle • Two arguments in RPC service routines: • a pointer to the defined argument • a pointer to a server request (struct svc_req)

13. List of msg_client.c // see on-line listing main(argc, argv) int argc; char *argv[]; { host = argv[1]; msgprog_3(host); } void msgprog_3(host) { CLIENT *clnt; int *result_1; char * savemsg_3_arg, * *result_2; clnt = clnt_create(host, MSGPROG, MSGVER, "netpath"); if (clnt == (CLIENT *) NULL) {exit(1);} result_1 = savemsg_3(&savemsg_3_arg, clnt); if (result_1 == (int *) NULL) {clnt_perror(clnt, "call failed");} clnt_destroy(clnt); }

14. Create client handles CLIENT *clnt_create ( const char *host, /* name of the server host*/ const rpcprog_t prognum, /* service program number */ const rpcvers_t versnum, /* service version number */ const char *nettype /* transport selection */ ); • Create and return a client handle that first matches prog #, ver # and nettype. • Contact rpcbind on the server host to get the transport address of the specified service. • early binding – done once, not per procedure call

15. struct CLIENT typedef struct { AUTH *cl_auth; /* authenticator */ struct clnt_ops *cl_ops; caddr_t cl_private; /* private stuff */ char *cl_netid; /* network identifier */ char *cl_tp; /* device name */ } CLIENT; struct clnt_ops { enum clnt_stat (*cl_call)(); /* call remote procedure */ void (*cl_abort)(); /* abort a call */ void (*cl_geterr)(); /* get specific error code */ bool_t (*cl_freeres)(); /* frees results */ void (*cl_destroy)(); /* destroy this structure */ bool_t (*cl_control)(); /* the ioctl() of rpc */ int (*cl_settimers)(); /* set rpc level timers */ } ;

16. Destroy client handle void clnt_destroy(CLIENT *clnt); • Destroy the client's RPC handle and reclaim private data structures, including clnt itself. • Use of clnt is undefined after calling clnt_destroy().

17. List of msg_clnt.c // see on-line listing int * savemsg_3(argp, clnt) char **argp; CLIENT *clnt; { static int clnt_res; memset((char *)&clnt_res, 0, sizeof (clnt_res)); if (clnt_call(clnt, SAVEMSG, (xdrproc_t) xdr_wrapstring, (caddr_t) argp, (xdrproc_t) xdr_int, (caddr_t) &clnt_res, TIMEOUT) != RPC_SUCCESS) { return (NULL); } return (&clnt_res); }

18. Send RPC request to server enum clnt_stat clnt_call( CLIENT *clnt, /* pointer to the client handle*/ const rpcproc_t procnum, /* server procedure number */ const xdrproc_t inproc, /* XDR filter to encode arg */ const caddr_t arg, /* address of the argument*/ const xdrproc_t outproc, /* XDR filter to decode result*/ caddr_t res, /* address to store result*/ const struct timeval tout /* time-out value for trying the call*/ ); • If the remote call succeeds, the status returned is RPC_SUCCESS. • Otherwise, an appropriate error status is returned.

19. “xdr_” routines • For each user-defined data type in the interface, there is an “xdr_XXX” routine in _xdr.c file that converts data between the local representation and the standard XDR format. • There are library routines for converting standard data types in XDR, e.g., xdr_string, xdr_int, xdr_double, etc. They are not defined in _xdr.c file. • In client’s “clnt_call”, xdr routines encode the input parameter to XDR and decode the return result to the local format. • In server’s dispatch routine, “svc_getargs” decodes arguments from XDR to local format and “sendreply” encodes the result to XDR format.

20. const MAXSUBJ = 64; typedef char id[8]; typedef char code[8]; struct registargs { code subj_code; id stud_id; }; struct status { code subj_code; int result; }; struct regist_status { int total_regist; struct status subjs_status<MAXSUBJ>; }; program REGISTPROG { version REGISTVER { int REGIST(registargs) = 1; int DEREGIST(registargs) = 2; regist_status VIEW_STATUS(id) = 3; } = 3; } = 100023; regist.x (an example)

21. List of regist_xdr.c #include "regist.h" bool_t xdr_id(register XDR *xdrs, id objp) { if (!xdr_vector(xdrs, (char *)objp, 8, sizeof (char), (xdrproc_t) xdr_char)) return (FALSE); return (TRUE); } bool_t xdr_code(register XDR *xdrs, code objp) { if (!xdr_vector(xdrs, (char *)objp, 8, sizeof (char), (xdrproc_t) xdr_char)) return (FALSE); return (TRUE); } bool_t xdr_registargs(register XDR *xdrs, registargs *objp) { if (!xdr_code(xdrs, objp->subj_code)) return (FALSE); if (!xdr_id(xdrs, objp->stud_id)) return (FALSE); return (TRUE); } ……..

22. XDR encode/decode routine bool_t xdr_proc( XDR * xdrs, /* a XDR handle to or from which the data type is to be converted */ <type> * argresp /* pointer to the structure to be converted */ ) • convert between built-in C data types and an external data representation • simple data structures xdr_bool, xdr_char, xdr_double, xdr_enum, xdr_float, xdr_free, xdr_hyper, xdr_int, xdr_long, xdr_longlong_t, xdr_quadruple, xdr_short, xdr_u_char, xdr_u_hyper, xdr_u_int, xdr_u_long, xdr_u_longlong_t, xdr_u_short, xdr_void • complex data structures xdr_array, xdr_bytes, xdr_opaque, xdr_pointer, xdr_reference, xdr_string, xdr_union, xdr_vector, xdr_wrapstring • encode/decode routines for each user-defined type

23. struct XDR typedef struct XDR { enum xdr_op x_op; /* operation; fast additional param */ struct xdr_ops *x_ops; caddr_t x_public; /* users' data */ caddr_t x_private; /* pointer to private data */ caddr_t x_base; /* private used for position info */ int x_handy; /* extra private word */ } XDR; enum xdr_op { XDR_ENCODE = 0, /*encode the type into the stream */ XDR_DECODE = 1, /*decode the type into the stream */ XDR_FREE = 2 /* release the space allocated by an XDR_DECODE request */ };

24. List msg_server.c int * savemsg_3(argp, rqstp) char **argp; struct svc_req *rqstp; { static int result; FILE *fp; fp = fopen("msg.dat", "w+"); fprintf(fp, "%s\n", *argp); fclose (fp); result++; return (&result); }

25. struct svc_req struct svc_req { rpcprog_t rq_prog; /* service program number */ rpcvers_t rq_vers; /* service protocol version */ rpcproc_t rq_proc; /* the desired procedure */ struct opaque_auth rq_cred; /* raw creds from the wire */ caddr_t rq_clntcred; /* read only cooked cred */ SVCXPRT *rq_xprt; /* associated transport */ };

26. Server Skeleton • Create server handles • Create socket connection • Register the program dispatcher, with program #, version #, etc. • Start to listen and wait to accept requests for services • When an RPC request arrives, the server calls the dispatcher • The dispatcher calls the corresponding service routine for the requested service, and sends a reply back to client

27. List of msg_svc.c main() { if (!svc_create(msgprog_3, MSGPROG, MSGVER, "netpath")) { _msgout("unable to create (MSGPROG, MSGVER) for netpath."); exit(1); } svc_run(); /* NOTREACHED */ }

28. Create server handles int svc_create ( /* transport-independent create routine*/ const void *dispatch, /* dispatching program*/ const rpcprog_t prognum, /* service program number */ const rpcvers_t versnum, /* service version number */ const char *nettype /* transport selection */ ); • Create server handles for all the transports belonging to the class nettype, and return the number of server handles it created • Register itself to the rpcbind with the dispatch routine and the service interface specified by prog # and ver # • The dispatch routine will be called whenever there is an RPC for the given prog # and vers # (done by svc_run()).

29. nettype Define a class of transports which can be used for a particular application: • netpath (default nettype) • Choose from the transports indicated by their token names in the NETPATH environment variable. If NETPATH is unset or NULL, it defaults to visible. • visible • Choose the transports with the visible flag (v) set in the /etc/netconfig file. • circuit_v • same as visible except that it chooses only the connection oriented transports • datagram_v • same as visible except that it chooses only the connectionless datagram transports • circuit_n • same as netpath except that it chooses only the connection oriented transports • datagram_n • same as netpath except that it chooses only the connectionless datagram transports • udp: Internet UDP • tcp: Internet TCP

30. svc_run() Server starts running: • Must be called after all the services are registered • After initialization, the server waits for client requests • Upon a request arrives, the dispatch routine matching the pair of requested prog # and ver # is invoked.

31. Dispatch routine (in msg_svc.c) msgprog_3 (rqstp, transp) struct svc_req *rqstp; register SVCXPRT *transp; { switch (rqstp->rq_proc) { case NULLPROC: (void) svc_sendreply(transp, xdr_void, (char *)NULL); _rpcsvcstate = _SERVED; return; case SAVEMSG: local = (char *(*)()) savemsg_3; break; case READMSG: local = (char *(*)()) readmsg_3; break; default: svcerr_noproc(transp); _rpcsvcstate = _SERVED; return; } if (!svc_getargs(transp, _xdr_argument, (caddr_t) &argument)) { svcerr_decode(transp); return; } result = (*local)(&argument, rqstp); if (_xdr_result && result != NULL && !svc_sendreply(transp, _xdr_result, result)) { svcerr_systemerr(transp); } return; }

32. Get the arguments of request bool_t svc_getargs( const SVCXPRT *xprt, /* service transport handle associated with the request */ const xdrproc_t inproc, /* XDR decode for the arguments */ caddr_t in /* address where the arguments will be placed*/ ); • decode the arguments of an RPC request • return TRUE if decoding succeeds, and FALSE otherwise

33. Send back the reply bool_t svc_sendreply ( const SVCXPRT *xprt, /* service transport handle associated with the request */ const xdrproc_t outproc, /* xdr encode of output parameter*/ caddr_t out /* address of the results*/ ); • encode the RPC result and send it back to the client • return TRUE if send succeeds, and FALSE otherwise

34. MT (Multi-Thread) modes “rpcgen –A” option to turn on the Automatic MT mode • The generated dispatch program contains the code of creating a new thread to serve each new request, and the necessary code for mutual exclusion to protect shared variables • Turn on “–M” option implicitly, which generates MT safe stubs for passing arguments & results