310 likes | 421 Views
Distributed Computing with DAFFIE. Glenn Bresnahan Boston University Scientific Computing and Visualization (http://scv.bu.edu/DAFFIE/Earlab-Oct02). What is DAFFIE?. D istributed A pplications F ramework F or I mmersive E nvironments
E N D
Distributed Computing with DAFFIE Glenn Bresnahan Boston University Scientific Computing and Visualization (http://scv.bu.edu/DAFFIE/Earlab-Oct02)
What is DAFFIE? DistributedApplicationsFrameworkForImmersiveEnvironments System for easily creating distributed and shared virtual environments Authored by Glenn Bresnahan, Erik Brisson, Robert Putnam Scientific Computing and Visualization
DAFFIE Messaging System • Application protocol • Client services • Client application library (API) • Message router (“event server”) Scientific Computing and Visualization
DAFFIE Message routing client client client proxy client client event server client client proxy client client Scientific Computing and Visualization
Application Architecture App thread App thread App thread … App thread Services (timing, streaming, file sharing) Event Messaging Message Transport Scientific Computing and Visualization
API - Control • id = event_join(server, &nc); • rc = event_leave(); • id = event_id(); • nc = event_nclient(); • millisec = event_clock(); • event_sleep(millisec); • event_register(class, species, instance); Scientific Computing and Visualization
Event headers information • EventType(event); • EventSize(event); • EventFrom(event); • EventTo(event); • EventSeq(event); Scientific Computing and Visualization
System Events – Join & Leave typedef struct { EVENT_HEAD head; int id; int nmember; } EVENT_JOIN; typedef struct { EVENT_HEAD head; int id; short detail; short exitcode; } EVENT_LEAVE Scientific Computing and Visualization
System Events – Identification typedef struct { EVENT_HEAD head; int tag; } EVENT_IDENTITY; typedef struct { EVENT_HEAD head; char host[MAXIDSTR]; char class[MAXIDSTR]; char species[MAXIDSTR]; char instance[MAXIDSTR]; int tag; } EVENT_IDENTIFICATION; Scientific Computing and Visualization
API – Send • rc = event_send(to, type, &event, size); • “to” is a client id or one of the following: • EVENT_BCAST_ALL • EVENT_BCAST_NOTME • EVENT_BCAST_SERVER • EVENT_BCAST_LOCAL • Macros • SEND(to, type, event); • BCAST(type, event); • event_drain(); Scientific Computing and Visualization
API - Receive • rc = event_receive(&event); • event_receive_type(n, type1, …); • event_select(on_off, et_low, et_high); • event_wait(); • event_wait_timeout(millisec); Scientific Computing and Visualization
API – Receive queue • event_scan_type(n, type1, …); • rc = event_scan(&event); • n = event_queued(); • event_flush(flag); • event_flush_sync(); • event_receive_enable(on_off); Scientific Computing and Visualization
API – Active messages • event_callback(type, func); • event_add_callback(type, func); • event_delete_calllback(type, func); • event_protect_type(on_off, et_low, et_high); Scientific Computing and Visualization
Client Services • Streaming • Data chunking • Bandwidth management • Dynamic tuning • File sharing • File servers • Conditional upload/download Scientific Computing and Visualization
Job Initiation & Monitoring • Rely on external authentication and job initiation (e.g. ssh, Globus) • Launch scripts (ad hoc) • cgl-video; visible.bu.edu:vgrab -compress -process; -:vswirl -steps 9999999 -np 4; -:vbroadcast; .:vdirect -object poolanim_grp ; • Agent initiation daemon • Logging daemon Scientific Computing and Visualization
Message Transports • TCP/IP • Single channel or striped • Shared memory • Myrinet • IP Multicast Scientific Computing and Visualization
Defining Upper Layer Protocol • Components • Event type ID (#define in a .h) • Data layout (compile-time) (C struct in a .h) • Data layout (run-time) (event_define) • Needed for canonical byte ordering • Types statically (compile time) defined • Protocol compiler Scientific Computing and Visualization
Message Routing • Message router (“event server”) • Multiple delivery service classes • Globally sequenced, reliable delivery • Client management • Message subscription • Multiprocessor, high bandwidth • Hierarchical servers • Proxy routers • Fan-in/fan-out • Tunneling • Direct routing for P2P Scientific Computing and Visualization
Messaging API • Multithreaded, non-blocking • Dynamic join and leave • Point-to-point and broadcast messages • Polled or callback (active messages) • Subscription based receive • Low level services • Registration, timing, synchronization, query Scientific Computing and Visualization
Service Discovery • Client registration/identification • Class (Viewer, Audio, Video, Agent) • Species (video: source, broadcaster, sink, processor) • Instance (video: stream-name) • ID • Hostname • Client location services • Service request/acknowledgement Scientific Computing and Visualization
Client Monitoring • Dynamic clients • JOIN, EXIT, IDENTIFICATION events • Event server monitoring • Heart beat • Latency monitor • Congestion control • Statistic generation • Client monitoring • Process level monitoring • Congestion management • Remote process termination Scientific Computing and Visualization
Leaky Example: Headers typedef struct { EVENT_HEAD head; char filename[200]; } EVENT_GETPARAM; typedef struct { EVENT_HEAD head; int logger, frame_size, sample_rate); } EVENT_START; typedef struct { EVENT_HEAD head; char msg[1000]; } EVENT_LOGINFO; Scientific Computing and Visualization
Leaky Example: Headers typedef struct { EVENT_HEAD head; int client; int size; float pressure[1]; } EVENT_ADVANCE; Scientific Computing and Visualization
Leaky Example: Headers #define ET_START 1001 #define ET_GETPARAM 1002 #define ET_LOGINFO 1003 #define ET_ADVANCE 1004 #define ET_STOP 1005 event_define(ET_START, “start”, 1, 3, EVENT_DATA_INT); event_define(ET_GETPARAM, “getparam”, 1, 200, EVENT_DATA_CHAR); event_define(ET_ADVANCE, “advance”, 3, 1, EVENT_DATA_INT, 1, EVENT_DATA_INT, 0, EVENT_DATA_FLOAT); Scientific Computing and Visualization
Leaky Example: Initialization id = event_join(“earlab.bu.edu”, &nc); printf(“Joined as client id %d of %d\n”, id, nc); event_register(“integrator”, “leaky”, “1.0”); event_receive_type(ET_GETPARAM, ET_START, ET_ADVANCE, ET_STOP); event_callback(ET_START, do_start); event_callback(ET_GETPARAM, do_getparam); … Scientific Computing and Visualization
Leaky Example: File Service GotParamFile = 0; efs_init(); efs_chdir_receive(“/tmp/earlab”); Scientific Computing and Visualization
Leaky Example: GetParameters do_getparams(EVENT_GETPARM *egp) { efs_download_cond(egp->filename, egp->filename, xfer_done, 0); return(1); } xfer_done(int reqid, int fsid, inr rc, char *file, int tag) { char path[1000]; strcpy(path, “/tmp/earlab/”); strcat(path, file); GotParamFile = 1; Leaky_GetParameters(path); ); Scientific Computing and Visualization
Leaky Example: Start do_start(EVENT_START *est) { Logger = est->logger; while (!GotParamFile) event_sleep(100); Leaky_Start(egp->frame_size, egp->sample_rate); return(1); } Scientific Computing and Visualization
Leaky Example: LogInfo void LogInfo(char *msg) { EVENT_LOGINFO einfo; strcpy(einfo->msg, msg); event_send(Logger, ET_LOGINFO, &einfo, sizeof(einfo)); } Scientific Computing and Visualization
Leaky Example: Advance do_advance(EVENT_ADVANCE *ein) { int bytes; EVENT_ADVANCE *eout; bytes = sizeof(EVENT_ADVANCE) + (ein->size-1)*sizeof(float); eout = (EVENT_ADVANCE *) malloc(bytes); Leaky_Advance(&ein->pressure, &eout->pressure); eout->client = event_clientid(); eout->size = ein->size; event_send(ein->client, ET_ADVANCE, eout, bytes); free(ein); return(1); } Scientific Computing and Visualization
DAFFIE Message routing client client client proxy client client event server client client proxy client client Scientific Computing and Visualization