1 / 41

Active Messages

Active Messages introduce low-overhead communication for high-speed networking, reducing switch latencies and software overhead. Originally developed for MPPs, both original and new implementations support various parallel programs and applications with unique endpoint structures and communication interfaces.

hfairley
Download Presentation

Active Messages

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Active Messages CS63995 - Topics in Software Engineering - Summer 01’ Raquel S. Whittlesey-Harris

  2. Active Messages • A Low overhead communication primitive (LWP - Light Weight Protocol) • Developed to take advantage of advances in high speed networking • Increased bandwidths • Decreasing switch latencies • Decrease the software overhead • Dominating communication costs

  3. AM - Original Implemenation • Originally developed for MPPs (Massively Parallel Processors) • Only sufficient for Single-Program Multiple-data (SPMD) parallel programs • All AM Comm was constrained within individual parallel processes • Inadequate for Client/Server Apps • Mutually trusting instances of parallel processes

  4. AM - Original Implementation • Error and fault models followed the fail-stop semantics • any application error causes the entire process to hang or abort • Supports only single-threaded applications • Supports asynchronous communication-event handling with nonstandard interrupt facilities

  5. AM - New Implementaion • Developed for the cluster with support for protected multiprogramming, MIMD style applications and scalability • Exploit hardware capabilities of a network (low latency & high throughput) for a variety of communication interfaces

  6. AM - New Implementation • MPI - Message Passing Interface • DSM - Distributed Shared Memory • DFS - Distributed File System • CORBA - Distributed Object Sharing

  7. Active Messages Interface • Provides an RPC styled Request/Reply programming model • Allows applications to communicate through communication ports called Endpoints

  8. AM - Endpoints • Object representing user or kernel network “PORT” • Potentially, any two endpoints can communicate • Contingent upon protection and authentication • Traditional protection boundaries and domains can be crossed • Kernel, file-system endpoint can communicate with user-level, client endpoints

  9. AM - Endpoints • Apps exchange two types of P2P messages • Requests • Replies • Handlers are associated with each message • Executed upon receipt of message

  10. Unlike general RPC mechanism (perform computation on data) Handlers extract data from the network & integrate it into the ongoing computation A handler index is passed in the message structure handlers are executed immediately Messages are not buffered Exception of those required for network transport AM - Endpoints

  11. AM - Endpoint Structure • Function AM_AllocateEndpoint() - returns an endpoint structure and name • Consists of • Send Pool - for sending messages from an endpoint • Not directly exposed to applications • Random access to entries (“pool” not FIFO) • May be dynamically resized

  12. AM - Endpoint Structure • Receive Pool - for receiving messages to an endpoint • Not directly exposed to applications • Random access to entries • May be dynamically resized • Handler Table - for translating handler indices into functions • A form of indirection that affords a level of protection • Eliminates requirement that senders have a priori knowledge of addresses of handlers • Initialized with 256 entries; point to abort() function

  13. AM - Endpoint Structure • Function AM_SetHandler() - registers a handler at a particular index • Virtual Memory Segment - for receiving bulk memory transfers • Base address and byte length specify a window into the receiver’s address space into which endpoints with valid tags can write • Base address initialized to zero; length = 0 bytes

  14. AM - Endpoint Structure • Translation Table - for associating indices with global-endpoint names and tags • Array associating indices with global-endpoint names and tags • Operations are atomic • Initialized with 256 entries; marked as unused • Function AM_Map() - maps a local name to a global-endpoint name

  15. AM - Endpoint Structure • Tag - for authenticating messages arriving at the endpoint • 64-bit integers that authenticate messages • Managed by applications • Initialized to AM_NONE • May be changed at any time • AM_ALL - accept all messages • AM_NONE - accept no messages • Function AM_SetTag() - sets the 64-bit integer endpoint tag • Special Values: never-match, wild-card

  16. AM - Endpoint Structure

  17. AM - Endpoints • Endpoints can send an AM to another endpoint if and only if • The tag in the sender’s translation table entry for that destination endpoint matches the destination’s endpoint’s tag at the time the message is received. • Applications may coordinate the management of their translations to produce a convenient virtual network • Require exchange of endpoint names and tags • Unaddressed by AM interface specification

  18. AM - Endpoint Bundles • Set of endpoints created by one process • Treated as a single unit of communication, event management, and synchronization • Has unique event mask • Function AM_SetEventMask() - set an event

  19. AM - Endpoint Bundles • Has unique synchronization variable • Binary Semaphore • Function AM_Wait() - block for an event • Address deadlock issues that arise when multiple independent software packages are composed in a single-threaded application

  20. AM - Endpoint Bundle Structure • Function AM_AllocateBundle() - create an endpoint bundle of either • AM_SEQ - at most one thread can access the bundle at any time • AM_PAR - multiple application threads concurrently access bundle

  21. AM - Endpoint Bundle Structure • Consists of: • Endpoints - collection of endpoints • An endpoint is a member of exactly one bundle at any given time • Thread Synchronization Variable - binary semaphore • Posted upon event from any endpoint in a bundle • Identity of endpoint responsible is unavailable

  22. AM - Endpoint Bundle Structure • Event mask - event bit indicator • Selects which endpoint states or transitions generate events which posts the synchronization variable • Initialized to disable the generation of all events • Under control of application • Access Mode Flag - indicates if concurrent use of the bundle or its endpoints is expected • Informs the system if multiple, application threads will access the bundle or its components concurrently. • Defaulted to serialize concurrent access (AM_PAR)

  23. AM - Endpoint Bundle Structure

  24. AM - Endpoint Naming Model • Goals & Issues • Maintain independence between interface and implemenation • Allows multiple, higher-level naming systems to be constructed on top of the primitive naming system • Globally unique names • Retain the ability to refer to endpoints locally with small integer values

  25. AM - Endpoint Naming Model • Maintain position-independent names • Facilitate migration of endpoints • The system assigns a name upon creation of an endpoint • Not convenient • Not easily manipulated

  26. AM - Endpoint Naming Model • Treats global-endpoint names as opaque types • Manipulates either the named endpoint object or indices to the named endpoint objects • Endpoint translation table accomplishes through indirection • Interface is independent of external name servers and global operating systems • Dependent on external agent(s) to manage mappings of global endpoint names to physical resources

  27. AM - Endpoint Naming Model • The Translation Table maps small, integer indices to arbitrary endpoint names and tags • Allows for the existence of multiple name managers and naming models • Supports group models • Endpoints are members of communication groups • Named with group name and number (similar to a bundle but in a virtual sense)

  28. AM - Endpoint Protection Model • Tags implicitly identify sets of communicating endpoints • Applications use tags to identify aggregates of endpoints • Providing a simple message authentication model Invalid Tag! P58 AM_Request() Aggregate Invalid Tag! P1 Index P23 Name & Tag Get aggregate Valid Tag! P32 AM_Reply()

  29. AM - Endpoint Protection Model • Upon receipt of a request, the destination endpoint must match the tag of the sending endpoint • Processes may change an owned endpoint’s tag at any time • Allows processes to revoke sending capabilities of other endpoints (atomic action)

  30. AM - Endpoint Protection Model • Special Tag Values • Never-match • Wild card - matches all except never-match • An endpoint with a wild-card tag may have the memory segment corrupted by other endpoints • May copy endpoint VM segments into private memory and inspect before corrupting • Non-bulk messages pass by value therefore are not a threat for corruption

  31. AM - Endpoint Synchronization • Upon an event • The system atomically posts the bundle’s binary semaphore and clears the corresponding event mask bit • If Threads are blocked on the semaphore • One is unblocked

  32. AM - Transport Operations • P2P interface • Matching requests and reply message pairs • Upon receipt of a request • Request Handler is invoked • Reply exactly once to a request • Error to do other wise • May or may not be detected and reported • Upon receipt of a reply • Reply Handler is invoked • Cannot perform request or reply operations

  33. AM - Transport Operations • Transfer Types • Small • Messages composed entirely within a processor’s register set • Pass a small number of words by value (4 or 8) • AM_RequestM() - carries M integers as payload • E.G. AM_Request4() - carries 4 integers as payload

  34. AM - Transport Operations • Medium • Typically up to 128 words by value • AM_MaxMedium() - return the maximum transfer size for medium transfers • Large • Source memory regions are copied to user-specified destination endpoint memory segments • Done before message is delivered • AM_MaxLong() - returns the maximum transfer size for large transfers

  35. AM - Transport Operations • Tokens • Opaque pointer to data that identifies endpoints • Passed to handler functions • AM_GetSourceEndpoint() - translates a token into the globally-unique endpoint name of sender endpoint • AM_GetDestEndpoint() - translates a token into the globally-unique endpoint name of destination endpoint • AM_GetMsgTag() - retrieves the message tag from a token

  36. AM - Error Model • AM are considered sent • The source storage (registers or memory) can be reused • AM are considered delivered • It is entirely written into its destination endpoint’s receive pool

  37. AM - Error Model • AM are considered received • Handler function is invoked in receiving process • AM are delivered exactly once barring • Catastrophic network or system errors • persistent congestion at the destination endpoint

  38. AM - Error Model • Active Messages are never dropped • Messages can be deemed undeliverable • Returned to sender • Behave like ordinary AM • Undeliverable Message Handler • First entry in every Handler (Handler0) • Notifies applications of undeliverable messages • Default handler is abort()

  39. AM - Error Model • void handler (int status, op_t opcode, void *argblock) • Status - details why a message was undeliverable • opcode - identifies type of returned message • argblock (request) - points to structure with original arguments • argblock (reply) - points to structure containing token and original arguments

  40. AM - Error Model • For all error conditions except EUNREACHABLE(the destination endpoint was unreachable due to a serious likely catastrophic, system or network error), the system guarantees no events were produced at the destination endpoint • Possibility exists that the message was received by the destination endpoint

  41. References • A. Mainwaring and David Culler, “Active Message Applications Programming Interface and Communication Subsystem Organization”, Computer Science Division, University of California at Berkeley, Draft Technical Report, 1995. • N. Parab and M. Raghvendran, “Active Messages”, Center for Development of Advanced Computing, Bangalore, India.

More Related