Event-Driven Concurrent Service Processing using the Reactor Pattern

E81 CSE 532S: Advanced Multi-Paradigm Software Development Reactor Pattern Venkita Subramonian,Christopher Gill, Guandong Wang, Zhenning Hu, Zhenghui Xie Department of Computer Science and Engineering Washington University, St. Louis cdgill@cse.wustl.edu

Motivating Example: A Logging Server From http://www.cs.wustl.edu/~schmidt/patterns-ace.html

Client1 CONNECT listen Port:30000 accept Port:24467 CONNECT Client2 accept Port:25667 Evolving to Concurrent Event Handling Goal: process multiple service requests concurrently Logging Server Port:27098 Port:26545

Where We’re Starting main() { bind listening port; listen for (;;) { new_conn_socket = accept (); run(handler(new_conn_socket)); // write, read } }

Logging Server: Threaded Approach main() { bind listening port; listen for (;;) { new_conn_socket = accept (); // fork a process or spawn a thread for handler thread.run(handler(new_conn_socket)); } }

Problems with Threaded Approach • Multi-threading may increase code complexity • Multi-threading/processing adds overhead • Context switching (especially among processes) • Synchronization for shared data, other resources • What if we could make 1 thread responsive? • Better resource utilization by aligning threading strategy to # of available resources (like CPUs) • Also, multi-threading may not be available in all OS platforms (e.g., embedded ones)

Alternative: Event Driven Server (reusable: e.g., from ACE) Event Dispatching Logic (pluggable: you write for your application) Event Handling Logic • Inversion of control • Hollywood principle – Don’t call us, we’ll call you (“there is no main”) Connection Acceptor handle_connection_request Event Handlers handle_data_read Data Reader

Reactor Pattern (Dispatching Logic) • An architectural pattern • Context: event-driven application • Concurrent reception of multiple service requests, but serial processing of each one • Dispatch service requests • Calls the appropriate event handler • Also known as • Dispatcher, Notifier, Selector (see Java NIO)

Design Forces • Enhance scalability • Maximize throughput • Minimize latency • Reduce effort that is needed to integrate new services into server

Solution – Separation of Concerns Application De-multiplexing & Dispatching Application logic Event sources Event Handlers Reactor Synchronous wait Serial Dispatching

Reactor Pattern Structure a.k.a “the reactor” From http://www.cs.wustl.edu/~schmidt/patterns-ace.html

Synchronous vs. Reactive Read Clients Server Clients Server read() select() data data read() HandleSet HandleSet

Serial Event Dispatching Reactor Application read() Clients select() Event Handlers handle_*() read() HandleSet

Interactions among Participants Synchronous Event Demultiplexer Concrete Event Handler Reactor Main Program register_handler(handler, event_types) get_handle() handle_events() select() event handle_event()

Implementation • De-multiplexer/dispatcher infrastructure • Anonymous de-multiplexing of events to handlers • Assumes specific event handler hook methods • Application • Defines concrete event handlers • Handlers perform service-specific processing (“Service Handlers”)

Event Handler Interface • Determine type of dispatching target • Objects vs. functions • Can have pointers to either • Command pattern can unify these • E.g., handle_event () • Event handling dispatch interface strategy • Single-method dispatch • handle_event (handle, event_type) • Multi-method dispatch • handle_input (handle) • handle_output (handle) • handle_timeout (handle) Note: singular, not plural

Reactor Interface • Handler registration/deregistration • E.g., register_handler()deregister_handler() • Consider visitor, observer patterns • Event loop • E.g., handle_events() Note: plural, not singular

Reactor Implementation • Reactor implementation hierarchy • Abstract base class or template concept • Concrete platform-specific implementations • Synchronous event de-multiplexing mechanism • E.g., WaitForMultipleObjects() on Win32 • E.g., select() or poll() on UNIX platforms • Implement a dispatch table • Complete concrete reactor implementation • Hook dispatch table into de-mux mechanism

Multiple Reactors • A single reactor instance will work in most cases • Sometimes desirable, e.g., for handler serialization • Can use Singleton (e.g., ACE_Reactor::instance()) • Limits on number of OS handles may restrict this • Total available (rarely an issue in a general-purpose OS) • Max a single thread can wait for • E.g., 64 in some Win32 platforms • May need multiple reactors, each with its own thread • Note that handlers are not serialized across Reactor instances – treat remote/concurrent reactors similarly

Concrete Event Handlers • Implement base interface / model concept • Determine policies for handler state • Stateless, stateful, or a combination • ACTs (cookies) can help offload some of the state • I.e., can keep state outside the handler objects, but index into a data structure, etc. using the ACT • Implement event handler functionality • I.e., add application logic to handler methods

Example Resolved: Part 1 Steps performed when a client connects to the logging server a.k.a. “the reactor” From http://www.cs.wustl.edu/~schmidt/patterns-ace.html

Example Resolved: Part 2 Steps performed by reactive logging server to for each record a.k.a. “the reactor” From http://www.cs.wustl.edu/~schmidt/patterns-ace.html

Variant: Integrated De-multiplexing of Timer and I/O Events • Timer-based and I/O-based events in same reactor • Extend reactors and event handlers • Register concrete event handlers for some time trigger • Relative vs. absolute time triggers • Periodic vs. one time invocation • Reactor calls handler’s handle_timeout() method • Can use same handler for time and event dispatching • E.g., an alert watchdog timer for some logging handler • Various timer strategies • E.g., select/WFMO timeout • E.g., hardware timer interrupt • E.g., polling Pentium tick counter • Key trade-offs between portability, overhead and responsiveness

Variant: Re-entrant Reactors • Event handlers re-invoke reactor->handle_events() • Result: nested event handlers • E.g., CORBA AMI  nested work_pending() • Reactor implementation must be re-entrant • Copy the handle set state onto the run-time stack • Any changes to handle set are local to that nesting level of the reactor • Use thread stack frame to record reactor’s logical “stack frame”

Variant: Thread-Safe Reactor • Synchronized reactor • Lock to synchronize access to the reactor’s internal state • Multiple threads could register/remove event handlers • Preventing self-deadlock • An event handler could register/remove other event handlers or itself • Explicitly notifying a waiting event loop thread • Notify the reactor of a change so that the wait handle-set could be updated

Variant: Concurrent Event Handlers • Event handlers with their own threads • In addition to event loop thread(s) • Related concurrency patterns • the Active Object • the Leader/Followers • the Half-Sync/Half-Async

Variant: Concurrent Event De-multiplexer • Event de-multiplexer concurrent in multiple threads • E.g., WaitForMultipleObjects() • Benefits • Can improve throughput significantly for some applications • Drawbacks • Need a thread-safe event de-multiplexer wrapper façade • Less portable (fewer platforms support this) • Implementation can become more complex

Event-Driven Concurrent Service Processing using the Reactor Pattern