830 likes | 1.04k Views
Son YoungSu indigoguru@gmail.com Microsoft MVP Devpia Architecture Sysop EvaCast Study Leader ( http://www.EvaCast.net ). JAWS (A Framework for High-Performance Web Servers) TP Version 1.1. What’s the Good Web Server?. Architect. Motivation. Motivation. Motivation. Seed Paper.
E N D
Son YoungSu indigoguru@gmail.com Microsoft MVP Devpia Architecture Sysop EvaCast Study Leader (http://www.EvaCast.net) JAWS(A Framework for High-Performance Web Servers)TP Version 1.1
What’s the Good Web Server? Architect Motivation http://www.EvaCast.net
Motivation http://www.EvaCast.net
Motivation http://www.EvaCast.net
Seed Paper • James C. Hu, Douglas Schmidt, JAWS: A Framework for High-Performance Web Servers, Domain-Specific Application Frameworks: Frameworks Experience By Industry, John Wiley & Sons, October, 1999. http://www.EvaCast.net
Agenda • Motivation • Applying Patterns & Frameworks to Web Server • Common Pitfalls of Developing Web Server • Overcoming Web Server Pitfalls • Relationship Between Frameworks, Patterns & Other Reuse Technique • The JAWS Web Server • Overview of the JAWS Framework • Overview of the Design Pattern in JAWS • Concurrency Strategy • I/O Strategy • Protocol Pipeline Strategy • File Caching Strategy • Web Server Benchmarking • Concluding Remarks http://www.EvaCast.net
2. Applying Patterns & Frameworks to Web Server • Common Pitfalls of Developing Web Server Software • Overcoming Web Server Pitfalls with Patterns and Frameworks • Relationship between Frameworks, Patterns, and Other Reuse Techniques http://www.EvaCast.net
2.1 Common Pitfalls of Developing Web Server SW • Excessive low-level detail • Continuous rediscovery & reinvention of incompatible higher-level programming abstraction. • High potential for errors • Lack of portability • Steep learning curve • Inability to handle increasing complexity http://www.EvaCast.net
2.2 Overcoming Web Server Pitfalls with Patterns & Frameworks • Limitations of Class Library • do not capture the control flow and collaboration among families of related SW components. • Class Library reuse often re-invent and re-implement the overall Software architecture for each new Application. http://www.EvaCast.net
2.2 Overcoming Web Server Pitfalls with Patterns & Frameworks • The Benefits of Patterns • Expert’s Design Experiences • Good Methods to use Framework • The infrastructure of Framework • The Benefits of Frameworks • Expert’s intellectual products • Help developers avoid costly re-invention http://www.EvaCast.net
2.3 Relationship Between Frameworks, Patterns, and Other Reuse Techniques Class Libraries vs Framework Class Library NETWORKING GUI Active Object State DATABASE ADTs App Specific Logic OO Design MATH MATH NETWORKING Invocations Event Loop Reactor GRAPHICS GUI App Specific Logic Event Loop Invocations Callbacks Singleton Strategy Selections ADTs Reactor Adapter DATABASE GRAPHICS Singleton Adapter State Active Object Design Pattern Class Library Component Architecture Application Framework Component Architecture http://www.EvaCast.net
2.3 Relationship Between Frameworks, Patterns, and Other Reuse Techniques • Frameworks define “semi-complete” application that embody domain-specific object structures and functionality. • Class Libraries don’t offer explicit guidance to system design • Components in a Framework manage the canonical flow of control within App • Frameworks are active and exhibit“IoC(Inversion of Control)” at runtime. • Template Method Pattern (Hollywood Principle) • In Practice, Frameworks, Class Libraries, and Components are complementary technologies. http://www.EvaCast.net
3. The JAWS Adaptive Web Server • Overview of the JAWS Framework, • Overview Design Pattern in JAWS. • 4 Core Components in JAWS • Concurrency. • I/O. • Protocol/Pipeline. • File Caching. • JAWS Framework Revisited http://www.EvaCast.net
3.1 Overview of the JAWS Framework http://www.EvaCast.net
3.2 Overview Design Pattern in JAWS • Strategic Pattern • Message Composition/Decomposition • Pipe & Filter , Layer, Composite Message • Dispatcher • Acceptor –Connector / Reactor / Proactor • Active Object • Service Configurator • Tactical Pattern • Strategy • Adapter • State • Singleton http://www.EvaCast.net
Strategic Pattern Overview Dispatching Data Layer Decomposite Message UnMarshaling Layer Composite Message Messages Marshaling
3.2.1 Strategic Pattern • Marshaling / UnMarshaling • Composite Message / Layer / Pipe & Filter • Dispatcher • Acceptor-Connector / Reactor / Proactor • Active Object • Service Configurator http://www.EvaCast.net
Composite Message http://www.EvaCast.net
Acceptor/Connector Pattern http://www.EvaCast.net
Acceptor Protocol Handler peer_acceptor_accept() peer_stream_open() Acceptor Pattern • Acceptor is Factory. • Creates, Accepts and Activates a new Protocol Handler Whenever the Event Dispatcher notifies Acceptor that a connection has arrived from a Client. Protocol Handler Task Concurrency Event Dispatcher http://www.EvaCast.net
Reactor Pattern Event Handler handle_event(in type) get_handle() Reactor handle_events() register_handler(in handle) remove_handler(in handle) Concrete Event Handler Concrete Event Handler Synchronous Event Demultiplexer handle_event(in type) get_handle() handle_event(in type) get_handle() select() dispatch handle set owns Handle <<uses>> notifies http://www.EvaCast.net
Reactive IO Handler handle_input() handle_output() Reactor send_file() recv_file() send_data() recv_data() Initiation Dispatcher Timer_Queue handle_event() register_handler(h) remove_handler(h) schdule_timer(h) cancel_timer(h) expire_timers(h) Reactor Pattern • decouples the synch demultiplexing and dispatching logic. • the Single-Thread Web Server Concurrency model InputOutput Handler Protocol Pipeline InputOutput Reactive IO Filecache Handle Synchronous IO Asynchronous IO IO Handle http://www.EvaCast.net
Proactor Pattern Completion Handler handle_event() Asynchronous Event Demultiplexer Asynchronous Operation Processor Asynchronous Operation Proactor get_completion_event() async_operation() execute_async_operation() handle_events() Initiator <<uses>> <<uses>> <<uses>> <<invokes>> is associated with Handle * <<enqueues>> <<demultiplexes & dispatches>> <<executes>> Concrete Completion Handler Completion Event Queue <<dequeues>> http://www.EvaCast.net
Thread Thread Thread 4. Link Socket and Port Proactor Pattern (IOCP) 7. Thread Returns Thread Pool 5. 1 Send Completion Socket Info 3. Create Thread Socket A 2.1 Create Completion Port Completion Port Socket B 5. 2 Send Completion Packet Socket C 1. Create Socket 2.2 Create Completion Queue Thread Pool 6. Process Completion Events http://www.EvaCast.net
Proactive IO Handler Asynch Op open() cancel() handle_read_file() handle_write_file() Proactor send_file() recv_file() send_data() recv_data() Timer_Queue Completion Dispatcher schdule_timer(h) cancel_timer(h) expire_timers(h) handle_event() register_handler(h) remove_handler(h) Proactor Pattern • decouples the Asynch demultiplexing and dispatching logic. • The Asynch variant of the Thread Pool • Use IOCP (IO Completion Port) InputOutput Handler Protocol Pipeline InputOutput Asynchronous IO Filecache Handle Synchronous IO Reactive IO IO Handle http://www.EvaCast.net
Active Object Pattern Scheduler ActivationQueue Proxy Dispatch() Insert() Insert() Remove() method_1() method_n() Servant method_1() method_n() 1 1 1 1 invoke creates creates maintains * MethodRequest Future write to Can_run() Call() obtain result from Concrete MethodRequest N Concrete MethodRequest 1 Client http://www.EvaCast.net
Protocol Handler http_request() ftp_request() snmp_request() Activation Queue insert() remove() Scheduler http_request() ftp_request() snmp_request() dispatch() Active Object Pattern • decouples method invocation from method execution. • Thread-per-Request, Thread Pool, Thread-per-Session Model use Active Object pattern. for(;;){ m = aq->remove(); dispatch(m); } aq->insert(http) delegates Method Object Resource Representation Protocol Pipeline http://www.EvaCast.net
Service Configurator Pattern Component Component Repository +Init() +Fini() +Suspend() +Resume() +Info() Service Configurator CoreComponent CoreComponent http://www.EvaCast.net
Service Configurator Pattern Service +init() +fini() +suspend() +resume() +info() Filecache Service Repository Protocol Handler Filter Repository Cache Strategy Repository DLL Cache Strategy Protocol Pipeline Read Request Filter LRU Strategy LFU Strategy Parse Request Log Request http://www.EvaCast.net • Provides System Configuration at Runtime. • Dynamically optimize, control and reconfigure the behavior of Web server strategies a installation-time or during run-time.
3.2.2 Tactical Patterns • Strategy • Configure Cache replacement strategies without afffecting the core SW Architecture. • Adapter • Uniformly encapsulate the operation of synch, asynchand reactive I/O. • State • Event Dispatcher use State to seamlessly support different concurrency strategies and both synch and asynch. • Singleton • To provide a global point to access object. • Exist one copy of Cached Virtual File System. http://www.EvaCast.net
Review http://www.EvaCast.net
3.3 4 Components in JAWS http://www.EvaCast.net
3.3 Concurrency Strategy • Design Challenge • A large portion of non-I/O related Web server overhead is due to the Web server’s concurrency strategy. • Key Overhead • Synchronization • Thread/Process Creation • Context Switching • Choosing an efficient concurrency strategy is crucial to achieve high-performance http://www.EvaCast.net
3.3 Concurrency Strategy • Alternative Solution Strategies • Thread-per-Request • Thread-per-Session • Thread Pool • Single-Thread • Experiments • JAWS Concurrency Strategy Framework http://www.EvaCast.net
3.3.1 Alternative Solution Strategies • Concurreny Strategy of Web Servers • Single Thread • Roxen • Process-Based • Apach • Zeus • MultiThread • PHTTD • JAWS Dynamic Factors # of simultaneous requests Memory Use Machine Load Memory Workload Static Factors Web Server Use Case OS Platform H/W Configuration http://www.EvaCast.net
Thread-per-Request • Allows each client request to run concurrently in a separate thread. • Each Request arrives, a new thread is crated to process request. • This design appropriate synch I/O mechanisms Protocol Handler Protocol Handler Task Acceptor Protocol Handler Protocol Handler Concurrency Event Dispatcher http://www.EvaCast.net
Thread-per-Request (cont’d) • Advantage • Simplicity • Exploit parallelism on multi-processor platform • Limitation • The # of running threads may grow without bound, exhausting available memory and CPU resources • Compatible Fields • Use Light loaded servers with low latency • Don’t use time consuming task based system http://www.EvaCast.net
Thread-per-Session • Allows each client connection to run concurrently • All of these requests are submitted through a connection between each client and a separate thread in the web server process • Advantage • Does not spawn a separate thread for each request. • Limitation • This model still vulnerable to unbounded resource consumption as the # of clients grows. http://www.EvaCast.net
Thread-per-Session (cont’d) • Suited for HTTP 1.1 but not HTTP 1.0 • HTTP 1.0 • support open/operation/close • One URL Fetch per Connection • Needs Reconnection • Transporting Data Size Limitation • URL Size Limitation • HTTP 1.1 • Persistent Connection • Pipeline Functionality • Many Requests -> Support Serial Response • Compressed Data Transporting • Use Proxy Server and Cache http://www.EvaCast.net
Thread Pool • Allows up to N requests to execute concurrently within a pool of threads. • Support prespawning of Thread Creation Protocol Handler Protocol Handler Task Acceptor Protocol Handler Protocol Handler Concurrency Event Dispatcher http://www.EvaCast.net
Thread Pool (cont’d) • Advantage • Lower overhead that Thread-per-Request. • Thread Prespawning • thread creation cost is amortized • Pool Size is Fixed • Resource Consume is bounded • Limitation • Small Pool Size • New incoming requests be dropped or wait indefinitely. • Large Pool Size • Resource Consumption may be no better than using Thread-Per-Request http://www.EvaCast.net
Single-Threaded • All connections and requests are handled by the same thread of control. • Single-Threaded servers process requests iteratively. • Compatible Filed • Asynch or Reactive I/O Multiple Request • Inadequate for high volume server • Subsequent requests are blocked until they are reached http://www.EvaCast.net
Experiments • Static adaptivity • Select the concurrency strategy that best meets The static requirements of the system. • multi-threaded concurrency • More suited to A multiple-processor machinethan a uni-processor machine • Dynamic adaptivity • Adapt dynamically to current server condition • In order to accommodate Unanticipated load • Support to increase the # of available threads in the Pool http://www.EvaCast.net
Event Dispatcher Concurrncy Thread Pool Thread-per-Connection Acceptor Protocol Handler dispatch() dispatch() open() accept() dispatch() dispatch() Task accept() activiate() enque() deque() svc() JAWS Concurrency Strategy Framework server handler server->dispatch() acceptor task acceptor->accept() task->accept() task->activate() task->enque() http://www.EvaCast.net
3.4 I/O Strategy • Design Challenge • is to devise efficient data retrieval and delivery strategies, collectively referred to as I/O. • Key Overhead • Arranging Multiple I/O operation. • Example • Synch • Web Transaction involving monetary fund transfer • ASynch • Web access to static information (Web page – ASP..) http://www.EvaCast.net
3.4 I/O Strategy • Alternative Solution Strategies • The Synchronous I/O Strategy • The Reactive I/O Strategy • The Asynchronous I/O Strategy • Experiments • JAWS I/O Strategy Framework http://www.EvaCast.net
The Synch I/O Strategy • Synch I/O describes the model of I/O interactionbetween a Web server process and kernel. • Kernel doesn’t return the thread of control to the server until the requested I/O operation either complete , completes partially, or fails. • Advantage (Easy Use) • Disadvantage (Resource Exhaust) • If combined with a single threaded concurrency strategy, It is not possible to perform multiple synch I/O operation simultaneously. • When using multiple thread, It is still possible for an I/O request to block indefinitely. http://www.EvaCast.net
The Reactive I/O Strategy • Reactive I/O alleviates the blocking problems of synchronous I/O without resorting to polling. • Web Server uses an OS event demultiplexing system call. • select() in UNIX • WaitForMultipleObjects() in Win32 • Refers Reactor Pattern • Disadvantage • May not make effective use of multiple CPU. http://www.EvaCast.net
The Asynch I/O Strategy • Asynch I/O simplifies the de-multiplexing of multiple events in one or more thread of control without blocking the Web server. • Advantage • The Web Server not block on I/O request. • Disadvantage • Asynch I/O is not available on many OS Platform. • more complicated than writing synch program. http://www.EvaCast.net
Experiments • The Result reveal that each I/O strategy behaves differently under different load condition. • JAWS I/O Strategy Framework addresses this issue by allowing the I/O Strategy to adapt dynamically to run-time server condition. http://www.EvaCast.net