1 / 19

SEDA: An Architecture for Well-Conditioned, Scalable Internet Services

SEDA: An Architecture for Well-Conditioned, Scalable Internet Services. Matt Welsh, David Culler, and Eric Brewer Computer Science Division University of California, Berkley. SEDA. An Application for:. servicing requests for Internet services

skah
Download Presentation

SEDA: An Architecture for Well-Conditioned, Scalable Internet Services

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. SEDA: An Architecture for Well-Conditioned, Scalable Internet Services Matt Welsh, David Culler, and Eric Brewer Computer Science Division University of California, Berkley

  2. SEDA An Application for: • servicing requests for Internet services • servicing a load (demand) that is never constant • providing a service that scales to load • keep in mind these issues of scalability: • load fluctuates, sometimes wildly • you only want to allot the resources servicing the load requires • every system has a limit! Scale responsibly! • don’t over commit the system

  3. SEDA Design Goals: • service requests for BOTH static content AND dynamic content • demand for dynamic content is increasing • work to deliver static content is predictable • retrieve the static content from disk or cache • send it back on the network

  4. SEDA Design Goals: • work to deliver dynamic content is unknown • build content on the fly • how many I/Os to retrieve the content? • layout, insert, and format content • may require substantial computation • posing queries of a database and incorporating results • all the more reason to scale to load!

  5. SEDA Design Goals: • adaptive service logic • different levels of load require different service strategies for optimum response • load is determined by BOTH the number of requests for service and the work the server must do to answer them • platform independence by adapting to resources available • load is load, no matter what your system is

  6. SEDA Consider (just) Using Threads: • the ‘model’ of concurrency • two ways to just use threads for servicing internet requests: • unbounded thread allocation • bounded thread pool (Apache) • hint: remember !! • (somewhat different reasons though!) Threads!

  7. SEDA Consider (just) Using Threads: • unbounded thread allocation • issues thread per request • too many use up all the memory • scheduler ‘thrashes’ between them, CPU spends all its time in context switches • bounded thread pool • works great until every thread is busy • requests that come in after suffer unpredictable waits --> ‘unfairness’

  8. SEDA Consider (just) Using Threads: • transparent resource virtualization (!) • fancy term for the virtual environment threads/processes run in • threads believe they have everything to themselves, unaware of constraint to share resources • t. r. v. = delusions of grandeur! • no participation in system resource management decisions, no indication of resource availability to adapt its own service logic

  9. SEDA DON’T Consider Using Threads! • threaded server throughput degradation • as the number of threads spawned by the system rises, the ability of the system to do work declines • throughput goes down, latency goes up

  10. SEDA Consistent Throughput is KEY: • well-conditioned service: • sustain throughput by acting like a pipeline • break down servicing of requests into stages • each stage knows its limits and does NOT over provision • so maximum throughput is a constant no matter the varying degree of load • if load exceeds capacity of the pipeline, requests get queued and wait their turn. So latency goes up, but throughput remains the same.

  11. SEDA Opaque Resource Allocation: • incorporate Event-Driven Design in the pipeline • remember -- only one thread! • no context switch overhead • paradigm allows for adaptive scheduling decisions • adaptive scheduling and responsible resource management are the keys to maintaining control and not over-committing resources to account for unpredictable spikes in load

  12. SEDA Back to the argument: • Event-driven Programming vs. Using Threads • the paper recommends event-driven design as the key to effective scalability, yet it certainly doesn’t make it sound as easy as Ousterhout • event handlers as finite state machines • scheduling and ordering of events ‘all in the hands’ of the application developer • certainly looks at the more complex side of event-driven programming

  13. SEDA Anatomy of SEDA: • Staged Event Driven Architecture • a network of stages • one big event becomes series of smaller events • improves modularity and design • event queues • managed separately from event handler • dynamic resource controllers (it’s alive!) • allow apps to adjust dynamically to load • culmination is a managed pipeline

  14. SEDA Anatomy of SEDA: • a queue before each stage • stages queue events for other stages • note the modularity • biggest advantage is that each queue can be managed individually

  15. SEDA Anatomy of a Stage: • event handler • code is provided by application developer • incoming event queue • for portions of requests handled by each stage • thread pool • threads dynamically allocated to meet load on the stage • controller • oversees stage operation, responds to changes in load, locally and globally

  16. SEDA Anatomy of a Stage: • thread pool controller manages the allocation of threads. Number is determined by length of queue • batching controller determines the number of events to process in each invocation of the handler

  17. SEDA Asynchronous I/O: • SEDA provides two asynchronous I/O primitives • asynchronous = non-blocking • asynchronous socket I/O • intervenes between sockets and application • asynchronous file I/O • intervenes to handle file I/O for application • different in implementation, but design of each uses event-driven semantics

  18. SEDA Performance: • througput is sustained!

  19. SEDA Summary: • load is unpredictable • most efficient use of resources is to scale to load without over committing • system resources must be managed dynamically and responsibly • staged network design culminates in a well-conditioned pipeline that manages itself • event-driven design for appropriate scalability • threads are used in stages when possible

More Related