1 / 18

Budget-based Control for Interactive Services with Partial Execution

Budget-based Control for Interactive Services with Partial Execution. Yuxiong He, Zihao Ye, Qiang Fu, Sameh Elnikety Microsoft Research. Motivation. Interactive services specify stringent SLA on response time Long response time causes user dissatisfaction and revenue loss

mabyn
Download Presentation

Budget-based Control for Interactive Services with Partial Execution

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Budget-based Control for Interactive Services with Partial Execution Yuxiong He, Zihao Ye, Qiang Fu, Sameh Elnikety Microsoft Research

  2. Motivation • Interactive services specify stringent SLA on response time • Long response time causes user dissatisfaction and revenue loss • Important to bound response time (e.g. mean, 95-percentile) • Address two challenges • Adapt to dynamic and changing environment • Achieve high response quality Goal: Develop a self-managed scheduling system to meet response time target while achieving high quality.

  3. Existing Techniques (1) • Static admission control approach • Define a fixed queue length limit; drop requests when queue is full. • Issues • Only works under a static system. • Determining an appropriate queue-length for every setting and load is challenging. • Small queue length => underutilize resources • Large queue length => long response time • Can not adapt to dynamic and changing environment.

  4. Existing Techniques (2) • Classic feedback control approach: • Feedback control on queue length • Decrease queue length when response time is above target • Issue • Dropping requests results in degraded quality • Does not consider partial execution of requests

  5. Partial Execution & Response Quality • Incomplete execution of requests may still return meaningful partial results • Many interactive services support partial execution • Web search, web server, video streaming, finance server • Quality profile • A function maps request execution time to response quality

  6. Our Contributions • Propose a budget-based control model for interactive services with partial execution • Use feedback control to meet response time target • Apply optimization procedure to improve response quality • Exploit partial execution and request quality profile • Evaluation • Implementation at Bing search server • Simulation on finance server

  7. Budget-based Control Model • Control Variable • Budget: amount of computation time for all pending requests • Control mechanism • Determine the budget based on response time feedback • Control budget to meet response time • Optimization procedure • Given a budget, assign processing time to requests • Exploit partial results of a query • Scheduling to improve quality

  8. Control Mechanism • Basic idea • If response time is larger than target, smaller budget • If response time is smaller than target, larger budget • Criteria • Meet response time target accurately and quickly • Incur little runtime overhead.

  9. Control Mechanism: Background • Integral control • Adjust budget based on the difference between the observed and target response time • Advantage: eliminate steady-state error • Limitation: response is slow (long settling time) • Adaptive control • Model estimator + Linear quadratic optimal controller • Advantage: quick adaptation, fast response • Limitation: computationally expensive, stead-state error

  10. Control Mechanism: Hybrid Control • Combine the integral and adaptive control • Run adaptive control periodically in a coarse-grain time interval • Use integral control for execution of each request for fine-grain adjustment • Meet our goal • Quick and accurate adaptation • Little runtime overhead.

  11. Optimization Procedure • Objective: maximize total response quality • Input: budget, pending requests • Output: assigned processing time to requests • Optimization procedure depends on applications

  12. Bing index server • Core part of Bing search • For a user query, match and rank docs, return top results • Concave quality profile • First-half of request execution receives higher quality gain than the second half.

  13. Optimization Procedure for Index Server • Run the portion of requests with higher gain • Prevent long requests from starving short ones • Combine two techniques • Reservation at light load: • Reserve time for later requests in the queue based on mean service demand • Equal sharing at heavy load: • Allocate resource equally among requests

  14. Evaluation • Implemented and evaluated at Bing index server • Meet response time target and achieve high quality • Simulation study on finance server • Double system throughput at desired quality

  15. Bing Index Server • Implementations • BudgetIS • Feedback control on budget • Hybrid control + optimization procedure • QueueIS • Feedback control on queue length • Evaluation • Production trace

  16. Compare Queue v.s. Budget Approach Mean response time = 35ms • Budget approach • Meet response time accurately • Achieve high quality

  17. Conclusion • Propose a budget-based control optimization model for interactive services with partial execution • Hybrid control mechanism to meet response time target • Optimization procedure to improve response quality • Evaluation • Implemented and evaluated at Bing index server • Meet response time target and achieve high quality • Simulation study on finance server • Double system throughput at desired quality

  18. Thank you & Questions?

More Related