Improving Performance of Internet Services Through Reward-Driven Request Prioritization

1. Improving Performance of Internet Services Through Reward-Driven Request Prioritization Alexander Totok and Vijay Karamcheti Computer Science Department New York University

2. Web Server Overload Conditions • Consequences • increased request response times • some requests are dropped • successful session throughput suffers dramatically • client dissatisfaction → reduced revenues • Current solutions work with static client identity • session-based admission control (SBAC) • service level agreements (SLA) • service membership • per-client history-based approach • looks at a client’s previous visits to the web site

3. Maximizing Profit Brought By Internet Services • Service profit (reward) maximization • shopping web site: number of items sold • Idea: assign higher execution priority to the requests, whose sessions are likely to bring more reward • how to predict a session’s reward? • Our solution:Reward-Driven Request Prioritization (RDRP) • predicts a session's activities by comparing requests seen in it with aggregated client behavior • uses Bayesian inference analysis to dynamically compute request priority in real time • contrasts with the per-client history-based approach

4. Service Usage Profiles (Patterns) • Session structure: first-order Markov chain • corresponds to a typical service usage profile (pattern) • Shopping scenario for TPC-W application (web store selling books) • “Mostly Buyers” profile – more buying activity • “Mostly Browsers” profile – more browsing activity

5. What Information Does RDRP Use? • User load structure: • {Profilei}; {pi} – percentage of sessions belonging to Profilek • on-line request profiling and clustering analysis [Menasce’99] • Request reward • rewardi:per request type • specified by the service provider • web shopping scenario: reward(addToCart)=1 • Relative request execution cost • for prediction of future server resource consumption • costi: per request type – average request processing time • fine-grained profiling of request execution • Only request reward is specified by the service provider

6. How Does The Algorithm Work? reward_attained + reward_expected priority = Step4: cost_expected cost_incurred +

7. Prototype Implementation in J2EE • Request priority used to allocate threads and DB connections

8. Evaluation • Shopping scenario for TPC-W • user load with two usage patterns: “Mostly Buyers”/“Mostly Browsers” • new sessions: bursty arrival process (B-model [Wang’02]) • Techniques compared • default – FIFO prioritization • session-based admission control (SBAC) • per-client history-based approach • success depends on how well prediction of a session’s behavior works • model different correlation between sessions’ rewards and assigned priorities: c = 0 (coin flip) c = 0.25 c = 0.5 (good oracle) c = 0.75 (very good oracle) c = 1 (perfect oracle) • Reward-Driven Request Prioritization (RDRP)

9. Overload (135%): Reward similar The bigger the better

10. Overload (135%): Response Times The smaller the better

11. Underload (80%): Response Times The smaller the better

12. Discussion • Main distinguishing features of RDRP • tries to predict future session’s reward • is oriented towards session completion • works in an abstract application-generic manner • May also take into account differences in user think times • helps to distinguish between different service usage profiles in the Bayesian inference analysis • What if clients do not show stable behavioral patterns?

13. file Thank You!