1 / 14

Testing adaptive workload management

Testing adaptive workload management. Harumi Kuno HP Labs. Stefan Krompass (TUM), Kevin Wilkinson, Umeshwar Dayal, Goetz Graefe, Janet Wiener. Performance. Expected conditions (work + resources). Actual conditions (work + resources). Compile-time Query Optimization. Run-time Query

hedley-ellison
Download Presentation

Testing adaptive workload management

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Testing adaptive workload management Harumi Kuno HP Labs Stefan Krompass (TUM), Kevin Wilkinson, Umeshwar Dayal, Goetz Graefe, Janet Wiener

  2. Performance Expectedconditions (work + resources) Actualconditions (work + resources) Compile-time QueryOptimization Run-timeQuery Execution QueryPlan

  3. Controlling resources for a complex (dynamic mixed) workload is hard Traditional solution is to isolate components, by partitioning work across hardware or time multiplexing.

  4. Grand Challenge (1):Managing dynamic mixed workloads • Ignore it • Avoid problem through isolating systems or using time multiplexing • Provide rich tools to be used with manual workload management • Adaptive workload management

  5. Dynamic mixed workloads are difficult to manage because resource contention … changes resource requirements Disk and memory usage needed to execute a sort changes with the amount of available memory Goetz Graefe, Harumi Kuno, Janet Wiener. Visualizing the Robustness of Query Execution. Proc. Conference on Innovative Data Systems Research (CIDR). January 4-7, 2009.

  6. Dynamic mixed workloads are difficult to manage because resource contention … changes performance Throughput/MPL for a single homogenous workload varies with different cache hit rates Janet L. Wiener, Harumi Kuno, Goetz Graefe. Benchmarking Query Execution Robustness. First TPC Technology Conference on Performance Evaluation and Benchmarking (http://www.tpc.org/tpctc2009), held in conjunction with VLDB 2009.

  7. Dynamic mixed workloads are difficult to manage because resource contention … and is difficult to predict Report 1 Report 2 Report 3 Throughput/MPL for OLTP queries changes as various report queries run Stefan Krompass, Harumi Kuno, Janet L.Wiener, Kevin Wilkinson, Umeshwar Dayal, Alfons Kemper. A Testbed for Managing Dynamic Mixed Workloads. Demonstration at VLDB 2009.

  8. Can static workload management policies handle unreliable cost estimates? under-informed admission control and scheduling decisions Stefan Krompass, Harumi Kuno, Janet L.Wiener, Kevin Wilkinson, Umeshwar Dayal, Alfons Kemper. A Testbed for Managing Dynamic Mixed Workloads. Demonstration at VLDB 2009.

  9. Can static workload management policies handle unobserved resource contention? monitored resource not the source of contention Stefan Krompass, Harumi Kuno, Janet Wiener, Kevin Wilkinson, Umeshwar Dayal, Alfons Kemper. Managing Long-Running Queries. Proc. EDBT 2009.

  10. Can static workload management policies handle system overload? No. Challenge: Static policies have a hard time correcting overload situations. Stefan Krompass, Harumi Kuno, Janet Wiener, Kevin Wilkinson, Umeshwar Dayal, Alfons Kemper. Managing Long-Running Queries. Proc. EDBT 2009.

  11. Adaptive Workload Management Hypothesis We can build a system that uses feedback to add a policy control loop. Stefan Krompass, Harumi Kuno, Janet Wiener, Kevin Wilkinson, Umeshwar Dayal, Alfons Kemper. Managing Long-Running Queries. Proc. EDBT 2009. ‹#›

  12. Testing Adaptive Workload Management • Functionality of management tools • Configuration for a particular anticipated workload • How it handles the unexpected. Grand Challenge (2)

  13. Underlying challenges • Characterize workloads and service classes: queries, mega-queries (graphs of queries considered as a single unit), loads, continuous inserts, etc. Plus usual features: estimated arrival rates, execution times, resource usage, etc., together with SLOs. • Map the components of the workload to service classes. • Develop recommendations for policies for the query control loop and policy control loop. • Consider elasticity (dynamic scale out of resources) and outage avoidance. • Evaluate responses to the unexpected. ‹#›

  14. You’d like to make decisions are made based on expected performance, but…. • BI queries -> skew + complex queries • Skew + complex queries -> unexpected amount of work. • More work -> more resource usage. • Multiple queries with unexpected resource usage -> unexpected resource availability. Runtime performance really, really hard to predict Performance harder to predict unexpected Resource Availability Performance predictable Performance harder to predict expected expected unexpected Resource Requirements (e.g., degree of skew) ‹#›

More Related