1 / 24

Experience Predicting Application Server Memory Usage

Experience Predicting Application Server Memory Usage. Thomas Reidemeister Mohammad A. Munawar Paul A.S.Ward University of Waterloo April 2005. Outline. Introduction Difficulty of predicting in dynamic systems Case Study: Java-based Application server

mkibbe
Download Presentation

Experience Predicting Application Server Memory Usage

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Experience Predicting Application Server Memory Usage Thomas Reidemeister Mohammad A. Munawar Paul A.S.Ward University of Waterloo April 2005

  2. Outline • Introduction • Difficulty of predicting in dynamic systems • Case Study: Java-based Application server • A simple approach to predicting memory-usage • Some experiments • Work in progress

  3. Introduction • Need for monitoring system health • Performance • Security • Availability • Correctness, etc • Sources of information • Events (logs, notifications, etc) • System status (resources, parameters, etc) • Purpose • Anomaly detection and diagnosis • Dynamic reconfiguration • Planning

  4. Building a Model WorkloadGenerator ResponseMonitor System Under Observation Model Builder Model

  5. Using the Model Workload ResponseMonitor System Under Operation System Model Problem Determination Tool

  6. Dynamic Environments • Changing nature • System configurations can change • System components (software/hardware) can be updated/added/removed • Examples • Operating systems, Applications using Virtual Machines • Java-based applications • Java Virtual Machine (JVM) • Automatic memory management • Dynamic class-loading • Just-in-time compilation • Online byte-code instrumentation, etc

  7. Uncertainty • Why is prediction is difficult? • Beside lack of knowledge of the future, we have to deal with: • Availability of information • Quality of available information • Performance impact • Dynamic environments (adaptive behaviour) • Complexity of the underlying process and dependencies

  8. Case Study • Predicting memory-usage of a Java-based application server • Memory leaks and resource exhaustion is a severe problem • Automatic memory management makes it interesting • Prediction can help with • Load-balancing • Capacity planning • Possibly leak-detection • What can we achieve with the simplest approach and information that is readily available?

  9. Approach

  10. Setup • Software: • IBM WebSphere Application Server 5.1 running Trade3 • IBM DB2 UDB 8.1 • ‘httperf’ and httpunit-based custom workload generators • Hardware • P4 2.0 GHz, 1024MB machine with Windows Server 2003 running WebSphere • P4 2.0 GHz, 512MB machine with Redhat Linux running DB2 • Dual-processor Xeon 2.8GHz with Redhat Linux for load generation • Connected with a Gbit-network

  11. Profiling • Obtain low-level memory allocation information from the JVM • Use timing of request execution to match associated memory allocation • Execute requests sequentially and at reasonably distant time-points to better isolate their effects • Repeat requests a sufficient number of times and with different parameter values to obtain good representative statistics • Build histograms for each request-type and select mode as the representative value

  12. Profiling (cont.) • Examples Memory bytes allocated by an idle server Histogram of a request-type

  13. Profiling (cont.) • Issues • Typically, only millisecond time-resolution is available. How much memory does a sub-millisecond request use? When requests overlap in a millisecond-interval, we apportion equally • Should we isolate unusual, irregular memory-allocation due to internal house-keeping? • Profiles can change. A search request may require more memory depending how many add/updates have taken place since the last profiling time • Profiling requires a dedicated and isolated server

  14. Monitoring • Precise memory-usage information is not readily available • For example, Websphere interface to query memory-usage has a KB resolution. • Reported memory-usage usually includes objects no more referenced and waiting to be freed • Relatively precise information can be obtained from the JVM, if we profile GC events. However, the profiling interface does not provide a complete picture • Let us monitor at regular interval, but at some reasonable granularity. We thus have an approximate actual usage.

  15. Monitoring (cont.)

  16. Prediction • Predict number of requests of each type that we expect to see • Use the above together with the profiles created earlier to estimate memory-usage • Any technique to project future requests can be used. e.g. simple Exponential Smoothing • What prediction period to use? • Fixed-duration: memory usage information is approximate • Variable-duration (e.g. time to next GC event): need for estimating interval length

  17. Prediction (cont.) • Experiments • How close do we get to actual memory usage, if we knew exactly the number of each request-type received during an interval? • Let us use variable-duration intervals, which correspond to GC events observed to estimate actual memory-usage • The prediction model is instantiated with request counts and request-type profile (mode-values) • No model fine-tuning

  18. Prediction (cont.)

  19. Results • We slightly under-estimate in general, but closely follow the trend when a sufficient number of requests are received during the prediction interval • With some calibration, we should be able to achieve better results • Caching is an important factor that needs to be incorporated in our prediction model

  20. Improving Prediction • Build better profiles • Isolate object allocation due to server house-keeping • Deal with intervals with small number of requests • Use a different model? • Take caching into account • Model effect of present memory allocations on future requests

  21. Fine-grained Monitoring • Our approach can be used for coarse-grained monitoring • What if we wanted closer monitoring of the absolute memory used at any time? • Can we develop a general technique that can, for example, help detect memory-leaks? • Again, we would need to model the behaviour of the JVM in details • E.g. take into account the effect of requests, occurrence of GC events, caching, etc • Exploring Dynamic Bayesian Networks

  22. Summary • Described a simple approach to predicting memory-usage of an application server • Mentioned difficulties and issues • Showed that this approach is able to closely follow memory-usage trend • Depending on purpose, such simple prediction models can be very effective • Prediction in dynamic systems is tricky and require that we resort to approximations • But, we need to be intelligent about how to approximate

  23. Our Research • Resource monitoring for anomaly-detection in dynamic environments • Associating status information (e.g. resource usage) with discrete system events for problem detection and diagnosis • Adaptive monitoring

  24. End Thank you. Questions are welcome.

More Related