360 likes | 1.75k Views
Agenda. Definitions and TermsSteps in Capacity Planning and ManagementProblems in Capacity PlanningCommon Mistakes in BenchmarkingComputer System Performance MeasurementLoad DriversRemote Terminal EmulationSummary and Conclusion. Definitions and Terms. Capacity planning: Ensuring that adequa
E N D
1. Capacity Planning and Benchmarking(Chapter 9 of �The Art of Computer Systems Performance Analysis� by Raj Jain) Presenter:
Mikael Fernandus Simalango
WISE Research Lab
2. Agenda Definitions and Terms
Steps in Capacity Planning and Management
Problems in Capacity Planning
Common Mistakes in Benchmarking
Computer System Performance Measurement
Load Drivers
Remote Terminal Emulation
Summary and Conclusion
3. Definitions and Terms Capacity planning:
Ensuring that adequate computer resources will be available to meet the future workload demands
Should be cost-effective and meet the performance objectives
Capacity management:
Ensuring that currently available computing resources are used to provide the highest performance
Performance tuning:
The process of adjusting system parameters to optimize the performance.
Benchmarking:
Comparing the performance of two competing systems
Is run on systems using automatic load drivers
4. Steps in Capacity Planning and Management Steps in capacity planning process
5. Steps in Capacity Planning and Management Steps in capacity management
6. Problems in Capacity Planning Common problems:
No standard terminology
Different definitions from different vendors
No standard definition of capacity
Maximum throughput ? Maximum number of users?
Different capacities for the same system
Nominal capacity: maximum achievable throughput under ideal workload conditions
Usable capacity: maximum throughput achievable without exceeding a prespecified response time limit
Knee capacity: optimal operating point of throughput and workload
No standard workload unit
Users, sessions, MIPS?
7. Problems in Capacity Planning Common problems:
Forecasting future applications is difficult
Regression vs technology advancement which may invalidate the previous assumption
No uniformity among systems from different vendors
Same workload but different amounts of resources on different system
Model input cannot always be measured
Input defined in simulation may not be conforming in real world
Validating model projections is difficult
Difficult to control the workload and configuration on a real system
Distributed environments are too complex to model
Performance is only a small part of capacity planning problem
8. Common Mistakes in Benchmarking List of repeated mistakes:
Only average behavior represented in test workload
Skewness of device demands ignored
Loading level controlled inappropriately
Caching effects ignored
Buffering sizes not appropriate
Inaccuracies due to sampling ignored
Ignoring monitoring overhead
Not validating measurements
Not ensuring same initial conditions
Not measuring transient performance
Using device utilizations for performance comparisons
Collecting too much data with very little analysis
9. Computer System Performance Measurement � Load Drivers Load driving: putting loads on the system
Load drivers: mechanism or tool for putting loads on the system
Purpose of load drivers:
Performance measurement
Component certification -> ensuring functionality of component under different workload demands
System integration -> verifying compatibility of components under different environments
Stress-load analysis -> Analysis of system stability for high load
Regression testing -> ensuring previous capabilities are functional along with the new ones
10. Computer System Performance Measurement � Load Drivers Load driving techniques:
Internal driver: loading programs directly into the memory and executing -> applying a batch job
Operators: having live operators utilize the system
Remote Terminal Emulators (RTEs): using computers to simulate many users in a very controlled and repeatable fashion
11. Computer System Performance Measurement � RTE About RTE:
A full fledged computer which includes disks, persistent storage, and at least one console terminal
Executes load driving by sending commands to the SUT at appropriate intervals through an automated script.
Has three components:
Pre-emulation: defining system configurations (terminals, users, programs)
Emulation: collecting data of SUT by RTE
Post-emulation: transferring data to another computer system for reduction and analysis
12. Computer System Performance Measurement � RTE RTE Components
13. Computer System Performance Measurement � RTE Limitations of current RTEs:
The conditions for sending successive requests may not be realistic (should use probabilistic transmissions with varying contents of requests instead)
It may not be possible to vary the input in successive repetitions
Modern user interfaces are not emulated (mouse click, function keys, etc)
Users are the shared resource (multiple requests at the same time)
Interfaces and definitions are different across RTEs
14. Summary and Conclusion Capacity planning and management is a chain of activities
Ensuring that user requests can be served at the optimized cost and performance
And anticipating future workload demands
RTEs:
Trying to emulate workloads of SUT
Resulting in optimized system profile/configuration for processing the workloads
Having some limitations which may render analysis of emulated data inappropriate with real-world scenario
15. Summary and Conclusion Linking capacity planning with cloud computing, we may derive:
Understanding that capacity planning is related to performance and scalability of the cloud
Questions about what parameters are essential for measuring the capacity of the cloud in relation with its performance
Questions about how to model a cloud system and get the optimized capacity
Questions about how to implement a high availability cloud based on the derived capacity model