Enabling Prediction of Performance

1. Enabling Prediction of Performance September 16, 2004 Roger Skidmore Wireless Valley Communications, Inc. Roger Skidmore

2. Abstract Introduction What is Performance “Prediction”? Different Categories of Prediction How Does TGT Fit? Conclusion and Suggested Focus Roger Skidmore

3. Introduction • One of the goals of TGT is to enable prediction of performance • Effectively, anyone who needs to predict 802.11 performance is in TGT’s audience • Problem: How do we enable prediction of performance? • Prediction means different things to different people • What does it mean to predict performance? • The purpose of this presentation is to take the first step toward how TGT can assist those who need to do predictions Roger Skidmore

4. What is Performance “Prediction”? • Prediction means different things to different people • Is “prediction” a form of analysis, simulation, or something else entirely? • Depends on the level of comfort and confidence of the user • Most engineers are comfortable with measurements because they are “real” and “repeatable” and “mean something”, whereas predictions are viewed as dealing with “uncertainty” • This is valid to a limited degree • It is important to consider that people make all kinds of predictions every day • For example, consider that you can not deploy an 802.11 network without making myriad assumptions and “predictions” • A decision must be made about what to buy, where to put it, and how to configure it • Note that measurements taken during the design phase may not reflect the “real” situation once the network is live, and may also not be very “repeatable” depending on the environment Roger Skidmore

5. What is Performance “Prediction”? (cont.) • Performance prediction is a method of analysis that combines assumptions with accepted facts using algorithms and/or logic in order to reach a conclusion regarding the expected behavior of a device or group of devices under study • The benefit is that (potentially) untenable problems are simplified and conclusions reached more quickly that (hopefully) fall within an acceptable margin of error • Note that there is another layer of complexity and diversity in the types of predictions one can perform • For example, what is a device? • Am I planning a network, or building a better access point? Roger Skidmore

6. Different Categories of Predictions • Performance predictions can be categorized many ways based on what is being predicted • By layer (e.g., MAC performance, PHY performance, etc.) • Site-specific vs. non-site-specific • Physical network/device vs. logical network/device • Combinations of the above • The person carrying out the prediction could have multiple goals, and these may even conflict • For example, minimize number of APs but maximize coverage • It is outside the scope of this presentation to catalog all the different types of performance predictions currently in use • Instead, look for commonalities across major categories Roger Skidmore

7. Assumptions Algorithm and/or Logic Conclusion Accepted Facts Commonalities Across Predictions • Combining assumptions and accepted facts with an algorithm and/or logic to reach a conclusion • For the most part, Garbage in => Garbage out • Remember that the goal is to reach a conclusion with an acceptable margin of error Roger Skidmore

8. How Does TGT Help? • TGT can help improve performance predictions in several ways: • Providing increased number of Accepted Facts • E.g., How does a device respond when placed under certain conditions? • Providing increased number of valid Assumptions • E.g., Is a device capable of supporting a certain traffic load? • Providing increased confidence in the validity of Assumptions • E.g., How has a device performed under similar tests conditions? • Providing conclusions that can be used to empirically tune the Algorithm and/or refine the Logic (i.e., calibrate the prediction) • E.g., Device A produced this output when under a certain set of conditions/inputs. Reproduce those conditions/inputs in the performance prediction and tune the algorithm so that the predictive analysis matches the measured result as closely as possible. • The end result is a more accurate prediction of performance Roger Skidmore

9. Conclusion and Suggested Focus • Most performance predictions are done using software today • If TGT specified standard test conditions, methodologies, and a standard reporting format that could be easily parsed in software, many things can be automated • A specification for conducting and reporting tests across a range of variable inputs would allow for more precise device characterization • The level of the test (e.g., What layer? What type of device?) will determine what subset of performance predictions can utilize the data • For example, a battery life test may not contain data directly beneficial to an analysis of an access point’s coverage area • It will be impossible to address all possible tests – identify key tests with the most direct benefit across all of TGT’s audience • Even standardized generic tests will still be beneficial to performance predictions Roger Skidmore