Bottom-Up Evaluation of 802.11 Performance Testing

1. September 2004 • doc.: IEEE 802.11-04-1156-00-000t Bottom-Up Evaluation of 802.11 Performance Testing Dr. Michael D. Foegelle ETS-Lindgren Submission Foegelle, ETS-Lindgren

2. Overview • Introduction • Layer Analysis • Basic Operational Testing • Environmental Effects • Summary & Conclusion Foegelle, ETS-Lindgren

3. Introduction • Top-down approach of defining usage cases and working down to resulting metrics and needed tests is a useful starting point. • However, tendency to focus on usage case scenarios is hindering determination of wireless performance parameters needed in order to progress towards development of equivalent laboratory techniques . Foegelle, ETS-Lindgren

4. Introduction • This presentation will use a bottom-up approach to identify and evaluate critical contributions of the wireless portion of the DUT, concentrating primarily on the physical layer/radio effects. Foegelle, ETS-Lindgren

5. Layer Analysis • In the ISO 7-layer network model, only the bottom two layers (Data Link and Physical) have anything to do with the device being wireless as opposed to any other network device. Foegelle, ETS-Lindgren

6. Layer Analysis • The tests necessary to evaluate the upper layers should not differ from testing performed on non-wireless network devices. • Only indirect contribution of lower levels make this any different from non-wireless devices. Foegelle, ETS-Lindgren

7. Layer Analysis • Effects of environment (location, interference, etc.) directly affect performance of physical layer, which in turn affects the MAC layer and so on, all the way up the chain. • Analysis of environmental effects on the physical layer, starting at the radio, will show that it is possible to determine well defined laboratory techniques to duplicate these effects. Foegelle, ETS-Lindgren

8. Basic Operational Testing • Before we talk about performance testing, there’s a level of testing required to verify that the device is actually operational. • Radio functionality • Ability to generate required signals and modulations at expected levels. • Ability to detect/decode input signals in a nominal range. Foegelle, ETS-Lindgren

9. Basic Operational Testing • Additional PHY layer functionality • Ability to encode packets and handle shared media. • MAC layer functionality • Ability to establish connection, perform handshaking, and transfer data. • Some level of standardized functionality testing is assumed prior to attempting performance testing. Foegelle, ETS-Lindgren

10. Environmental Effects: Distance • The simplest environmental effect to evaluate is that of distance on direct line-of-sight communication. • Signal level drops as distance increases or as signal passes through intervening obstructions. • Relative phase relationship of signal remains constant over time. • No actual degradation of signal content. Foegelle, ETS-Lindgren

11. Environmental Effects: Distance • Implies evaluation of basic radio performance (transmit power and sensitivity) • Ability to detect a coherent signal • Ability to demodulate/decode a given complexity of modulation (data rate control) at a given signal level • “Clean” signal – No interference • Secondary effect of propagation delay unlikely to be significant for range of 802.11 • ~0.33 S at 100 meters Foegelle, ETS-Lindgren

12. Environmental Effects: Distance • This testing can easily be performed as a conducted test using a calibrated radio and a reference 802.11 interface. • Reference interface provides known good communication behavior • Calibrated radio provides traceable: • Power and signal quality measurements of DUT transmit signal at various data rates. • Variable forward power received by the DUT to test sensitivity and data rate control/feedback. Foegelle, ETS-Lindgren

13. Environmental Effects: Multipath • Multipath effects can be sub-divided and investigated separately. • Multipath fading in its traditional sense refers to the reduction in received signal due to the location of the receive antenna in a field null. • Null caused by the summation of one or more out-of-phase signals with similar amplitudes. • Dynamic fading refers to the effect seen when motion causes the receiver to move through nulls. Foegelle, ETS-Lindgren

14. Environmental Effects: Multipath • The signal level effects of fading (i.e. similar path-length, out-of-phase signals) would have a similar effect on performance as that for distance. • Since fading (null location) is frequency dependent, spread spectrum signals offer some rejection to fading. • Accurate test for fading would entail use of variable notch filters (fading simulators) to create a frequency dependent fade behavior. • Testing for dynamic fading would entail determining the device’s response to dynamically changing faded signals. Foegelle, ETS-Lindgren

15. Environmental Effects: Multipath • The term fading refers to audio behavior of traditional amplitude modulation. • As signal amplitude fades, so does audio amplitude. Sound fades out. • At audio frequencies, modulation effects of multipath are negligible. • At high data rates, modulation rates approach path lengths (11 MHz chip rate ~ 27 meters), so modulation corruption due to varying multipath lengths becomes an issue. Foegelle, ETS-Lindgren

16. Environmental Effects: Multipath • Worst cases of modulation corruption likely for two similar strength, in-phase signals with considerably different path lengths. • E.g. LOS through wall (lossy) and NLOS from distand reflector • Results in strong signal for portions of some bits, nulls for others (dependent on phase modulation at given bits). • SS demodulation techniques are designed to decode through multipath effects, but suitable test cases are desirable. Foegelle, ETS-Lindgren

17. Environmental Effects: Multipath • Again, behavior can be simulated through cabled test. • Brute force: split signal and use delay lines and attenuators to create a true multi-path input. • Custom phase locked transmitters capable of generating identical signals with desired delay and amplitude variations. • Note that multipath tests need only be receiver side tests. Foegelle, ETS-Lindgren

18. Environmental Effects: Antenna • In general, the antenna is a modifier to the RF performance previously discussed. • An antenna simply integrates (or adds up) the various signals directed towards it, with a weighting factor as a function of both propagation and polarization directions. • Directionality can change magnitude of multipath effect seen by radio by minimizing contribution of other paths, but does not change radio’s innate ability to handle a given multipath effect. • Does interact with circuitry and nearfield environment, changing performance (see 11-04-0675) • Radiated over-the-air tests need only verify radio TX/RX performance with antenna loading and interaction. -- No need to perform all tests OTA. Foegelle, ETS-Lindgren

19. Environmental Effects: Network • Once an 802.11 device becomes part of a wireless network with multiple endpoints, there are a variety of interactions relating to both the PHY and MAC layers that should be evaluated. • These include physical effects: • Out of band signals and tolerances • Collisions, cross-talk, etc. • Hidden endpoints & network overlap. • And protocol related effects: • Timing, holdoff, etc. • Multi-point access control • Handshaking Foegelle, ETS-Lindgren

20. Environmental Effects: Network • There are a number of “network stress tester” type products on the market designed to simulate multiple devices on a network. • These products typically simulate a multitude of MAC layers operating through one radio. • Single radio approach can’t simulate real world collisions and other RF interaction behaviors. • Some level of synchronization already exists above the PHY layer to allow unit to operate. • A multi-radio device is needed to properly simulate real-world network behavior. Foegelle, ETS-Lindgren

21. Environmental Effects: Interference • Interference testing may be more of a functional test than a performance test. • Any properly operating radio/PHY layer should expect to have similar susceptibility to a given level and type of interference. • Outside interference is not a predictable event. • Can simulate a wide variety of interference, but the resulting metric is not likely to represent any particular real-world behavior. • As much EMC as operational or performance related. Foegelle, ETS-Lindgren

22. Summary & Conclusion • This is an incomplete list intended to get the task group thinking in terms of the input parameters to overall performance metrics. • It’s not necessary to set up real-world networks to simulate real-world effects. Most tests can be done with a cabled connection to appropriately designed test and measurement equipment or systems. • Any number of the metrics discussed by the TG may be evaluated, as needed, but they should be evaluated in terms of the factors that contribute to variation in each level of the 802.11 layer structure. Foegelle, ETS-Lindgren

23. Summary & Conclusion • Many metrics may have the same interaction with the available input parameters. It may not be necessary to test all conceivable metrics as a function of these parameters. • We need to avoid getting caught up in upper level metrics that may not change due to the use of a wireless network as opposed to any other network. • We should pick a starting point and develop some experimental test plans around currently available equipment for simulating the wireless effects. Foegelle, ETS-Lindgren

24. Summary & Conclusion • Point-to-point behavior is easy to understand and should be an initial starting point for radio/PHY evaluation. • Network stress testers are a good starting point for MAC level testing. • Additional effects and interactions may be defined and investigated as we proceed, building on what we learn with the simpler tasks. • New test equipment or combined rack systems will need to be developed to carry this effort forward. Foegelle, ETS-Lindgren