Measurement Methodology Proposal based on Approved Framework

1. Measurement Methodology Proposal based on Approved Framework November 2004 Liam Quinn, Fahd Pirzada, Pratik Mehta <Dell> Paul Canaan <Intel> Amer Hassan <Microsoft> Dell

2. Abstract • Background and Scope • Proposal for Measurement Methodology • Measurement methodology for Case #1 (Data Applications) • Measurement methodology for battery-operated devices • Partioning approach for Case #2 (Streaming Media Applications) • Next Steps Dell

3. Proposed Timeline for TGt • Metrics/Sub-metrics, Methodology development Letter Ballot Mar’05 Framework Adoption Report In Berlin Sep’04 Framework Adoption Pre-Berlin 9thSep’04 Usage Case #1 Nov’04 Usage Case #2 Usage Case #3 Jan’05 Dell

4. Scope This proposal is for Usage Case #1 – Data-oriented usages… Dell

5. Usage + Environment  Metrics  Methodology  … Partitioning Approach for Usage Case #1 Case1: C 1. Data Oriented Applications + Home, Corporate Environment Approved as part of TGT focus from Berlin Meeting • Chariot Throughput + Indoor, Chamber Environment • Metrics: • Throughput, Range, Directionality • Sub-Metrics • Path loss, EVM measurements, Interference, Multipath, Noise, SNR, Receive sensitivity, Transmit power • Methodology: ? Case1: B • Web Downloads + Indoor Environment • Metrics: • Latency, Throughput, Range • Sub-Metrics • Path loss, EVM measurements, Interference, Multipath, Noise, SNR, Receive sensitivity, Transmit power • Methodology: ? Case1: A • File transfer, File sharing + Indoor Environment • Metrics: • Throughput, Range • Sub-Metrics • Path loss, EVM measurements, Interference, Multipath, Noise, SNR, Receive sensitivity, Transmit power • Methodology: ? Dell

6. Usage + Environment  Metrics  Methodology  … Usage Case #1 - Updated Case1: C 1. Data Oriented Applications + Home, Corporate Environment • Chariot Throughput + Indoor, Chamber Environment • Metrics: • Throughput, Range (dB), Directionality • Sub-Metrics • .. noted on later slide • Methodology: • - Over the air testing in controlled environment • - Conducted testing with Antenna Radiation Patterns Case1: B • Web Downloads + Indoor Environment • Metrics: • Latency, Throughput, Range • Sub-Metrics • Path loss, EVM measurements, Interference, Multipath, Noise, SNR, Receive sensitivity, Transmit power • Methodology: ? Case1: A • File transfer + Indoor Environment • Metrics: • Throughput, Range (meters) • Sub-Metrics • SNR, Walls (Number, Construction) • Methodology: • LOS/NLOS, 1 AP to 1 STA, Minimal interference, Turn table Dell

7. Case1: A – File Transfer + Indoor Environment • Metrics: Throughput (Mbps), Range (meters/feet) • Sub-Metrics: SNR, Walls (material, number) • Methodology: • Equipment and Setup • Single AP to single client (methodology for both APs and STAs) • Turn table (for averaging over orientations) • Minimal adjacent or co-channel interference • Measurement Steps • Timed file transfers: Upload and download • Line of sight and Non-LOS • Output to template • Described on next page • Concerns: • Repeatability Dell

8. Case1: A – File Transfer + Indoor Environment Output to template Throughput Range Measurement Equipment & Setup Wireless Access Point Device under Test File Server Upload Obstruction (if any) Download Turn Table x meters y meters Dell

9. Case1: A – LOS/NLOS Data (Indoor Testing) Dell Latitude D600 – 802.11b/g Linksys WRT54G – Mixed mode – Channel 6 QCheck TCP Throughput (1kbps data size – 3 iterations) 65 feet 5 feet 85 feet 125 feet • Note: • Existing 802.11g deployment in corporate environment (worst case scenario) • Mixed mode scenario for 802.11g (worst case scenario) • Low data size for QCheck Throughput tests (worst case scenario) • Variability in data points (standard deviation from average) • Conclusion • Better understanding of real-life scenarios • Significantly lower standard deviation with less interference and 802.11g only mode. Dell

10. Case1: A – LOS Data (Outdoor Testing) Dell Latitude D600 – 802.11g Netgear FWAG114 – Mixed mode – Channel 6 NTTCP Throughput • Note: • Low variability of throughput from run to run • Ability to perform crisp comparisons between various APs and WLAN clients • Close range data resembles testing in indoor LOS environment • Conclusion • The next best thing if a RF chamber is not available • Does not correlate to NLOS, multipath based indoor environments. Dell

11. Case1: B – Web downloads + Indoor Environment • Upon further study, this can be collapsed onto Case-1A Dell

12. Case1: C – Throughput + Indoor, Chamber Over-the-air Testing • Metrics: Throughput (Mbps), Range(dB) • Sub-Metrics: • EVM Measurements • Interference, Multipath, Noise, SNR • Total Isotropic Sensitivity, Total Radiated Power at various data rates (Foegelle) • FRMOL (Forwarding rate under max. offered load) (Mandeville) • MSDU loss rate (MAC service data unit) (Mandeville) • Methodology: • Equipment and Setup Foegelle - IEEE 802.11-03/675r1 • Measurement Steps • Output to template Dell

13. Case1: C – Throughput + Indoor, Chamber Conducted Testing + Antenna Radiation Pattern • Metrics: Throughput (Mbps), Range(dB) • Sub-Metrics: All the metrics mentioned on last slide + ARP • Methodology: • Equipment and Setup • Measurement Steps • Output to template Reference: Charles Wright et al - 802.11-04/1157 • Concerns: Multiple APs and impact on clients (roaming/switching/load balancing) Dell

14. Analysis & Correlation: Usage Case #1 Case1: C 1. Data Oriented Applications + Home, Corporate Environment Throughput + Chamber Environment Case1: B Web Downloads + Indoor Environment MAC/ PHY Characteristics Case1: A Throughput + Indoor Environment Analysis & Validation Application layer characteristics Correlation • Propagation models: Attenuation + Interference (multipath, fading etc.) = Range • PHY Layer: Receiver Sensitivity + Antenna Radiation Pattern = Range • MAC Layer: Forwarding Rate + Loss = Throughput • …... Dell

15. Case1: D – Battery Usage + Indoor Environment Methodology for Battery-operated 802.11 devices • Metrics: Battery Life, Response Time • Sub-Metrics: Power draw (mW), Throughput • Methodology: • Equipment and Setup • Single AP to single client • Minimal adjacent or co-channel interference • Data + Multimedia workload downloaded to web browser application • Measurement Steps • Response Time (Latency) of workload on client • Battery life at the end of run • Output to template • Concerns: • Relationship of WLAN sub-system power draw with system battery life Dell

16. Partitioning Approach for Usage Case #2 Case2: C 2. Streaming Media Apps + Home, Corporate Environment • AV streaming + Controlled Environment • Metrics: • Frames dropped, Image quality, Range. • Sub-Metrics • Multipath, Noise, SNR, Forwarding rate, loss rate. • Methodology: ? Case2: B • Multiple AV streams + Indoor Environment • Metrics: • Frames dropped, Image quality, Range, Bandwidth utilization. • Sub-Metrics • Interference, Multipath, Noise, SNR, Throughput • Methodology: ? Case2: A • SDTV/HDTV quality streaming + Indoor Environment • Metrics: • Frames dropped, Image quality, Range • Sub-Metrics • Interference, Multipath, Noise, SNR. • Methodology: ? Dell

17. Next Steps • Get agreement on the methodology for Usage Case #1 based on the approved framework • Align and approve in San Antonio meeting in TGT • Develop the Usage Cases #2 and #3 further • Sanitize and update the metrics and sub-metrics • Hone in on the Methodology components, test environment, procedure, etc. • Finalizing the test case templates for reporting and analysis • Draft recommended practices specification • Include definitions and terminology Dell

18. Backup Dell

19. Parking Lot for Metrics and Sub-metrics Proposing a “parking-lot” list of parameters – determine whether they are metrics or sub-metrics, or part of the methodology, and which of the usage cases they apply to … Performance metrics by OSI model layers Dell