1 / 16

Choosing an Accurate Network Model using Domain Analysis

Choosing an Accurate Network Model using Domain Analysis. Almudena Konrad, Mills College Ben Y. Zhao, UC Santa Barbara Anthony Joseph, UC Berkeley The First International Workshop on Performance Modelling in Wired, Wireless, Mobile Networking and Computing (PMW2MNC, July 2005).

alissa
Download Presentation

Choosing an Accurate Network Model using Domain Analysis

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Choosing an Accurate Network Modelusing Domain Analysis Almudena Konrad, Mills College Ben Y. Zhao, UC Santa Barbara Anthony Joseph, UC Berkeley The First International Workshop on Performance Modelling in Wired, Wireless, Mobile Networking and Computing (PMW2MNC, July 2005)

  2. Modeling Network Measurements • Model-driven traces as experimental tool • Real measurements costly to perform • Produce models by extracting statistics from measurements • Use model to generate traces w/ “similar” characteristics • New networks (wireless) difficult to model • Traditional models require stationary statistics • Gilbert model, Higher order DTMC, Hidden Markov Models (HMM) • Artifacts disrupt stationarity of wireless traces • Bursty error or delays due to signal interference or loss • Result: traditional models generate inaccurate results • Our solution: • Data preconditioning: preprocess traces before extracting model • Result: MTA, MMTA accurately model wireless traces

  3. A New Challenge • Many mathematical models are available • Gilbert, Higher order DTMC, HMM, MTA, MMTA… • Each optimized for certain characteristics • Each with different computational and memory costs • Different networks exhibit different characteristics • IP networks, wireless (802.11), sensor nets • The challenge • Match up network type with more accurate, lowest cost model • No tools or techniques to do this • Our solution: • Domain analysis

  4. Outline • Motivation • Data preconditioning and its models • The Markov-based Trace Analysis model (MTA) • The Multiple states MTA model (MMTA) • Quantifying modeling accuracy • Domain Analysis • Conclusion

  5. Modeling using Data Preconditioning • Collect network characteristic trace • Binary traces show occurrence of an event, e.g. lost or delayed frame • Identify data patterns or “states” (regions with stationary behavior) • Precondition the data to fit traditional models • Calculate probability distribution for each state • Determine transition probabilities among states Collected Trace Subtrace 1 Subtrace 2 Subtrace 3

  6. Preconditioning Model (MTA) • MTA: Markov-based Trace Analysis Algorithm • Two states: lossy & error-free states • Create two subtraces  lossy & error-free subtraces • Algorithm to optimize the change-of-state variable, C • For a given trace, executes stationary test for large range of values C • Choose the highest C that yields a stationary lossy states • Model subtraces as Exponential distributions Lossy Error-free Lossy Error-free States: C C …10001110011100….0 0000…0000 11001100…00 00000..000... Trace: ...10001110011100….0 11001100…00 ... Error-free sub-trace: Lossy sub-trace: … 0000…0000 00000..000...

  7. Preconditioning Model (MMTA) • MMTA:Multiple states MTA Algorithm • Allows for multiple states in the data • Identifies states in original trace • Concatenates similar states to create subtraces • Uses a DTMC to model subtraces & transition between states

  8. Network Path Traces • A selection of end to end & wireless layer path traces …0000000 10101111 00000000000… • End-to-end IP traces (IP-1) • End-to-end WLAN delay trace (WLAN-D) • Wireless WLAN trace (WLAN-E) • Wireless GSM trace (GSM-E) • Trace collected at the Radio Link Protocol (RLP) • Traces can be decomposed into two stationary states • Lossy & error-free states • Characterize traces with three parameters (Lexp, EFexp, Lden) • Exponential length distributions of lossy and error-free states • Lossy error density

  9. Quantifying Accuracy of Models • Previously used metrics • FER and simulated transmission time • No actual correlation to error distribution • How accurate is a particular model? • Synthetic traces comparison (Lexp, EFexp, Lden) • Compute correlation coefficient between error-burst distributions • Reference trace: • (Lexp, EFexp, Lden)= • (0.006, 0.1, 1) • Relationship between cc • and accuracy: • CC > 0.96 indicates • an accurate model

  10. Example of Model Accuracy:GSM Error Burst Distributions • GSM, MTA & MMTA experience similar error burst characteristics • Gilbert, 3rd order and HMM don’t reproduce large error burst CC values Gilbert: 0.74 HMM : 0.89 MTA : 0.99 MMTA : 0.99

  11. Choosing the Right Network Model:Domain Analysis • Preconditioning models also work well for traditional networks • There could be a “bursty” nature to congestion loss in IP networks • Key is the presence of “bursty” behavior • Need a way to choose optimal model for given network • Solution: domain analysis • Domain analysis • Create Domain Analysis Plots (DAPs) for wide range of parameters • Collect empirical packet trace: T= …00110110100… • 1: corrupted/delayed packet, 0: normal packet • Calculate parameters (Lexp, EFexp, Lden) for T • Use DAPs to lookup optimal model

  12. real network metric trace trace analysis algorithm network model artificial network metric trace Domain Analysis Plots (DAPs) • Generate reference traces (T1..Tx) • Range of values (Lexp, EFexp, Lden) • For each reference trace Ti create • mathematical models • artificial traces from models • Plot error and error-free distribution • Calculate CC between reference and artificial distributions • Optimal model for Ti => models that yield highest CC value

  13. DAPs for (Lexp,EFexp) 0.001 -> 0.1 • DAP 1 Lden=0.2 • Gilbert region CC values (Gilbert = 0.99) • MMTA = 0.98 • MMTA region CC values (MMTA =0.97) • Gilbert = 0.96 • DAP 2 Lden = 0.4 • Gilbert region CC values (Gilbert = 0.99) • MTA = 0.97 • MTA region CC values (MTA = 0.98) • MMTA region CC values (M3=0.97)

  14. DAP for (Lexp,EFexp) 0.001 -> 0.1 • DAP 3 Lden=0.7 • MMTA region CC values (MMTA=0.98) • MTA=0.97 • MTA region CC values (MTA=0.99) • MMTA=0.99

  15. Conclusions • The challenge • How to choose the most accurate, least cost model for each network type • The solution • Key metric for modeling accuracy by error burst distributions • Domain analysis plots show optimal model for different networks • Using domain analysis • Take your own traces for network X • Calculate the three parameter values • Use DAP to determine optimal model for your trace

  16. Thank you Questions? akonrad@mills.edu

More Related