Capturing an Experiment: A Wireless Testbed Perspective

Capturing an Experiment: A Wireless Testbed Perspective • Ivan Seskar • Seskar (at) winlab (dot) rutgers (dot) edu • Rutgers, The State University of New Jersey • www.winlab.rutgers.edu

Experiment Support Architecture Deploy & Configure Control Definition System Experimenter Measure Results

Orbit Services

Orbit Measurement Library (OML)

Defining an Experiment

Experiment Definition OMF Experiment Description Language, OEDL # # Configure environment # node("/*/*").net.w0 {|n| n.essid = "HelloWorld" n.mode = "ad-hoc" n.channel = 1 n.xmitPower = 1 n.bitrate = "11Mbps" # Force single rate n.ip = "%10.0.%x.%y" } # Start experiment whenReady {|e, n| p.packetSize = 1024 p.rate = 250 e.sleep 5.0 e.done } Experiment.name = "tutorial-1" Experiment.project = "Orbit::Tutorial" Experiment.startMode = Experiment::REBOOT # # Define nodes used in experiment # node([2, 3], :sender) {|n| n.prototype("http://apps.orbit-lab.org/sender", { :if => Node::W0_IF, :packetSize=>128 Experiment.parameter("packetSize"), :rate => Experiment.parameter("rate") }) } node([3, 4]) {|n| n.prototype("http://apps.orbit-lab.org/receiver", {:if => Node::W0_IF}) }

ORBIT Grid

IssuesAffecting Archiving/Repeatability Hardware • Differences in commercial hardware • Difference in tolerances and ageing for various low cost components Software • Software or firmware bugs that may manifest as inconsistent experimental results Environment • Uncontrolled interference over time and space • Co-located infrastructure, opening and closing of doors, movement of people etc.

Device Calibration Setup

Calibration: Device Characterization Transmitter calibration • Calibrate transmit and receive side of the wireless cards • Record corrections to be applied for each card based on the discrepancies observed Reciver calibration

Calibration: Temporal repeatability The Experiment

Calibration: Spatial repeatability Blue sender sends to blue receiver Yellow sender sends to yellow receiver Experiment repeated on a different set of nodes with same topology Spatial throughput variations w.r.t mean for experiment duration averaged over different experimental runs

Calibration • Node calibration procedure necessary to address hardware dependencies • Temporal variation less than device variability • Periodic re-calibration necessary • Need to automate the on-site calibration procedure so that it can “monitor” the testbed “drift” • Bounds on acceptable performance • Baseline standard

Archiving ORBIT Experiments Existing Support: Image Experiment script and/or commands on the console State of services and environmental traces Measurements (if using OML) Device calibration info What is missing Full metadata State of devices Full RF environment (not necessarily critical for repeatability)

Wild Environment: WiMAXBasestation Basestation (IDU) Unit RF (ODU) Amplifier Roof Antenna • Wide-area packet based communication system (4G) • Operational with an educational license

Wild Environment: Coverage Measured RSSI

Wild Environment: Mobility Trajectory RSSI

Instead of Conclusion Repeatability/replayability incentive Standardized anomaly monitoring Storing metadata in archives Baseline performance tests and calibration

Capturing an Experiment: A Wireless Testbed Perspective