Time Synchronization

1. Time Synchronization Saurabh Ganeriwal – saurabh@ee.ucla.edu EE202A Lecture #10 Nov 22, 2004

2. Reading list: required • RATS - S. Ganeriwal, D. Ganesan, M. Hansen, M. B. Srivastava, D. Estrin, “Predictive time synchronization for long-lived sensor networks,” NESL Technical Report, November 2004. http://www.ee.ucla.edu/~saurabh/upload/Technical_Report_RATS.pdf • FTSP - M. Maroti, B. Kusy, G. Simon, A. Ledeczi, “The Flooding Time Synchronization Protocol,” In the Proceedings of the Second ACM Conference on Embedded Networked Sensor Systems (SenSys), 2004. http://portal.acm.org/citation.cfm?id=1031495.1031501 NESL

3. Reading list: recommended • RBS - J. Elson, L. Girod, D. Estrin, “Fine-Grained Network Time Synchronization using Reference Broadcasts,” In Proceedings of the Fifth Symposium on Operating Systems Design and Implementation (OSDI 2002), Boston, MA. December 2002. http://portal.acm.org/citation.cfm?id=844143 • TPSN - S. Ganeriwal, R. Kumar, M. B. Srivastava, “Timing-sync protocol for sensor networks,” In Proceedings of the First ACM Conference on Embedded Networked Sensor Systems (SenSys), 2003. http://portal.acm.org/citation.cfm?id=958508 • NTP - D. Mills, “Internet Time Synchronization: The Network Time Protocol,” In Zhonghua Yang and T. Anthony Marsland, editors, Global States and Time in Distributed Systems, IEEE Computer Society Press, 1994. http://www.eecis.udel.edu/~mills/database/papers/trans.pdf • Leslie Lamport, “Time, clocks, and the ordering of events in a distributed system}, journal”, ACM Journal of communications, 1978. http://portal.acm.org/citation.cfm?id=359563 NESL

4. The Myth of Simultaneity: “Event 1 and event 2 at same time” Event 1 Event 2 Observer A: Event 2 is earlier than Event 1 Observer B: Event 2 is simultaneous to Event 1 Observer C: Event 1 is earlier than Event 2 Why synchronized time? NESL

5. The Myth of Simultaneity: “Event 1 and event 2 at same time” Event 1 Event 2 Observer A: Event 2 is earlier than Event 1 Observer B: Event 2 is simultaneous to Event 1 Observer C: Event 1 is earlier than Event 2 Why synchronized time? • Ordering of events • Coordinated actuation • Data logging • Absolute time of occurrence • Performance measurement • ….. NESL

6. Time & Clocks • How to define time? • Sun • 1s = 1day/(24*60*60) • Atomic • 1s = Time a cesium atom needs for 9192631770 state transitions • Universal Time Coordinated (UTC) • Now a standard for time measure. • Clocks: Measure of time! • Oscillator and counter • Synchronized time -> Synchronized clocks • Errors • Clock skew (offset): Difference between time on two clocks. • Different start times • Clock drift: Count at different rates. • Different frequency of the oscillator. NESL

7. Oscillators Cost Drift rate Rubidium ~10-15 s / day High Atomic Clocks Cesium ~10-12 s / day Desktop ~10-6 s / day ~10-6 s / hr IPAQs Quartz ~10-6 s / s Low Motes NESL

8. Atomic clock CPU power drawn by motes ~ 25 mW Cost will be the deciding factor! NESL

9. Time e11 e12 e13 P1 e11 e12 e13 P1 e21 e22 e23 e24 e25 P2 e21 e22 e23 e24 e25 P2 e32 e33 e34 e32 e33 e34 P3 P3 e31 Logical clock! Time e31 NESL

10. Time synchronization approaches being used in distributed systems NESL

11. GPS overview T1 = t1 + |X-s1|/c Gather four satellite signals and solve the non-linear system of equations. Figure source: http://www.dependability.org/wg10.4/timedepend/03-Schmi.pdf NESL

12. GPS Evaluation • Pros • Accuracy ~ 100ns • Reliable operation • Cons • Cannot work indoors • If something goes bad, delay for correcting it can be as large as 30 minutes. • Neutral • Expensive ~ Cheapest GPS receiver 50US dollars. • Energy hungry! NESL

13. Send at T1 802.11 Synchronization Base station Clients just adopt the timestamp in the beacon packet Very simple, Provides ms accuracy. Neglects packet delay and delay jitters • Same approach being used by NIST to synchronize electronic products such as wall clocks, clock radio, wrist watches etc. worldwide. • WWVB signals are being transmitted from colarado. NESL

14. Recv at T2 Send at T1 T2 = T1 + DELAY + OFFSET Client Peer A B Recv at T4 Send at T3 T4 = T3 + DELAY- OFFSET OFFSET = {(T2-T1)-(T4-T3)}/2 DELAY = {(T2-T1)+(T4-T3)}/2 NTP: Internet Synchronization NESL

15. Peer 1 Filter 1 Intersection and Clustering Algorithms Peer 2 Filiter 2 Combining Algorithm Loop Filter P/F-Lock Loop Peer 3 Filter 3 LCO NTP Messages Timestamps NTP details • Level n synchronizes to level n-1 • Level 1 synchronizes to UTC • Multiple synchronization peers -> redundancy and diversity • Data filters -> Sliding window • Intersection and clustering algorithms -> Discard outliers • Combining algorithm -> Weight samples • Loop filter and local clock oscillator (LCO) -> implement hybrid phase/frequency-lock feedback loop to minimize jitter NESL Figure source: RFC on NTP

16. NTP Evaluation • Pros • Readily available • Industry standard • Achieves secure and stable sync to ms accuracy • Cons • Designed for ms accuracy only! • Not flexible • Neutral • Not energy friendly! NESL

17. Time synchronization in sensor networks. Why it is so crucial? NESL

18. Parametric, Time-domain beamforming M2 1 3 d1 2 d2 Y Search angle P d3 f M3 X • Collect synchronized samples from 3 motes • Hypothesize a search angle f • Calculate delays d1, d2, d3 for each search angle f • Shift and Sum the synchronized waveforms as if the source was at angle f • Choose the angle fm that makes the sum maximum Start of sampling Figure source: Courtesy of Vlassios Tsiatsis NESL

19. Performance dependence on timesync Localization Experiment NESL Figure source: Courtesy of H. Wang, Prof K. Yao, et. al.

20. MAC layer design Radio On Radio Off Sender Radio Off Receiver Guard band due to clock skew; receiver can’t predict exactly when packet will arrive Time NESL

21. Importance of time sync • Beam-forming, localization • Data aggregation & catching • MAC layer design • Adaptive topology management schemes • Absolute time of occurrence • Coordinated robotics • Debugging • Cryptography • User Interface • …… NESL

22. What is so different in sensor networks? Why cannot we use NTP or GPS? NESL

23. Varying demands Beam-forming, localization, distributed DSP: small scope, short lifetime, high precision NESL Figure source: Courtesy of Jeremy Elson

24. Varying demands Target tracking: larger scope, longer lifetime, but lower required precision t=2 t=3 t=1 t=0 NESL Figure source: Courtesy of Jeremy Elson

25. Synchronization Demands Multi-modal • Event ordering • “Did event 1 happened before/after/with event 2?” • Internal Synchronization • “All cameras should be on after 5 minutes” • External Synchronization • “Show me targets detected between 2:00 & 8:00 p.m.” • Hybrid Synchronization • Target tracking: Convert propagation delay of 1.5ms to distance using speed of 345m/s. Energy, Energy, Energy,…. Scalable µs – ms accuracy NESL

26. Comparison Specialized costly hardware Cannot work indoors NESL

27. Synchronizing a pair of sensor nodes NESL

28. RBS: Synchronize receivers Based on CesiumSpray system by Verissimo and Rodrigues Receiver-receiver synchronization NESL Figure source: Courtesy of Jeremy Elson

29. Magic behind RBS NIC Sender Receiver Critical Path Time NIC Sender Receiver 1 Receiver 2 Critical Path Traditional critical path: From the time the sender reads its clock, to when the receiver reads its clock RBS: Only sensitive to the differences in receive time and propagation delay NESL Figure source: Courtesy of Jeremy Elson

30. Relative performance of RBS with NTP Clock Resolution With higher network load RBS degraded slightly (6us to 8us); NTP degraded severely (51us to 1542us) NESL

31. Recv at T2 Send at T3 Send at T1 T2 = T1 + DELAY + OFFSET A B Recv at T4 T4 = T3 + DELAY- OFFSET OFFSET = {(T2-T1)-(T4-T3)}/2 DELAY = {(T2-T1)+(T4-T3)}/2 TPSN: Conventional sender-receiver synchronization Based on NTP Enemy is non-determinism! Asymmetric delays and varying offset NESL

32. Node B clock Node A clock Local node time Ideal clock t1 Real Time t4 Relative offset Variability in clock offset NESL

33. Use low level time stamping Sender uncertainty Propagation uncertainty Receiver uncertainty Magic behind TPSN ALL DELAYS ARE VARIABLE ! sender receiver software software MAC TX propagation RX Bottleneck NESL

34. Error comparison • RBS eliminates impact of sender side jitters • TPSN reduces by 2x the impact of propagation and receiver side jitters, and variations in clock drift. • With low-level time stamping, sender side jitters will be negligible • TPSN will give ~2x better performance than RBS NESL

35. Relative comparison between TPSN and RBS • All results have been obtained via 100 independent experimental runs. • Only consider the magnitude of synchronization error. Average Error (in microseconds) NESL

36. FTSP: Estimate Clock Drift • TPSN and RBS were estimating only offset. • FTSP takes multiple readings to estimate both offset and drift. • Over short time scales linear model works well! Slope = drift Time at Node A } Offset Time at Node B Based on both TPSN and RBS NESL

37. Magic behind FTSP • Multiple timestamps to accurately estimate the byte boundaries. • RBS -> propagation, decoding, interrupt handling. • TPSN -> [interrupt handling, encoding, propagation, decoding, interrupt handling] / 2 • FTSP -> [interrupt handling, encoding, propagation, decoding, byte alignment, interrupt handling] / X , {2 < X < 4} Extra overhead of 4 bytes per message NESL Figure source: Courtesy of Miklos Maroti

38. Mean = -1.8e-16 ≈ 0 Standard deviation = 1.79e-6 ≈ 1.8 microseconds Mean = -6.5e-15 ≈ 0 Standard deviation = 2.81e-6 ≈ 2.8 microseconds Relative Comparison between TPSN and RBS Comparison is not fair: skew v/s [skew, offset estimation] What does superior time stamping buys us? TPSN RBS FTSP ~ 1.4 microseconds Wins of FTSP lies in doing network-wide time sync! NESL

39. On-demand multihop synchronization NESL

40. Post-facto Synchronization “Red pulse 2 secafter blue pulse!” “Here 3 sec after red pulse!” “Here 1 sec after blue pulse!” “Here 1 sec afterred pulse!” “Here 0 sec after blue pulse!” • Timestamp the events with unsynchronized clocks. • Do reconciling at a later stage. NESL Figure source: Courtesy of Jeremy Elson

41. 1 2 5 6 3 4 7 8 9 10 11 Time Routing The physical topology can be easily converted to a logical topology; links represent possible clock conversions 1 2 5 A B 6 3 4 7 C 8 9 D 10 11 Use shortest path search to find a “time route”; Edges can be weighted by error estimates NESL Figure source: Courtesy of Jeremy Elson

42. Evaluation on IPAQs (RBS based) Error (and std dev) over multiple hops, in usec 3.68 +/- 2.57 2.73 +/- 2.42 2.73 +/- 1.91 1.85 +/- 1.28 For Gaussian Error: Variance (n hops) = n.Variance (1 hop) NESL

43. Evaluation on motes (TPSN based) • Theoretically error increases with hop distance. • Randomness of drift and sign of error helps. Microseconds NESL

44. Network-wide time synchronization NESL

45. TPSN – Level Discovery Phase 2 1 K B 0 Levels now all assigned 1 A H 2 1 F C 1 2 E L 3 2 2 3 I D G J NESL

46. TPSN – Synchronization phase • Node synchronize to a node belonging to one upper level • Use pair wise synchronization. • Needs symmetric links! NESL

47. FTSP – Synchronization phase • Use simple flooding • Receiver gets the tuple (TA,TB) • Uses this to update the estimate on the clock drift. NESL

48. What will happen in fixed-area networks? Evaluation using TPSN • Overhead • Energy expenditure per node remains a constant • Error • Just increases by 10% even after doubling the total number of nodes. Time taken Number of nodes Scalable! NESL

49. What will happen in expanding networks? Evaluation using TPSN • Error increases as distance from root node increases • Optimal results will be obtained if we instead establish a minimum spanning tree structure • Energy v/s Accuracy tradeoff Y coordinate X coordinate Not Scalable! Position of root node NESL

50. 1st leader 2nd leader FTSP experimental evaluation topology: avg. error: 1.6 μs max. error: 6.1 μs per hop 50% turned off 1st leader turned off random nodes turned off/on all turned back on all turned on NESL