Global Time in Distributed Real-Time Systems

1. Global Time in Distributed Real-Time Systems Dr. Konstantinos Tatas

2. OUTLINE • Revision of real-time system • Distributed real-time system requirements • Global time • Clock synchronization

3. What is real-time? Is there any other kind? • A real-time computer system is a computer system where the correctness of the system behavior depends not only on the logical results of the computations, but also on the physical time when these results are produced. • By system behavior we mean the sequence of outputs in time of a system.

4. Real-time means reactive • A real-time computer system must react to stimuli from its environment • The instant when a result must be produced is called a deadline. • If a result has utility even after the deadline has passed, the deadline is classified as soft, otherwise it is firm. • If severe consequences could result if a firm deadline is missed, the deadline is called hard. • Example: Consider a traffic signal at a road before a railway crossing. If the traffic signal does not change to red before the train arrives, an accident could result.

5. Distributed RT system model • From the POV of an outside observer, a real-time (RT) system can be decomposed into three communicating subsystems: • a controlled object (the physical subsystem, the behavior of which is governed by the laws of physics), • a “distributed” computer subsystem (the cyber system, the behavior of which is governed by the programs that are executed on digital computers) • a human user or operator • The distributed computer system consists of computational nodes that interact by the exchange of messages. • A computational node can host one or more computational components.

6. Design Challenges in Distributed Systems • Theoretically, a distributed system features the same design challenges as a centralized embedded system in terms of performance, power consumption, battery life, etc. • However, an additional challenge exists: synchronization between nodes

7. Definition of time • Newtonian physics time model is adequate for most temporal phenomena and is much simpler than relativistic time • Time is modeled as an infinite set [T] with the following properties: • {T} is an ordered set, that is, if p and q are any two instants, then either p is simultaneous with q, or p precedes q, or q precedes p, where these relations are mutually exclusive. We call the order of instants on the timeline the temporal order. • {T} is a dense set. This means that there is at least one q between p and r iff p is not the same instant as r, where p, q, and r are instants.

8. Events in the time model • A section of the time line between two different instants is called a duration. • An event takes place at an instant of time and does not have a duration. • If two events occur at the same instant, then the two events are said to occur simultaneously. • Instants are totally ordered; however, events are only partially ordered, • Events can be totally ordered if another criterion is introduced to order events that occur simultaneously

9. Causal order • In many real time systems determining cause and effect relations between events is of interest, especially determining the primary event • Temporal order is necessary but not sufficient to establish causal order

10. Global time • An important yet challenging task is maintaining a consistent global time in a distributed real-time system • There is no global clock, only local clocks • Local clocks drift arbitrarily • No local clock is always correct • A global time is an abstract notion that is approximated by properly selected microticks from the synchronized local physical clocks of an ensemble.

11. Global time • Assume a set of nodes, each one with its own local physical clock that ticks with granularity . Assume that all of the clocks are internally synchronized with a precision Π, i.e., for any two clocks j, k, and all microticks i • It is then possible to select a subset of the microticks of each local clock k for the generation of the local implementation of a global notion of time. • We call such a selected local microtick i a macrotick (or a tick) of the global time. • For example, every tenth microtick of a local clock k may be interpreted as the global tick, the macrotick, of this clock (see Fig. 3.2). • If it does not matter at which clock k the (macro) tick occurs, we denote the tick ti without a superscript. • A global time is thus an abstract notion that is approximated by properly selected microticks from the synchronized local physical clocks of an ensemble.

12. Reasonable global time • The global time t is called reasonable, if all local implementations of the global time satisfy the condition • g>Π • Then for a single event e, that is observed by any two different clocks of the ensemble, their global time-stamps can differ by at most one tick. • This is the best we can achieve.

13. Temporal order • When to events differ by less than two ticks temporal order cannot be maintained

14. Errors in duration measurement

15. Internal clock synchronization • The purpose of internal clock synchronization is to ensure that the real-time clocks of each correct node are within precision Π, independently of their drift rates. • The global time ticks of each node must be periodically resynchronized within the ensemble of nodes to establish a global time base with specified precision. • The period of resynchronization is called resynchronization interval. After that the clocks are left to drift again until they are resynchronized.

16. Internal clock synchronization • The synchronization algorithm must bring the clocks so close together that the amount of divergence during the next free-running resynchronization interval will not cause a clock to leave the precision interval. • Φ+Γ≤Π • Where Φis the convergence function and Γis the drift offset • Φ=2ρRint • Where Rint is the length of the resynchronization interval and ρ is the maximum specified drift rate

17. Malicious clock • clock synchronization can only be guaranteed in the presence of Byzantine errors if the total number of clocks N (3k + 1), where k is the number of Byzantine faulty clocks.

18. Central Master Synchronization • the central master, periodically sends the value of its time counter in synchronization messages to all other nodes • the slave records the time-stamp of message arrival. • The difference between the master’s time, contained in the synchronization message, and the recorded slave’s time-stamp of message arrival, corrected by the known latency of the message transport, is a measure of the deviation of the clock of the master from the clock of the slave. • The slave then corrects its clock by this deviation to bring it into agreement with the master’s clock. • Used at system startup • Not fault-tolerant

19. External clock synchronization • External synchronization links the global time of a cluster to an external standard of time. • For this purpose it is necessary to access a timeserver, i.e., an external time source that periodically broadcasts the current reference time in the form of a time message. • GPS (Global Positioning System). • The accuracy of a GPS receiver is better than 100 ns and it has an authoritative long-term stability – in some sense, GPS is the worldwide measurement standard for measuring the progression of time. Alternatively, the external time source can • temperature compensated crystal oscillators (TCXO) • Typical drift rate of better than 1 ppm, causing a drift offset of better 1 μs/s • atomic clocks • Rubidum clock typical drift rate in the order of 10^-12 causing a drift offset of about 1 μs in 10 days.

20. Example 1 • Given a clock synchronization system that achieves a precision of 90 μs, what is a reasonable granularity for the global time? • What are the limits for the observed values for a time interval of 1.1 ms?

21. Example 2 • Given a • latency jitter of 20 μs, • a clock drift rate of 10^-5 s/s, and • a resynchronization period of 1 s • what precision can be achieved by the central master algorithm?

22. Example 3 • A distributed system uses GPS for clock synchronization: • What is the reasonable granularity for the global time? • What are the limits for the observed values for a time interval of 200 ms?

23. References • H. Kopetz, “Real time systems: Design principles for distributed systems” Springer • Kopetz, H. & W. Ochsenreiter. (1987). Clock Synchronization in Distributed Real-Time Systems. IEEE Trans. Computers. Vol. 36(8). (pp. 933-940). • Kopetz, H. (1992). Sparse Time versus Dense Time in Distributed Real-Time Systems. Proc. 14th Int. Conf. on Distributed Computing Systems. IEEE Press. (pp. 460-467).