Applying SDL for Real-Time Systems Development: A Lip-Synchronisation Case Study

1. Applying SDL for Real-Time Systems Development: A Lip-Synchronisation Case Study Dr Richard Sinnott EPK Ericsson

2. Overview of Presentation • Intention of presentation to address issues in SDL usage for general modelling vs validation • not all (small sized) models can be validated! • Outline of Lip-Synchronisation system • outline of algorithm • outline of modelling approaches for Producer/Consumer sides • Conclusions on SDL usage

3. Lip-Synchronisation Model • Two producers wanting to send audio flow and video flow respectively to consumer • strict timing requirements for individual flows • within some jitter bounds • strict timing requirements between flows • avoid flows drifting apart • rates of production/consumption can vary • allow faster/slower rates of production/consumption

4. Basic Timestamping Modelling Approach For Producer Side • Producers use timestamping of frames with timestamps based on expiry of local timers • local timers set at controllable rates (durations) • rate of production dependent on rate variable • sendFaster/sendSlower -> lower/higher value for rate results in higher/lower productivity • producer can also ask consumer to consume faster/ slower, e.g. if producer has too many/not enough buffered frames to send

5. Producer Timestamping Approach ready startFlow tnow:= NOW length(fs)>0 False = stop flow True set(tnow+r,t1) • Change rate (r) = change sending freq. • Frames to send(fs) value below/above some limit = consume slower/faster resp. wait tnow + r = NOW t1 FrameSend( (. first(fs)!d,tnow+r .) ) tnow:=tnow+r fs:= substring(fs,2,length(fs)-1)

6. Basic Timestamp Approach For Consumer Side • Consumer receives two streams of arriving timestamped frames • Ensure both streams satisfy individual timing constraints • ensure timestamps of arriving frames within consumption rate +/- jitter for individual flows • Ensure both streams satisfy joint time constraints • ensuring timestamps arriving frames within bounds compared to timestamped frames of other flows

7. wait ConsumerTimestampingApproach FrameSend(f) f!d similar for video case ‘audio’ Check timestamp of this frame vs time stamp of last video frame (or expected next video frame if buffered audio = buffered video) If False then increment error count: if error count > max then resetClocks call checkAudioSynchronisation(f) True Check early f!ts < act + ar - aj False True ae:= ae +1 f!ts > act + ar + aj False • checks if audio/video frame (f!d) • checks value of frame timestamp (f!ts) with last video/audio timestamp, if ok • compares with own local clock (act) and rate consumption (ar) +/-jitter (aj) • if ok, add to buffered frames, else increment no. early/late (ae/al) frames, add copy of data to buffered frames • if ae/al exceed maximum value, set producer clock with appropriate offset True Check late al := al +1 al > maxLateAudio OR ae > maxEarlyAudio True False lab Frame time OK df!d := last(afs)!d; setClock(aId, ae:= 0, al:=0, df!l := f!l; act + ar - f!ts) afs := afs // df!ts := f!ts; MkString(f) afs := afs // ae := 0, al:= 0 MkString(df) lab act := act + ar -

8. ready startFlow ‘’ #addclock(ac1) wait #postpone(ac1>=9, ac1<=11) true length(fs)>0 False = stop flow True FrameSend( (. first(fs)!d,now .) ) NOW=? fs:= substring(fs,2,length(fs)-1) resetclock(ac1) wait Problems Validating Using Timestamp Approach • Consider same approach using INTERVAL time extensions • Problem of accessing/reading time • now does not explicitly change in FrameSend so consumer cannot check if early/late • actually value 0 sent but when try to access this causes “value out of range condition” • hence consumer cannot check timing information, e.g. frame early, late etc • if try to send value of ac1 would result in non-validatable system • due to current techniques for clock constraint representation (polyhedra) during validation

9. Problems validating …continued • Validation of following systems causes deadlock • does not result in signal X being sent • if interactively simulate system, then X sent • e.g. NOW takes value of last timer that has fired set(now+9,t1) wait t1 NOW>=8 - X -

10. Conclusions • Role played by specification important • interactive simulation vs • validation (via state space exploration) • Often restrictions imposed which are not immediately apparent when specifying • As well as language extensions and tool development Interval should (is!) • identify these restrictions • give guidelines how best to specify/verify such systems • don’t use time stamps accessing NOW • don’t use continuous signals based on NOW • … etc