1 / 43

Synchronization

Synchronization. TTM4142, 2007 Harald Øverby/Leif Arne Rønningen. Outline. What is synchronization Relationships Logical Data Units (LDU) Reference model for synchronization Specification. Definition of synchronization. Synchronization

emery
Download Presentation

Synchronization

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Synchronization TTM4142, 2007 Harald Øverby/Leif Arne Rønningen

  2. Outline • What is synchronization • Relationships • Logical Data Units (LDU) • Reference model for synchronization • Specification

  3. Definition of synchronization • Synchronization The act or result of synchronizing; the state of being synchronous • Synchronize To happen at the same time; to represent or arrange (events) to indicate coincidence or coexistence; to make synchronous in operation; to make (motion or picture sound) exactly simultaneous with the action Source: Merriam-Webster dictionary

  4. Definition of synchronization • Synchronization creates relationships between independent objects • Time dependent / time independent objects • Three types of relationships • Content • Spatial • Temporal

  5. Relationships

  6. Content relationships • Connection between media objects and data • Example • Between data in a table and a picture • Update in data triggers update in other views • Consistency

  7. Spatial relationships • Layout relationships • Define space of a media object • Objects arranged in a two dimensional area • Output device • Content assigned to frames • Positioning: fixed, relative • Example • Powerpoint slide • Web page

  8. Temporal relationships • Time relationships between objects • Important for time dependent media objects • Examples • Timing relationships between audio object and video object • Further on: focus on temporal relationships

  9. Intra object synchronization • Time relation between various presentation units of one time-dependent media object • Example: bouncing ball

  10. Inter object synchronization • Time relation between two or more media objects • Examples: Audio, Pictures (P), Video, Animation

  11. Logical data unit (LDU) • LDU: Information unit of a time-dependent media object • Granularity of LDU • Video: Frame, object, pixel

  12. Logical data unit (LDU) • Closed LDU • Predictable duration • Ex: Stored media objects • Open LDU • Duration not known before the presentation starts • Ex: Input from live source

  13. Logical data unit (LDU) • Physical units as LDU or blocks of samples as LDU

  14. Logical data unit (LDU) • User-defined vs capturing defined

  15. Logical data unit (LDU)

  16. Synchronization in multimedia systems • Television systems • Sound and video are multiplexed • Delay affects both sound and video • Synchronization always maintained • Focus on time-independent media • LDU to be put in relationship are independent media objects • Quality of Service • Hard guarantees • Soft guarantees

  17. Synchronization examples • Lip synchronization • Coupling between audio and video

  18. Lip synchronization experiment • Skew between video and audio • +/- 40 ms intervals • Positive skew: audio ahead of video • Short clips

  19. Example questionnaire

  20. Experiments - results

  21. Experiments - results

  22. Results summary • Asymmetry • Positive skew more disturbing • Difficult to detect errors when object far away • Statistical errors • Long movie • +/- 80 ms : no distraction • -240 ms, +160 ms : disturbing

  23. QoS: Skew values

  24. Reference Models for Multimedia Synchronization • Support to understand requirements • Many proposals • Focus on the following four-layer model • Specification Layer • Object Layer - presentation object synch • Stream Layer - inter-stream synch of media streams • Media Layer - intra-stream synch of continious media

  25. Reference model for synchronization

  26. Media Layer • Application handles only one continuous media stream as an LDU sequence • Device-independent interface, file and device access • Read (device-handle, LDU) • Write (device-handle, LDU) • etc • Application runs one process for each stream • Guarantees single LDU processing

  27. Stream Layer • Handles continuous media streams and media stream groups • Guarantees intra-stream synch • Group presents streams in parallel • Gurarantees inter-stream synch of streams in a group • Handles resource reservation and scheduling of LDU processing • Typical operations • start (stream), stop (stream) • create_group (list_of_streams) • start (group), stop (group) • setcuepoint (stream/group, at, event)

  28. Object Layer1 • Deals with all kinds of media • Hides the difference between discrete and continuous media • Accepts a synchronization specification as input • Media-oriented QoS (such as allowed skew, allowed jitter) • Plans and coordinates presentation execution • Initiates presentation of time-dependent and time-independent media objects 1In DMP the term could have been Scene Layer

  29. Specification Layer • Does not provide a specific interface, is open • Tools for generating synch specification • Edit • Format • Maps user interface QoS specifications to the Object Layer

  30. Synchronization Specification, main categories • Interval-based • Temporal relationships between the presentation of media objects • Axis-based • Presentation objects placed in common axes • Control-flow-based • Presentation flow synchronized at certain points • Event-based • Events trigger presentation actions

  31. Synchronization in a distributed environment • More complex than a local environment • Distributed storage of media • Distributed storage of synchronization information • Communications delay and delay jitter • Multi-party communication patterns

  32. Transport of synchronization specification • Additional synchronization channel • Multiplexed channel

  33. Location of synchronization operations

  34. Multiple communications

  35. • Multi-step Synchronization • Synchronization • during object acquirement • • of retrieval • • during delivery of the LDUs to the network • • during the transport • • at the sink • • within the output device

  36. Synchronization Specification Complex specification comprise: • multiple object synchronization • user interactions Requirements for a specification method: • Support of object consistency and for maintenance of specifications • Media objects should be kept as logical units in the specification • All types of synchronization relations should be easily describable • Support for the integration of time-dependent and time-independent media objects • Quality of service requirements must be describable • Support of hierarchical levels of a synchronization

  37. Interval-based Specification

  38. Synchronization Based on a Global Timer Media objects

  39. Synchronization Based on Virtual Axes

  40. Synchronization Based on Reference Points

  41. Petri Net Specification

  42. Event-based Synchronization

More Related