Ertan DENIZ MilSOFT A.S, Teknokent ODTU,Ankara/Turkey edeniz @milsoft.tr

1. Ertan DENIZ MilSOFT A.S, Teknokent ODTU,Ankara/Turkeyedeniz@milsoft.com.tr Huseyin Kutluca, hkutluca@milsoft.com.tr Video Over DDS DDS Information Day Brussels, Belgium-June-26,2007 UNCLASSIFIED

2. Outline • Introduction • Image Exploitation System for UAVs • Why Use DDS for Video Streaming? • MilSOFT DDS • DDS Topics • QoS choices • Future Work • Conclusion UNCLASSIFIED

3. Introduction • DDS middleware • Location transparency with pub/sub mechanism • Redundancy and fault tolerance mechanisms • Multiple readers/writers • Transient and persistent services • Asynchronous, time-independent data distribution • Rich set of QoS, suitable for real-time systems • Wide range of Application Areas • C4I • Industrial automation • Distributed control and simulation • Telecom equipment control • Sensor networks • Use Case: Image Exploitation System for UAVs UNCLASSIFIED

4. Pre-processor Data Receiver Exploitation 2 Screening Exploitation 1 DDS on UAV Image Exploitation System DB Server Raw Data Recording High Capacity Storage Unit Pre-processed & Georecorded Data Sensor Data (EO/IR, SAR, GMTI) Raw Data DDS based Data Streaming Telemetry Artifacts All Data in DB UNCLASSIFIED

5. DDS on UAV Image Exploitation System • Data • Events • User information • Resource Management • Radar data streaming • SAR streaming • GMTI streaming • Telemetry streaming • Video streaming • Live data streaming • Recorded data streaming • Data Publisher: DataReceiver • Data Subscribers: Consoles and Pre-Processor • Content of data MPEG-2 TS UNCLASSIFIED

6. MPEG-2 TS (Transport Stream) • Elementary Stream • Video, audio, telemetry data, etc... • From each elementary stream Packetized Elementary Stream (PES) is formed • And each PES is broken into fixed-size Transport Packets • TS packets are 188 bytes of which 4 bytes are for the header and 184 bytes are for the payload • Transport Stream can multiplex multiple elementary streams • The TS packet header contains a Packet Identifier (PID) to identify different elementary streams • 8Mbit data rate for UAV Image Exploitation System UNCLASSIFIED

7. Why use DDS for video streaming? • Publish-Subscribe simplifies the development process • The clients do not need to know where the server is, and the server does not need to know the clients • DDS specification implies mutlicast communication and the interoperability wire protocol RTPS supports and recommends multicasting • Dramatically decreased bandwith usage with DDS implementations which support multicast • Quality Of Service attributes like reliability, durability, transport priority directly can be used for video streaming channels • Generally DDS implementations have very low latencies in data transmission compared to other middlewares, which makes it a good choice UNCLASSIFIED

8. Why MilSOFT DDS • MilSOFT DDS is compliant with current DDS specification, version 1.2 • Provides Minimum Profile QoS attributes, which are neccessary for video streaming • Completely decentralized design • No dynamic memory allocation after initialization • Zero-copy access to data • Implements multicast communications UNCLASSIFIED

9. MilSOFT DDS Middleware More information on http://dds.milsoft.com.tr UNCLASSIFIED

10. Video Over DDS Architecture UNCLASSIFIED

11. DDS Types and Topics Used For Video Streaming • So how do we encapsulate MPEG2-TS packets into DDS types? • There are two DDS types and two corresponding Topics. • VideoHeader  VideoHeaderTopic • VideoPacket  VideoPacketType • VideoHeader • Contains the header of the video, which is needed to initialize the decoder. (FPS, height, width, etc... ) • VideoPacket • Contains MPEG2-TS packets UNCLASSIFIED

12. VideoHeader Topic struct VideoHeader { long headerType; // key sequence<char, HEADER_MAX_DATA> headerData; }; • Published once for each Transport Stream • Contains the necessary information • initialize the video decoder • information about the other multiplexed elementary streams • Key to distinguish the video header from other header data • Late joiners also get this topic UNCLASSIFIED

13. QOS for VideoHeader Topic • VideoHeaderTopic is necessary to initialize decoding of the video • Header information is critical for clients • Use Reliability QoS (RELIABLE) • Both publishers and subscribers are Reliable • Late joining viewers need this information • Use Durability QoS (TRANSIENT_LOCAL) • There are different instances of the header according to it’s type. We need only the last update of each instance! • History QoS is default, kind = KEEP_LAST, depth = 1 UNCLASSIFIED

14. VideoPacket Topic struct VideoPacketType { long frameNumber; sequence<char, PACKET_MAX_DATA_SIZE> packetData; }; • MPEG2-TS packets are too small in size (188 bytes). • VideoPacket type sends multiple TS packets of a video frame at each VideoPacket sample • Decreases the overhead and increases the bandwidth utilization • A frame can span multiple VideoPacket samples (fragmented) • Each sample contains TS packets of only one frame UNCLASSIFIED

15. QOS for VideoPacket Topic • For VideoPacketTopic we may need both Reliable and Best Effort delivery, according to client needs • Use Offered >= Requested semantics • Server publishes with a Reliable writer ( Offer Reliable communication ) • Clients subscribe with either Reliable or Best Effort readers • Data is not needed by late joiners • Durability is Volatile • VideoPacket Type has no key, but we can use History QoS for buffering • History QoS, kind = KEEP_LAST, depth value can be used as a buffer (set it to more than 1) UNCLASSIFIED

16. Other Concerns And Solutions • We don’t want out of sequence frames • DDS already satisfies this, no out of order packets! • We don’t want frames which are too late • Use Lifespan QoS • Decoding takes time • Either read data from a DataReaderListener and immediately put into the queue of another thread • Use WaitSets UNCLASSIFIED

17. Future Work • Implement advanced services like rewind, replay, fast-forward etc... • Network reservations • Use RSVP (IntServ) to reserve bandwith • Analyze different methods to give DDS video streams a priority • Use Transport_Priority QoS (not completely standard) • More standardization is needed in this area • Standardize Flow Control mechanisms in DDS for better utilization of network bandwith UNCLASSIFIED

18. Conclusion • Video over DDS is possible! • DDS provides a good set of QoS attributes for video streaming • Performance requirements • Image Exploitation system for UAVs uses 8 Mbit MPEG-2 TS • 20 Mbit video streaming is possible with DDS UNCLASSIFIED

19. Thank You! UNCLASSIFIED