A distributed video streaming platform supporting multiple description coding

1. A distributed video streaming platform supporting multiple description coding Marco Grangetto Universita’ degli Studi di TORINO Dipartimento di Informatica Gabriella Olmo Politecnico di TORINO Dipartimento di Elettronica Enrico Baccaglini ST Microelectronics Istituto Mario Boella - Torino Mario Mirabelli, Enrico Vinciarelli Telecom Italia Research and Trends Dept. - Torino

2. Outline • TiLAB – PoliTO project: motivation of the work • Proposed approach • Overview about MDC • Integration of MDC in VLC • Experimental results • Conclusions

3. Motivation of the work • Scenario: download or stream multimedia data using terminals equipped with different capabilities • in terms of power consumption, memory, computational resources and visual resolution • and also because the contents may be accessed using broadband networks such as DSLs, optical cable or WiMax, but also full mobility GPRS/UMTS or beyond 3G networks. • Scalability is a key feature. • Data are transmitted on a non prioritized network • Retransmission is unfeasible due to delay constraints. • It is important that all the received packets can be exploited at the application level.

4. Proposed approach • A full video streaming system making use of H.264/AVC and the MDC algorithm [1] • This system (encoding + distribution + decoding), has been integrated in the DynamicTV prototype [2], developed by Telecom Italia Lab for distributed, multi server video streaming based on RTP/UDP. • Performance evaluation for both pre-encoded video distribution and live streaming. • The major contributions of the present work are the prototyping and performance evaluation of MDC in a full streaming application. • The scheme adopts a datagram best-effort transmission, characterized by low complexity and low protocol overhead. [1] T. Tillo, M. Grangetto, and G. Olmo, “Redundant slice optimal allocation for H.264 multiple description coding,” IEEE Trans. on Circ. and Syst. for Video Tech., vol. 18, no. 1, pp. 59 – 70, Jan 2008. [2] R. Simeoni, M. Geymonat, E. Guercio, M. Perrero, A. Rapp, F. Tesauri, and R. Montanari, “Where have you ended up today? Dynamic TV and the intertainment paradigm,” in Proc. of EuroITV, July 2008.

5. Overview: MDC • In multiple description coding (MDC) non hierarchical representations of the source are generated, yielding mutually refinable information. • The quality at the receiver side only depends on the number of successfully received packets, and not on the particular subset or arrival order. • These features reduce the compression efficiency, due to the need of inserting redundancy among the descriptions. • This redundancy is generally measured in terms of the extra rate required by the MDC scheme, compared to a single description reference system achieving the same performance.

6. DiVA (Distributed Video Applications) • DiVA libraries is a MD coding system which generates 2 balanced description • Each description is a H.264/AVC compatible stream • Descriptions are transmitted as a combination of primary slices e redundant slices

7. MDC coding with VLC ./vlc <live input> --sout '#transcode{venc=mdc{rho=0.2,mdcmode=md0,mblim=80,sizelim=500,keyint=50},vb=512} :split{dst=rtp{dst=<ip_addr1>,port-video=9348,port-audio=9450,ssrc-video=aa, ssrc-audio=bb,sn-video=1,sn-audio=1,sdp=file://<PATH/>mdc1.sdp}, dst=rtp{dst=<ip_addr2>,port-video=9352,port-audio=9354,ssrc-video=cc, ssrc-audio=dd,sn-video=1,sn-audio=1,sdp=file://<PATH/>mdc2.sdp}}' ./vlc input.mp4 --sout '#transcode{venc=mdc{rho=0.01,mdcmode=md0,mblim=150,sizelim=800,keyint=25},vb=512}:split{dst=std{access=file,mux=mp4,dst=1.mp4},dst=std{access=file,mux=mp4,dst=2.mp4}}‘ 9348 (Video 1) Modulo SPLIT 9450 (Audio 1) Modulo MDC 9352 (Video 2) 9354 (Audio 2)

8. mdc plug-in for vlc transcode{venc=mdc{rho=0.2,mdcmode=md0,mblim=80,sizelim=500,keyint=50},vb=512}..} • The plug-in acts as an interface between DiVA, x264 and VLC • The plug-in can code in two different ways: • a single H264 standard-compliant stream (mdcmode=avc) or • two H264 descriptions (mdcmode=md0) • mblim : max number of 16x16 MBs in a single NAL. • sizelim max size (in bytes) of a H.264 NAL. • keyint number of frames between 2 Intra frames. • rho in [0,0.5] refers to the amount of rate to be dedicated to redundancy. • Example: coding rate @ 512 kbps (vb=512), redundancy rho=0.2 leads to 0.2 x 512 kbps = 102 kbps of the total output rate which will be used as redundancy. • Rate(primary) + Rate(redundant) = Rate(primary) + [rho * Rate(total) ] = Rate(total) • Example (as above): primary rate @ 512-102=410 kbps with each description coded @ 512/2=256 kbps.

9. mdc: encoder settings (example) #transcode{venc=mdc{rho=0.02,mdcmode=md0,mblim=150,sizelim=800,keyint=250},vb=800} ---Analyser--- encoder.c : analyse.intra: 1 encoder.c : analyse.inter: 273 encoder.c : analyse.b_transform_8x8: 0 encoder.c : analyse.b_weighted_bipred: 0 encoder.c : analyse.i_direct_mv_pred: 1 encoder.c : analyse.i_direct_8x8_inference: 1 encoder.c : analyse.i_chroma_qp_offset: 0 encoder.c : analyse.i_me_method: 1 encoder.c : analyse.i_me_range: 16 encoder.c : analyse.i_mv_range: 512 encoder.c : analyse.i_mv_range_thread: -1 encoder.c : analyse.i_subpel_refine: 5 encoder.c : analyse.b_bidir_me: 0 encoder.c : analyse.b_chroma_me: 1 encoder.c : analyse.b_bframe_rdo: 0 encoder.c : analyse.b_mixed_references: 0 encoder.c : analyse.i_trellis: 0 encoder.c : analyse.b_fast_pskip: 1 encoder.c : analyse.b_dct_decimate: 1 encoder.c : analyse.i_noise_reduction: 0 encoder.c : analyse.b_psnr: 0 encoder.c : analyse.b_ssim: 0 ---Rate control--- encoder.c : rc.i_rc_method: 2 VLC media player 0.8.6f Janus mdc: Redundancy red=0.02 diva_h264_common: NAL size limit: 800 bytes, 150 MB 16x16 per slice, 5 slices per frame, 24000/1000 fps (24.000000), w/h=640/268, CABAC: 1 diva_h264_common: Rate control is active. Rate= 800 kbps, Primary rate=785 kbps, Red rate=15 kbps TESTING PARAMETERS... ---General--- encoder.c : i_threads: 1 encoder.c : b_deterministic: 1 encoder.c : i_csp: 1 encoder.c : i_level_idc: 51 encoder.c : i_frame_total: 0 ---Bitstream parameters--- encoder.c : i_frame_reference: 1 encoder.c : i_keyint_max: 250 encoder.c : i_keyint_min: 25 encoder.c : b_deblocking_filter: 1 encoder.c : b_cabac: 1

10. split module :split{dst=std{access=file,mux=mp4,dst=1.mp4},dst=std{access=file,mux=mp4,dst=2.mp4} :split{dst=rtp{dst=130.192.9.182,sdp=file://mdc0.sdp, port-video=9348, ssrc-video=aa, sn-video=1, port-audio=9350, ssrc-audio=bb, sn-audio=1}, dst=rtp{dst=130.192.86.141,sdp=file://mdc1.sdp, port-video=9352, ssrc-video=cc, sn-video=1, port-audio=9354, ssrc-audio=dd, sn-audio=1}} • This module split the MDC stream into two files or RTP/SDP sessions • It works as the VLC duplicate module with dst option.

11. split module: live :split{dst=rtp{dst=130.192.9.182,sdp=file://mdc0.sdp, port-video=9348, ssrc-video=aa, sn-video=1, port-audio=9350, ssrc-audio=bb, sn-audio=1}, dst=rtp{dst=130.192.86.141,sdp=file://mdc1.sdp, port-video=9352, ssrc-video=cc, sn-video=1, port-audio=9354, ssrc-audio=dd, sn-audio=1}} • 2 SDP file • v=0 • o=- 1247216830831359 2 IN IP4 127.0.0.1 (in mdc1.sdp: o=- 1247216830833278 2 IN IP4) • s=NONE • t=0 0 • a=tool:vlc 0.8.6f • c=IN IP4 130.192.9.182 (in mdc1.sdp: c=IN IP4 130.192.86.141) • m=video 9348 RTP/AVP 96 (in mdc1.sdp: m=video 9352 RTP/AVP 96) • b=AS:800 • a=rtpmap:96 H264/90000 • a=fmtp:96 packetization-mode=1;profile-level-id=4d4033;sprop-parameter-sets=Z01AM5p0BQEf3CAAAAMAIAAABhHjBlQ=,aO48gA==; • m=audio 9450 RTP/AVP 97 (in mdc1.sdp: m=audio 9354 RTP/AVP 97) • a=rtpmap:97 mpeg4-generic/44100 • a=fmtp:97 streamtype=5; profile-level-id=15; mode=AAC-hbr; config=1210; SizeLength=13;IndexLength=3; IndexDeltaLength=3; Profile=1;

12. Decoder/Mixer: VLC description 1 video H264 Main@5.1, 86 s, 213 kbps, 640x268 @ 24 fps description 2 video H264 Main@5.1, 86 s, 213 kbps, 640x268 @ 24 fps Total output rate: 426 kbps (received by decoder) Central quality: 380 kbps (46 kbps allocated for redundancy)

13. Experimental results (setup) • The measurement scenarios were both on the local area networks (100 Mbps LAN) and across the public broadband network of Telecom Italia. • The client can receive the video from three different video servers. • The client can access the public network over both a fixed broadband access (12 Mbps ADSL) and a mobile broadband access (7 Mbps HSDPA). • The Politecnico server is connected through 14 hops while the Tilab servers can be reached in 4 hops

14. Experimental results (metrics and sequences) • The measurement techniques used are: • Network protocol layer tracing. In order to monitor the traffic of the MDC applications, a network layer snooping tool has been adopted to analyze the IP level traffic received at client side. Post-processing of the captured network data can provide information about loss rates at the client side. • Application instrumentation. In commercial players, objective evaluation of the application performance is seldom possible. However, in our study, we were able to compare the received video with the original one. Once all frames have been saved by the mixer, a post-processing algorithm over the captured frames provides information about the video quality perceived by the final user by means of the PSNR metric. • Test video sequence: • Here, we report the results for the “Made for each other” sequence, 640×360 pixels, 24fps, which encompasses about 3300 frames. • The original video sequence has a rate of approximatively 1300 kbps and is coded using H.264/AVCMain Profile@L3.0 with 2 reference frames. • we re-encoded the sequence to have a total output rate of 1100 kbps. For SDC this is all devoted to video data, while for MDC each description has a rate of 550 kbps including both video data and redundancy. As for this latter parameter, we set redundancy to 0.2 and 0.05.

15. Experimental results – packet loss and PSNR (1/2) • The transmission over LAN is lossless (SDC transmission allows to achieve the best performance) Insertion of unused redundancy in the video data is detrimental when no losses occur, but these results show that the performance degradation is limited (about 0.3 dB). • MDC proves its efficiency in lossy scenarios. When the application is required to operate across the public internet, there is a significant differences between the number of hops separating the client from server in Tilab (which is small) and the client from server in Polito (which is high). • The gain over traditional SDC is significant for the HSDPA scenario where the mean packet loss rate is about 8%. • Comparing the redundancy of MDC in the ADSL and HSDPA we highlight that a higher r allows to better recover from packet losses. In ADSL, the gain between the two redundancy levels is about 2 dB and increases to 5 dB for HSDPA.

16. Experimental results (2/2) • we report the PSNR of each video frame in the range [2700-2850] for the ADSL scenario in which the packet loss rate is approximately 4%. • MDC can rely on two video streams to recover the frames so that, in case of loss of one of the two representations, the other may be available and this allows the decoder to recover the video (e.g. around frame 2700). • MDC can reduce the video quality fluctuations, which lead to a very annoying display visualization. • The impairment between different values of r can be seen around frame 2800 where the two MDC transmissions undergo the same losses and the performance with r = 0.05 are slightly higher than with r = 0.2.

17. Conclusions • We tested a MDC scheme based on the redundant slice feature of H.264/AVC over different network scenarios provided by Telecom Italia Lab. • Results show that MDC can significantly outperform traditional coding when packet losses occur, with a gain of up to 6 dB when the network is particularly unreliable (HSDPA). • In case of lossless transmission, the MDC redundancy reduces the mean quality by approximately only 0.3 dB. • Future work will include analysis of live content streaming by means of SDP protocol.

18. More information… or let’s meet in ST Torino LAB @ Istituto Superiore Mario Boella!