Introduction

1. Introduction Topics in Multimedia Computing and Networking Ketan Mayer-Patel CS 294-9 :: Fall 2003

2. The medium is the message. • Marshall McLuhan (1967) • Themes of the course: • In multimedia communication, how the message arrives is often as important as what the message is. • The two are entangled. • How the message arrives: Networking • What the message is: Multimedia Encoding CS 294-9 :: Fall 2003

3. Characterizing media data • Continuous media is a bit more accurate. • Periodic • Robust • Often large • Correlated • Endless? • Video, audio, stock ticker?, depth? • Compare with web pages. • Web pages are definitely multimedia, but don’t fit the characteristics of continuous media. CS 294-9 :: Fall 2003

4. Digital Betacam MPEG-1 D-1 (CCIR 601) Betacam MPEG-2 DVCPro (D-7) D-2 VHS MPEG-4 D-5 S-Video DV Video-8 M-JPEG DVCAM Hi-8 H.261 Digital-8 Betamax H.263+ D-9 Real Sorenson DVCPro50 Cinepak Indeo Video for Windows Video Taxonomy Compressed Uncompressed Tape Streaming Digital Analog CS 294-9 :: Fall 2003

5. m-law A-law ADPCM LPC-10E Linear PCM Dolby AC-3 CELP MPEG-1 L3 GSM MPEG-2 AAC DTS SDDS THX MiniDisc (ATRAC) Audio Taxonomy Compressed Uncompressed Speech-based General CS 294-9 :: Fall 2003

6. Network Basics Throughput = Amount of data arriving per unit time. Delay Source Dest Loss Interpacket Arrival Time Jitter = Variance of interpacket arrival times CS 294-9 :: Fall 2003

7. Problems of Delay • Interactive applications • Video conferencing • Human communication starts to break down at around 500 msec delays. • Feedback loops • Haptic interfaces • Quality of Service • Best effort and end-to-end mechanisms can do little to control delay. CS 294-9 :: Fall 2003

8. Problems of Jitter • Buffer management • Buffers are used to smooth jitter, but how big should the buffer be? • Unexpected jitter can lead to starvation. • Over-provisioning buffers creates delay. • Jitter is hard to bound even with QoS. • Media jitter vs. packet jitter • Variable bitrate media (where number of packets for “unit” of media varies) can make this relationship hard to model. CS 294-9 :: Fall 2003

9. Problems of Throughput • Estimating throughput is problematic. • Media coding may be tunable, but generally changes in quality are perceptually annoying. • Throughput will often vary faster than feedback delay which makes estimation hard. • Multicast makes this worse. • Scalable encoding is one solution path. • QoS mechanisms are another. CS 294-9 :: Fall 2003

10. Problems of Loss • Effect of loss is very media and encoding specific. • Loss of one packet in a video stream can result in the loss of multiple frames. • Other packets received can be rendered useless. • Goodput vs. Throughput • Retransmission often not feasible. • Encoding may be designed to alleviate packet loss effects. • Forward Error Correction CS 294-9 :: Fall 2003

11. Administrivia • Who am I? • kmp@cs.unc.edu • On campus Mon. and Wed. (629 Soda) • Who are you? • Class web page • http://inst.eecs.berkeley.edu/~cs294-9 • Schedule of topics and readings posted here. • Need to sign up for presenting papers. • More on this next week. • Reading for next week. • Jim Blinn’s article on NTSC CS 294-9 :: Fall 2003

12. TCP/UDP Review • Transport protocols on top of IP • TCP: ordered, byte-stream, reliable • UDP: datagram, unreliable, unordered • Programming API: sockets • Originally from UNIX, but available in Windows as WinSock and Java • Network byte-order is big endian . CS 294-9 :: Fall 2003

13. UDP Packet Format 32 bits Src Port # Dest Port # Length Checksum Data CS 294-9 :: Fall 2003

14. UDP Review • Connectionless • No communication necessary other than the datagram itself. • Unreliable • No guarantee or feedback whether data got ther or not. • Unordered • Packets may arrive in any order • Limit on delay: 1 minute (comes from IP) CS 294-9 :: Fall 2003

15. TCP Packet Format 32 bits Src Port # Dest Port # Sequence # Acknowledgement # Hdr Len. Un-used Flags Window Size Checksum Urgent Ptr Options Data CS 294-9 :: Fall 2003

16. TCP Lifecycle • 3-way handshake • Establishes connection. • Data transfer • Congestion controlled • Reliable • In order. • FIN • Connection teardown. CS 294-9 :: Fall 2003

17. 3-way handshake • One side must be “listening” for a connection. • Whoever is listening is “the server.” • Client sends a SYN packet. • SYN flag is set. • Seq. # is randomly chosen starting byte number. • Server sends back SYN-ACK packet. • SYN flag is set, ACK flag is set. • Seq. # is set and Ack # reflects client’s Seq. # • Client sends back ACK CS 294-9 :: Fall 2003

18. Data Transfer • Full duplex connection. • Both sides can send data simultaneously. • Seq. # set to byte number of first byte. • Ack # set to next byte expected. • Receiver generates an ACK even if it doesn’t have data to send back. • Some implementations delay sending ACK to optimize piggyback case. • In general, implementations expected to generate one ACK for each data packet received. CS 294-9 :: Fall 2003

19. Reliability • Sender maintains data until acknowledged. • This is done at the OS. The application is free from this responsibility. • Sender estimates RTT from ACK’s. • Timeout values dynamically calculated from RTT and RTT variance. • If timeout occurs, data retransmitted. • Holes in the byte stream are possible. • Receiver still generates ACK, but ACK # reflects next byte needed in seq. • Multiple ACK’s used to retransmit quickly. CS 294-9 :: Fall 2003

20. Transfer Example Seq. 0 Seq. 10 Seq. 20 Seq. 30 Ack 10 Ack 10 Ack 10 Timeout Seq. 10 Ack 40 CS 294-9 :: Fall 2003

21. Flow Control Sender Receiver Application Application OS OS Last ACK’d byte TCP TCP First un-ACK’d byte Recv Buffer Send Buffer Free Space Last un-ACK’d byte Recv. Window Recv. Window is advertised to the sender in ACK. CS 294-9 :: Fall 2003

22. Congestion Control • Basic question: how fast to send? • Too slow: underutilize network • Too fast: no one gets anything useful through • Not responsive: congestion collapse • TCP congestion control is window-based. • TCP maintains a “window” of data in the network. • Successful transmissions allow window to increase. • Packet drops cause window to shrink. • Reliability feedback and congestion control coupled together (may not have been the best idea). CS 294-9 :: Fall 2003

23. Slow Start • Slow Start • When starting, don’t know what the window should be, so probe for capacity. • Initial window: 1 segment size • A “segment” is a set number of bytes (i.e., packet size). • For every sucessful ACK, increase window by 1 segement size. • Doubles window size every RTT. CS 294-9 :: Fall 2003

24. Slow Start Illustrated Seq. 0 Ack 10 Seq. 10, 20 Ack 20, 30 Seq. 30, 40, 50, 60 Ack 40, 50, 60, 70 CS 294-9 :: Fall 2003

25. Congestion Avoidance • Eventually slow-start results in a packet drop. This establishes “ssthresh”. • Ssthresh set at 1/2 of current window. • Current window shutdown to 1 segment. • Data retransmitted. • Slow-start begins again. • When window gets back to ssthresh, every ACK results in window += 1/window. • Additive increase, multiplicative decrease. CS 294-9 :: Fall 2003

26. AIMD Illustrated Window Size ssthresh ssthresh ssthresh Time CS 294-9 :: Fall 2003

27. Misc. TCP Oddities • Window is actually minimum of receiver advertised window and congestion window. • Duplicate ACK’s used to detect congestion before actual timeout. (Fast retransmission) • Special-case calculation of congestion window after fast retransmission to allow new data to be sent. (Fast recovery) • Lots of tweaks and options (SACK, T/TCP) CS 294-9 :: Fall 2003

28. Multimedia and TCP/UDP • Streaming multimedia rarely uses TCP. Why? • Retransmission is pointless • By the time data is recovered, too late. • AIMD congestion makes jitter and throughput changes unbearable. • So, why did we just review TCP? • Everything else in the world is TCP. • How non-TCP traffic affects TCP traffic is an important question. • In general, streaming media protocols built on top of UDP. CS 294-9 :: Fall 2003