WebRTC PeerConnection

1. WebRTCPeerConnection Halloween, 2011 Cullen Jennings fluffy@cisco.com

2. The Current Situation • PeerConnectionemits messages like this: “SDP\n v=0\n o=- 2890844526 2890842807 IN IP4 192.0.2.1\n s= \n c=IN IP4 192.0.2.1\n t=2873397496 2873404696\n m=audio 49170 RTP/AVP 0”

3. Well, that’s kind of lame • What if we want to add something else? • For instance, a message ID • Maybe some structure would be nice….

4. A JSON version • { • “sdp”:”v=0\n • o=- 2890844526 2890842807 IN IP4 192.0.2.1\n • s= \n • c=IN IP4 192.0.2.1\n • t=2873397496 2873404696\n • m=audio 49170 RTP/AVP 0” • } • Now at least we can extend things • }

5. How do we tell offers from answers? • Easy to get confused if we have glare • We both send SDP at the same time • Is the SDP I just got a new call or an answer to my offer?

6. Message types • { • “messageType”: “OFFER”, • “sdp”:”v=0\n • o=- 2890844526 2890842807 IN IP4 192.0.2.1\n • s= \n • c=IN IP4 192.0.2.1\n • t=2873397496 2873404696\n • m=audio 49170 RTP/AVP 0” • } • }

7. How do we identify sessions? • We might have multiple sessions going on • Either simultaneously or in sequence • Message delays can make these look like each other

8. Session ID • { • “messageType”: “ANSWER”, • "callerSessionId":"13456789ABCDEF", • "calleeSessionId":"abc1234356", • “sdp”:”v=0\n • o=- 2890844526 2890842807 IN IP4 192.0.2.1\n • s= \n • c=IN IP4 192.0.2.1\n • t=2873397496 2873404696\n • m=audio 49170 RTP/AVP 0” • } • Session ID must be globally unique

9. How session IDs get established • { • “messageType”: “OFFER”, • "callerSessionId":"13456789ABCDEF", • … • } • { • “messageType”: “ANSWER”, • "callerSessionId":"13456789ABCDEF", • "calleeSessionId":"abc1234356", • … • } • Each side contributes a session ID • Session defined by combination of each side

10. Multiple offer/answer pairs • A session has a sequence of offer/answer pairs • Example: upgrade to video • We have an audio call • You decide to add video • This requires multiple offer/answer pairs • How do we distinguish them?

11. Sequence field • { • “messageType”: “OFFER”, • "callerSessionId":"13456789ABCDEF", • “seq”:1, • “sdp”:”v=0\n • o=- 2890844526 2890842807 IN IP4 192.0.2.1\n • s= \n • c=IN IP4 192.0.2.1\n • t=2873397496 2873404696\n • m=audio 49170 RTP/AVP 0” • } • Sequence indicates the current offer/answer exchange • OFFER and ANSWER have same sequence number

12. ANSWER confirmation • It’s not safe to have multiple OFFERs outstanding • What happens if I do two changes in succession? • Example: • Other side adds video • I accept but then user changes camera to one with different capabilities • I need to re-OFFER but when? • After he’s gotten my ANSWER

13. OK message • { • “messageType”: “OK”, • “seq”:”2”, • "callerSessionId":"13456789ABCDEF", • "calleeSessionId":"abc1234356”, • } • OK message indicates ANSWER received and accepted • New session parameters are active • Safe to do a new OFFER/ANSWER pair

14. ICE Pipelining • ICE is slow (can take 3-5 seconds to converge) • We want to start ICE as soon as possible • Best experience is to start ICE when Bob receives the OFFER even before he accepts the call • But we don’t know the media parameters till he answers the call • For instance, he might accept audio but not video

15. moreComing flag • { • “messageType”: “ANSWER”, • "callerSessionId":"13456789ABCDEF", • "calleeSessionId":"abc1234356", • “moreComing”:true, • “sdp”:… // ICE candidates but recvonly media • } • moreComing flag means that another ANSWER will follow to this OFFER • OFFER/ANSWER transaction isn’t complete till moreComing=false • No OK for moreComing=true

16. Configuration • Need to pass in TURN and STUN servers and credentials • Often optimal to use more than one • Proposal, array of URL • Note Current text broken for IPv6 IP addresses • Draft has: • new PeerConnection( “TURNS 203.0.113.2:3478”, … • Proposal: • New PeerConnection( [“turns://203.0.112.2:3478”,”stun://example.com”], …

17. Syntax for Offer / Answer Messages • JS Objects • JSON strings • Other • Use ROAP? • Move def of application/html-peer-connection-deta to ROAP?

18. Label • Help ! • Draft has • All SDP media descriptions for streams represented by MediaStream objects must include a label attribute ("a=label:") whose value is the value of the MediaStream object's label attribute. [SDP] [SDPLABEL] • Is this label really the same as the SDP “a=label” ? • Can’t require that we have this with legacy things? • Why do we have this?

19. Stats • Proposal • Have a stats method on PeerConnection • Have it return a dictionary of stats • Define an IANA registry of well known stat names • The dictionary value for a stat can be primitive type or an array or object

20. Hints • Tell CODECs something about the application that they may need to understand to make good encoding choices • Audio: is music or is spoken voice • Video: prefer spatial or temporal fidelity • Proposal • IANA registry of well known hints • Have some sort of setHints method on a media streams

21. Glare • It happens (lets add video … ) • Deal with this in IETF • ROAP takes care of it

22. Data Channel • Proposal – Focus on the API • Send and receive a string/blob • Unreliable delivery ? • Reliable delivery ? • Out of order delivery? • Stream of data ?

23. DTMF on Audio Stream • Need to callback when received DTMF and way to tell audio stream to send DTMF • Proposal: • Add callback and method to send to audio streams • Add a way to disable it when creating stream

24. Track Order • Draft has • The relative order of all tracks in a user agent must be stable. All audio tracks must precede all video tracks. • Why do we need audio before video? • Complicates mapping to SDP which is not this way