ORTC Walk Through IETF 88

1. ORCA – Object-RTC API Community Group ORTC Walk Through IETF 88 Robin Raymond, Hookflash Bernard Aboba, Microsoft Adalberto Foresti, Microsoft Open Technologies, Inc.

2. Meeting Purpose • The purpose of this meeting is explain: • Why a community has come together to work on an alternative API • High level features of the ORTC API • How to use the API to implement a simple video chat demo • How you can get involved with developing the spec and code • Introduce open source effort / call to action • There will be time for refreshments and Q&A the last 30 minutes • Session is recorded and streamed to the web(sponsored by Hookflash)

3. Introduction to ORTC ORTC – Object Realtime Communications • A next-generation JavaScript API for browser, mobile, and cloud realtime audio / video scenarios • ORTC is an alternative API to the W3C WebRTC 1.0 draft API • It is being developed in the ORCA W3C Community Group

4. The ORCA W3C Community Group W3C Community Groups are forums open to all without fee • Simple IPR policy – royalty-free commitments to one’s own contributions • Not on standards track • Mature CG specs can get broader IPR commitments, transition to standards track • Don’t have to be W3C member!

5. The ORCA W3C Community Group Object RTC API Community Group, • Formed in August 2013Chair and editor: Robin Raymond • Goal to develop WebRTC API that exposes an object API rather than blobs of SDP • Community group founded by JS developers coming from both SIP/other protocol signaling backgrounds, node.js and WebRTC enthusiasts.

6. This is not a community group meeting • Goal here is to explain mission and progress, recruit new members • To suggest API changes, join the Community Group – we can’t accept suggestions from those who have not signed the Contributor License Agreement

7. Special Thanks to… ORCA Community Group asked Microsoft Open Technologies if they would implement ORTC as demo / validation of concept. Special thanks for Microsoft Open Technologies, Inc. for implementing demo / validation of concept. (and the refreshments!)

8. WebRTC Needs • Mobile: SDKs enabling development of mobile apps protocol-compatible with WebRTC. • Analytics. Diagnostics and monitoring for carrier scale operation. • Cloud. Easy setup and management of WebRTC systems. • Tools. Enabling connection of WebRTC clients to legacy infrastructure (e.g. node.js) • APIs. Enabling management of STUN/TURN credentials, audio/video mixers, SIP proxies, XMPP connection managers, etc. • Extensibility: Enabling addition of functionality beyond the core (e.g. mobile-friendly video codecs such as H.264/SVC)). • A next generation JavaScript API compatible with the above.

9. What are ORTC potential uses? • Browser code (simplified RTC control) • Mobile development via “ortclib” SDKunder development https://github.com/openpeer/ortclib • Server code via Node.jsnew services or legacy inter-op solutions – call to contributeunder development https://github.com/openpeer/ortc-node Open Source projects are under way!

10. Write ORTC apps using existing WebRTC 1.0 Browsers An WebRTC 1.0 “shim” will be made available to maintain backwards compatibility If you like the WebRTC 1.0 API, you can still use the 1.0 API!

11. ORTC Migration Strategy 1.0 Strategy 2.0 Future ? Web App Web App SDP / SIP (shim) ORTC (shim) ORTC Browsers Node.JS Mobile WebRTC 1.0 (SDP) Node.JS Browsers (see demo) Mobile (ortclib) Chrome / Firefox / etc

12. Isn’t ORTC just CU-RTCWeb? No Both are object models but at different levels While CU-RTCWeb inspired ORTC, they are are different. Issues with CU-RTCWeb: • too powerful/difficult to use • exposing too much socket level control (i.e. security risks) • couldn’t shim CU-RTCWeb

13. Where will ORTC be supported by vendors? Support is via the shim over WebRTC 1.0 for now for browsers. Direct ORTC API source code is being developed. Microsoft Open Tech developed a shim plugin for IE (but not a product announcement).

14. openpeer/ortcGitHub Specification Repo https://github.com/openpeer/ortc • Original home of the ORTC API discussion • Spec discussion spun off to CG, OSS code for ORTC stays in GitHub • Why do we need the GitHub repo? • Pull requests – discussed on public-orca mailing listhttp://www.w3.org/community/orca/

15. openpeer/ortclibGitHub Mobility Repo https://github.com/openpeer/ortclib • ORTC mobile implementation • Maintained by Hookflash • Exposes C++ WebRTC library as an ORTC API • To be wrapped in iOS, Java (for Android) and JavaScript hybrid

16. openpeer/ortc-nodeGitHub Node.JS Repo https://github.com/openpeer/ortc-node • ORTC Node.JS implementation • Call to action – seeking contributors • What should implementation look like? • What are native code integration points?

17. Will contributing to ortcbreak protocol compatibility with web browser? No Wire level compatibility is maintained Maintaining wire compatibility is a tenet of ORTC implementations with WebRTC enabled browsers* We encourage expansion of features, not removal of basic wire support. * as much as they are compatible with each other

18. Motivation for ORTC Design

19. Traditional Browser Rendering Engine Resource JS Description Format (HTML/CSS) Render Engine Output

20. This is not how we program HTML on the web… JS asks browser “Give me webpage that meets this criteria” 1 Browser generates some HTML (gives to JavaScript) 2 Modify browser provided HTML 3 4 Set HTML

21. WebRTC 1.0 Capture / Render Engine { Cloud } JS JS Description Format (SDP) Description Format (SDP) Resource Resource Capture/Render Engine Capture/Render Engine Input/Output Input/Output

22. ORTC Capture / Render Engine { Cloud } JS JS Capture/Render Engine Capture/Render Engine Input/Output Input/Output

23. ORTC Simple Use Case { cloud } signaling signaling Browser Browser Application Application RTCConnection RTCConnection RTP / DATA / DTLS / ICE

24. Questions?

25. ORTC API

26. ORTC Simple Use Case { cloud } signaling signaling Browser Browser Application Application RTCConnection RTCConnection RTP / DATA / DTLS / ICE

27. ORTC - Object Relationships remote RTCIceCandidate RTCIceCandidate RTCConnectionOptions RTCConnection RTCSocket local remote local RTCStream MediaStream RTCConnectionInfo RTCStream MediaStream RTCIceCandidate RTCIceConnectionInfo RTCConnectionInfo RTCIceConnectionInfo RTCTrack MediaStreamTrack RTCIceCandidate RTCTrack MediaStreamTrack RTCTrack MediaStreamTrack RTCDtlsConnectionInfo RTCTrack MediaStreamTrack RTCDtlsConnectionInfo

28. RTCConnection [Constructor(RTCConnectionOptions options)] interface RTCConnection : EventTarget { readonly attribute RTCSocket socket; readonly attribute RTCConnectionState state; attribute RTCConnectionRole role; readonly attribute RTCConnectionInfo local; attribute RTCConnectionInfo remote; void addRemoteCandidate(RTCCandidateDescription candidate); void connect(); void gather(); (RTCSTream? or RTCTrack?) send((MediaStream or MediaStreamTrack or RTCTrack or RTCStream) streamOrTrack); (RTCStream? or RTCTrack?) receive((RTCStream or RTCTrack) streamOrTrack); sequence<RTCStream> sendStreams(); sequence<RTCTrack> sendTracks(); sequence<RTCStream> receiveStreams(); sequence<RTCTrack> receiveTracks(); void close (); attribute EventHandleroncandidate; attribute EventHandleronendofcandidates; attribute EventHandleronactivecandidate; attribute EventHandleronstatechanged; attribute EventHandleronnetworkchange; attribute EventHandleronunknowntrack; };

29. RTCConnection (p1) • RTCSocket (attached to one network socket) • States • “new” – connection not started • “searching” – searching for connectivity • “halted” – no connectivity found • “connected” – found connectivity (ready) • “closed” – connection is finished (one-time-use)

30. RTCConnection (p2) • role (“controlling” vs “controlled”) • Local and remote information • ICE credentials (usernameFrag / password) • DTLS fingerprint (generated or discovered) • ICE Candidates – local and remote (for connectivity checks)

31. RTCConnection (p3) • Gathers local ICE candidates • Performs ICE connectivity checks • Causes sending / receiving of media tracks RTP data over the “nominated” ICE connection

32. RTCConnection (p4) RTCStreams and RTCTracks (sending or receiving) • Contains mapping of on-the-wire RTC/RTP/RTCP/etc information to media streams / media stream tracks) • Contains respective “MediaStream” or “MediaStreamTrack” objects.

33. RTCConnection “connect” Set remote “RTCIceConnectionInfo” ICE “usernameFrag” and “password” Call “connect” to cause candidate search Will causes state changes from “new”, “searching”, “haulted”, “connected”

34. RTCConnection “gather” Find new candidates - Restarts network topology ICE candidate gathering Does not affect connection state by itself Start gathering candidates before “connect”

35. RTCConnection “send” Send “MediaStream”, “MediaStreamTrack”, “RTCStream”, “RTCTrack” Convenience to auto-map “MediaStream” and “MediaStreamTrack” into RTP layer constructs Manipulate RTP mapping by first creating “RTPStream” / “RTCTrack” from “MediaStream” and “MediaStreamTrack”

36. RTCConnection “receive” Send “RTCStream”, “RTCTrack” Setup mapping for “MediaStream” and “MediaStreamTrack” for RTP layer constructs Manipulate mapping by constructing “RTPStream” / “RTCTrack” objects

37. RTCConnection “close” Shuts down RTCConnection object Object goes to “closed” state

38. RTCConnection events oncandidate Fires for local candidate (can be cancelled) onendofcandidates Fires when all local candidates are discovered onactivecandidate Fires when local:remote candidate connection changes onstatechanged Fires as connection state changes onnetworkchange Fires when network topology change has been detected (auto “gather” called but can be cancelled) onunknowntrack Fires when track discovered which has no mapping

39. RTCSocket interface RTCSocket { };

40. RTCSocket • No exposed attributes, members or events • Represents one physical local socket (as well as virtualized paired firewall/relay socket) • Re-use the same RTCSocket in another RTCConnection to send over same physical local socket

41. RTCStream [Constructor, Constructor(MediaStream)] interface RTCTrack { attribute MediaStream source; attribute DOMString msid; sequence<RTCTrack> tracks(); void start(); void stop(); void remove(); };

42. RTCStream • References single MediaStream • Collection of RTCTracks • Change collection then call RTCConnection “send”/”receive” again • Simplifies MediaStream usage in RTCConnection • Convenience “start” / “stop” methods • “remove” method to remove from RTCConnection

43. RTCTrack [Constructor, Constructor(MediaStreamTrack)] interface RTCTrack { attribute MediaStreamTrack source; attribute DOMString id; attribute DOMString kind; attribute unsigned long ssrc; attribute sequence<RTCCodec> codecs; attribute sequence<RTCMediaAttributes> mediaAttributes; attribute Object rtpExtHeaders; void start(); void stop(); void remove(); };

44. RTCTrack • References single MediaStreamTrack • Mapping of RTP/RTCP/etc wire information such as: • ssrc • codecs • media attributes (e.g. max width / height) • Change properties then call RTCConnection “send” / “receive” again • Convenience “start” / “stop” methods • “remove” method to remove from RTCConnection

45. RTCConnectionInfo dictionary RTCConnectionInfo { attribute RTCIceConnectionInfo ice; readonly attribute RTCDtlsConnectionInfo dtls; }; RTCIceConnectionInfo dictionary RTCIceConnectionInfo { DOMString usernameFrag; DOMString password; }; RTCDtlsConnectionInfo / CertificateFingerprint dictionary RTCDtlsConnectionInfo { attribute CertificateFingerprint fingerprint; }; dictionary CertificateFingerprint { getter ArrayBuffer (DOMString hashFunction); };

46. RTCIceCandidateInfo dictionary RTCIceCandidateInfo { DOMString foundation; int component; DOMString transport; int priority; DOMString connectionAddress; int connectionPort; DOMString type; DOMString relAddress; int relPort; };

47. Object Realtime Communications API (ORTC) • Next generation WebRTC API, under development in the W3C ORCA community group: • http://www.w3.org/community/orca/ • Rationale: http://tools.ietf.org/html/draft-raymond-rtcweb-webrtc-js-obj-api-rationale • Specification: • http://openpeer.github.io/ortc/ • Githubrepository: • ORTC API drafts and shim: https://github.com/openpeer/ortc • ortclib: https://github.com/openpeer/ortclib • Demos: https://github.com/openpeer/webrtc-demo

48. To Learn More • ORCA Community Group: http://www.w3.org/community/orca/Demo and discussion tomorrow to follow…