1 / 57

Telepresence Tutori A l

Telepresence Tutori A l. July 30, 2012. Overview. Introduction to Telepresence (Stephen Botzko ). What is Telepresence: Co-location. At its core, Telepresence uses technology and “ stagecraft ” to create a sense of co-location (meeting participants feel they are in the same space).

aline-chapman
Download Presentation

Telepresence Tutori A l

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. TelepresenceTutoriAl July 30, 2012

  2. Overview

  3. Introduction to Telepresence(Stephen Botzko)

  4. What is Telepresence: Co-location At its core, Telepresence uses technology and “stagecraft” to create a sense of co-location (meeting participants feel they are in the same space). Key Aspects: Gaze Awareness, Eye Contact, Actual Size Rendering Telepresence Dinner

  5. History • “Toward the Telehandshake” 1983 Media for Interactive Communications; Bretz and Schmidbauer • Commercial systems began in the 90s • TeleSuite founded in 1993 • Cisco, HP, Polycom, etc. by 2010.

  6. Some Product Examples

  7. Telepresence: Definition • Telepresence: An interactive audio-visual communications experience between remote locations, where the users enjoy a strong sense of realism and presence between all participants by optimizing a variety of attributes such as audio and video quality, eye contact, body language, spatial audio, coordinated environments and natural image size.

  8. How is it done? Lay out physical space / Identify sight lines

  9. How is it done? Partition the space

  10. How is it done? Place cameras and displays

  11. Essential Co-location Requirements • Preserve spatial relationships between streams • Maintain coherence of audio and video “stage” • Ability to scale images to true size • Ability to select best sight line • Many of these facilities can also be used to enhance other non-telepresence applications.

  12. IETF CLUE Working Group (Mary Barnes)

  13. What is CLUE? • CLUE = ControLling mUltiple streams for tElepresence • Motivation: • Currently deployed telepresence systems are not interoperable

  14. What is CLUE? • Objectives: • Describe the data required for the handling of multiple streams • Define the behavior required to negotiate the use of multiple streams of audio and video media flows

  15. Scope of CLUE • RTP and SIP based systems • Define signaling for transporting CLUE information • Apply existing protocols for signaling and transport • Extensions to existing protocols in appropriate WGs (e.g., AVTCORE and MMUSIC)

  16. Working Towards a Solution Requirements Use Cases Framework Data Model Signaling Call Flows RTP Usage

  17. CLUE Telepresence Scenarios(Roni Even)

  18. Overview • Telepresence systems (TP) • Primary objective is for an immersive experience as close to “being there” as possible • Life-size video display • Eye contact • Gaze direction • Spatial audio

  19. Central Camerassemi circular seating

  20. Cameras located with screensSemi circular or Linear classroom seating

  21. Telepresence architecture • TP systems will typically have multiple cameras and microphones • Typical system will have the same number of monitors and cameras (1 and 3 are common but some systems will have 2 and 4)

  22. Additional Use Cases • Dynamically add video sources from an endpoint based on meeting context • E.g. turn on a document camera or provide video stream of presentation • Different number of cameras and screens. Example 3 cameras with six screens or with one big screen.

  23. CLUE Framework (AllynRomanow, Andy Pepperell)

  24. Extensible Interoperable Power of the Framework • Different vendors • Types of devices • New functionality • Future

  25. Extensible Interoperable Power of the Framework • Different vendors • Types of devices • New functionality • Future • Receiver driven • Chooses what to receive and encoding • Media captures • Description used by renderer • Advertised by provider • Chosen by consumer

  26. What is the Framework? Vendor One Vendor Two 30

  27. Provider and Consumer I am a provider. I advertise I am a provider. I advertise I am also a consumer. I choose I am also a consumer. I choose I am a provider. I advertise I am also a consumer. I choose I am a provider. I advertise I am also a consumer. I choose MCU

  28. Basic Idea I can send you one image, 2 images, or 3 images; 1 or 2 mono audio streams . I can send streams at 1080p , 720p, and 360 as long as total not over 4896 Mbps. All at 4Gps not exceeding 6Gbps. Audio at 64 kbits each. Woof woof. I’ll take the single stream at 720p, single mono audio. I can send you 1 image of both of us, or 2 images each of 1 of us. I can send them at 1080p, 720p or 360. 1 mono audio at 64k Meow.. Send me 2 streams of 360 at 1080p, and 1 audio..

  29. Media Captures • Fundamental CLUE concept • Media capture is a media representation of some portion of the provided scene • Eg #1: video from the left camera of 3 (maybe show in diagram) • Eg #2: a stereo audio capture of a room’s audio

  30. Capture Attributes • Each capture described via its attributes • High-level categorization, audio vs video • Spatial information (“3 – D cartesian co-ordinates”) to enable correct rendering • Switched capture • Mechanism for extensibility

  31. Capture Scene Capture Scene Entries VC0 VC1 VC2 Main Media (VC0, VC1, VC2) (VC3, VC4) (VC5) (AC0) Three cameras VC3 VC4 Two cameras, moved and zoomed out VC5 Each alternative representation of a Scene is a capture entry in a Capture Scene Switched (based on voice), composed PiP

  32. Basic CLUE Messaging Consumer Provider Consumer Provider Provider capture advertisement Provider capture advertisement Consumer stream choice Consumer stream choice media streams media streams Provider capture advertisement Potentially multiple further exchanges Consumer stream choice

  33. Provider Capture Advertisement • Provider tells consumer about its media captures • Enumeration of available media captures • Includes organisation of captures into scenes • Physical constraints • Center camera may also be used for “zoomed out” view • Encoding constraints • Provider expresses its overall encoding capabilities • Allows modelling of multiple constituent physical units

  34. Consumer Choice • Consumer tells provider which captures it wishes to receive • Encoding parameters such as max resolution, mbps etc. • Instantiates provider media captures to “real” streams • Captures can have multiple instantiations; not a simple one to one mapping between captures and encodings • Media model no longer simply “transmitter chooses”

  35. Receiver Choosing is Powerful • Consumer do its own layout • Knows its display hardware • Number of streams, bw, resolution • Receiver multiple representations of same scene • Recording • MCU switch different versions out • Expanded functionality, flexibility

  36. Framework Realization (Rob Hansen)

  37. Example Endpoint - Alice

  38. Example Call-Flow Alice Bob SIP: INVITE SIP: 200 OK SIP: ACK (optional) Single-stream RTP + RTCP CLUE: Advertisement CLUE: Configure Multi-stream RTP + RTCP

  39. Example SIP INVITE • Acceptable to non-CLUE endpoints • As always, SDP defines limits of RTP sessions • INVITE contains CLUE transport details • Alice’s SDP has 1 audio m-line, 1 video m-line: v=0 o=alice 2890844526 2890844526 IN IP4 client.atlanta.example.com s=- c=IN IP4 192.0.2.101 t=0 0 b=AS:6064 m=audio 49172 RTP/AVP 0 a=rtpmap:0 PCMU/8000 m=video 49174 RTP/AVP 96 b=AS:6000 a=rtpmap:96 H264/90000 a=fmtp:96 profile-level-id=42e016;max-mbps=244800;max-fs=8160

  40. Example CLUE Advertisement • Capture Scene • Captures • Entries • Simultaneous Transmission Sets • Encoding Group • Encodings

  41. Example Captures Video Audio Capture 1 Static video Spatial parameters Capture 2 Static video Spatial parameters Capture 5 Mixed audio No spatial parameters Capture 3 Static video Spatial parameters Capture 4 Switched video No spatial parameters

  42. Capture Spatial Parameters • Point of Capture • Axis of Capture • Area of Capture Capture 1 Static video Region A Capture 2 Static video Region B Capture 3 Static video Region C

  43. Example Entries • Entries of the same media type define alternative views of the scene. • Alice advertises three entries: Entry 1: Video captures 1, 2 & 3 (three static cameras) Entry 2: Video capture 4 (switched video stream) Entry 3: Audio capture 5 (mixed audio stream)

  44. Encoding Group & Encodings • Encodings define the maximum encoding parameters available for streams. • Alice advertises the ability to encode up to three streams at 1080p, 4Mb, but with an overall limit of 6Mb: Encoding group limit: Max bandwidth 6Mb Video Audio Max 1080p @ 4Mb Max 1080p @ 4Mb Max 64kb Max 1080p @ 4Mb

  45. CLUE Configure • Bob selects the three static camera streams at 720p, and the mixed audio stream: Video Audio Static capture 1 Max 720p @ 2Mb Static capture 2 Max 720p @ 2Mb Mixed capture 5 Static capture 3 Max 720p @ 2Mb

  46. Multi-stream media • Alice sends 1 audio stream • Alice sends 3 multiplexed video streams Alice Bob Audio RTP session audio port Video RTP Session video port

More Related