1 / 23

Making tele-presence work or Making the AG work better

Making tele-presence work or Making the AG work better. Markus.Buchhorn@anu.edu.au Head, ANU Internet Futures. It’s all about tele-presence. The feeling of being there with somebody or something Requires sufficient “quality” to make the technology invisible Quality:

rafal
Download Presentation

Making tele-presence work or Making the AG work better

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Making tele-presence workorMaking the AG work better Markus.Buchhorn@anu.edu.auHead, ANU Internet Futures

  2. It’s all about tele-presence • The feeling of being there with somebody or something • Requires sufficient “quality” to make the technology invisible • Quality: • Image resolution, colour, framerate, … • Audio realism • Latency, jitter, errors, … • “Feel” for remote site(s) • Minimise cognitive distractions • Avoid loss of functionality • ‘Sorry, you can’t do that with this system’

  3. www.accessgrid.org(open-source ANL project) • Multicast-based (mostly) • Many-2-many interaction (in principle) • Multiple A/V streams per site (usually) • Some Application sharing (sort of) • PPT, WWW • Text Edit, Whiteboard, Instruments • No bandwidth constraints (ideally) • Today (h.261 x [2 - 4] + audio) ~ 2Mb/s per site • Can scale down to desktop and PDA

  4. Access Grid usage • Formal multisite events • Seminars, distributed classes, virtual conferences • Board/committee meetings • Casual usage • Walk-in discussions, ad-hoc meetings • Desktop monitoring • Wrapper to larger applications • Visualisation/Virtual Reality, • Computational Steering • Data browsing

  5. Who, Where? • 400+ sites today • 150+ U.S., 40+ Europe, 30+ Asia • 30 live in Oz, 15+ in plan/build (4+ at ANU) • Mostly at Universities, so far • Developers: • 20+ groups on shared applications • 10+ groups on visualisation/VR • 10+ groups on high-quality video • 3+ groups on user interfaces

  6. Spkr Cam Cam Mike Mike Proj Proj Proj Cam Display PC Audio PC VideoCap PC Control PC An AG (room sized) Node Screen

  7. Now: Making the AG work better • The power of the AG is its flexibility • The problem with the AG is too much flexibility • Lots of effort on new functionality • Often in domain-specific areas • Limited effort on • Robust-ifying existing functionality • Enhancing the common functionality

  8. The problem • 4+ AG (room) Nodes at ANU • 3 almost complete • Computer Science, HPC, Observatory • 1 under way • Medical School • 4 more being discussed • Including a 1500 seat theatre! • Plus various desktops and test systems • Pressure to make the AG nodes • Production quality (robust) • Cheaper to operate (less qualified operators) • Remotely operable • USER FRIENDLY !!!!!! • Maximise the feeling of tele-presence

  9. AG problems • Performance, benchmarking • Multicast • User Interface • Capture, codecs • Integration with “standard” videoconf. • Recording, recoding and playback • Hardware add-ons, control, and applications

  10. 1-Performance measurement • RTPreplicate • Stress your display/audio machines • One stream goes in, N come out • Use feedback from rtcp to benchmark • Derive a meaningful number • Full throughput test • network, interfaces, CPU/RAM/graphics bus

  11. 2-Multicast • Multicast isn’t everywhere • Bandwidth isn’t everywhere • Multicast/unicast bridges don’t scale well in management • Host must reconfigure all the time • RCbridge • Web interface, per request bridge set up and configured on demand, any number of clients/sessions • rtp stream selection (home use, wireless use, …)

  12. RCBridge • http is scriptable – command line clients • Can build a P2P network of unicast fowarders,e.g. for failover, or application-layer multicast – but prefer multicast to be FIXED!!!

  13. 3-User Interface • Vic (video) and rat (audio) • old, unfriendly • Few new codecs (DV, MPEG-x, MJPEG, HDTV, surround/hi-fi sound, …) • And don’t scale all that well • No lip-synch • No audio-localisation, to match display • Manageability is a problem • Major effort of node operator • No ability to scale hardware further, across clusters • Performance and reliability advantages

  14. VP site-tiles and layout managers

  15. VP – a new start • Site-tiles (grouping and prioritising) • Display manager (layout of site tiles) • Use of OpenGL capabilities • Smooth (not just quantised) scaling, done in hardware • Much more flexible user interfaces possible • Other metadata • Stream layout (e.g. panorama) • Local time (why is that person asleep??) • Network issues (have we lost multicast again??) • Flags, icons (graphics to identify a site)

  16. VP – a new start • Remote Operation • “VP-master” controls the “VP-client”, over the network • Cluster display: • MULTIPLE VP clients, still single VP-master • Works across different operating systems at the same time! • Audio (via existing audio tool for now) • lip-synchronisation, • Visual highlights • Virtual audio localisation, • Runs on Windows, Linux; FreeBSD and MacOSX

  17. VP – with DV and MPEG4 and H263 in the next release

  18. 4 – Capture, Encoding • “VC” – new project in 2004 • Supported by AUDF; Mac first! • Any number of input devices • Various filters • Watermarks • Captions, banners • Chromakey • Metadata • Layout, site information, priorities, … • Various codecs • H.261, m(j)peg-X?, DV?, H.263, H.264

  19. 5 - Integration with H.323, … • H.323 is wide-spread, AG is not (yet) • Bridging between two worlds • Analogue • VRVS • Silver323 • Open source, software gateway • Some MCU-like functionality • Hardware constrained, but cluster-able • http managed, very flexible and “automate-able” stream selection • Provides an easy path into VoIP and PSTN • SIP, streaming easy to add

  20. 6 – Recording, recoding, playback • Current practice: rtpdump, rtpplay • Poor design, does not scale, • no remote control • New toolkit • Recording, playout daemons, remote control • Scalable file format • Additional media types • Whiteboard, shared-apps, annotations, … • Tools to extract subset of streams • Tools to convert to e.g. DVD formats

  21. 7 – Hardware, Applications • Integrate electronic pen-tracking whiteboard • Astronomers want blackboard/chalk integration! • E-beam and Mimio are excellent • Existing whiteboard apps are mouse driven, and old • Remote control of nodes • Design nodes to be remotely managed, user managed (cameras, projectors, vcr/dvd, …) • Maintain rapid user support in rooms (operator-cam, with dedicated audio) • More AG-literate applications • Shared Acrobat • Library for generic apps (cross-platform) to code to • Most generic: VGA capture!

  22. Conclusion: AG development • Performance, benchmarking • Multicast • User Interface • Capture, codecs • Integration with “standard” videoconferencing • Recording, recoding and playback • Hardware add-ons, control, applications • But we need your help, your code, your ideas, and your advice – you, the users! • http://if.anu.edu.au

More Related