On Distributed Real-time Systems:

1. On Distributed Real-time Systems: the mStar Environment,Net-based Learning andContext-aware Applications Kåre Synnes Department of Computer Science and Electrical Engineering Centre for Distance-spanning Technology Luleå University of Technology

2. Table of Contents • Background • Publications • The mStar Environment • Robust Audio Transport • Light-weight Application Level Tunneling • Context-aware Applications • Privacy • Net-based Learning • Distributed Real-time Systems • Conclusions “ The best way to predict the future is to invent it. ” - Alan Kay, 1971

3. BackgroundDistributed Real-time Systems • Distributed • Internet (TCP, UDP, IP) • IP-Multicast, Peer-to-peer • Real-time • Non-hard real-time characteristics • Synchronous • Systems • From protocols to applications • Working prototypes • Feedback from real users

4. BackgroundDistributed Real-time Systems • Two frameworks for distributed real-time systems have been designed: • The mStar Environment • The Alipes architecture • The two frameworks are primarily discussed from two areas of usage: • Distributed Teamwork • Net-based Learning

5. Table of Contents • Background • Publications • The mStar Environment • Robust Audio Transport • Light-weight Application Level Tunneling • Context-aware Applications • Privacy • Net-based Learning • Distributed Real-time Systems • Conclusions “ The best way to predict the future is to invent it. ” - Alan Kay, 1971

6. PublicationsThe mStar Environment Paper A Peter Parnes, Kåre Synnes, Dick Schefström, “The CDT mStar Environment: Scalable Distributed Teamwork in Action”. In the proceedings of Group 1997, Phoenix, Arizona, USA, November 1997. Paper B Peter Parnes, Kåre Synnes, Dick Schefström, “Lightweight Application Level Multicast Tunneling using mTunnel”. In the Journal of Computer Communication, Volume 21, Issue 15, pp. 1295-1301, October 1998. Paper C Peter Parnes, Kåre Synnes, Dick Schefström, “A Framework for Management and Control of Distributed Applications using Agents and IP-multicast”. In the proceedings of IEEE Infocom 1999, New York, USA, March 1999. Paper D Peter Parnes, Kåre Synnes, Dick Schefström, “mStar: Enabling Collaborative Applications on the Internet”. In the Journal of Internet Computing, September/October 2000. Paper E Kåre Synnes, Peter Parnes, Dick Schefström, “Robust Audio Transport using mAudio”. Research Report 1999:04, ISSN 1402-1528, ISRN LTU-FR--99/04--SE, Luleå University of Technology, April 1999.

7. PublicationsNet-based Learning Paper F Kåre Synnes, Serge Lachapelle, Peter Parnes, Dick Schefström, “Distributed Education using the mStar Environment”. Research Report 1997:25, ISSN 1402-1528, ISRN LTU-FR--97/25--SE, Luleå University of Technology, November 1997. In the proceedings of WebNet 1998 (Top Paper Award), Orlando, Florida, USA, November 1998. An extended version was also published in the Journal of Universal Computer Science, Vol. 4, Issue 10, pp. 807-823, October 1998. Paper G Kåre Synnes, Peter Parnes, Johnny Widén, Dick Schefström, “{Student 2000:} Net-based Learning for the Next Millennium”. Research Report 1999:05, ISSN 1402-1528, ISRN LTU-FR--99/05--SE, Luleå University of Technology, April 1999. In the proceedings of WebNet 1999, Honolulu, Hawai’i, USA, October 1999. Also presented in a shorter version at SCI/ISAS 1999, Orlando, Florida, USA, August 1999. Paper H Kåre Synnes, Tor Söderström, Peter Parnes, “Learning in Desktop Video-Conferencing Environments”. In the proceedings of WebNet 2001, Orlando, Florida, USA, October 2001.

8. PublicationsContext-aware Applications Paper I James Nord, Kåre Synnes, Peter Parnes, “An Architecture for Location-aware Applications”. In the proceedings of HICSS-35 (Nominated for Best Paper), Big Island, Hawai’i, USA, January 2002. Paper J Kåre Synnes, James Nord, Peter Parnes, “Location Privacy in the Alipes Platform”. Research Report 2002:08, ISSN 1402-1528, ISRN LTU-SR--02/08--SE,Luleå University of Technology, April 2002. Submitted for review to HICSS-36.

9. Table of Contents • Background • Publications • The mStar Environment • Robust Audio Transport • Light-weight Application Level Tunneling • Context-aware Applications • Privacy • Net-based Learning • Distributed Real-time Systems • Conclusions “ The best way to predict the future is to invent it. ” - Alan Kay, 1971

10. The mStar EnvironmentResearch Issues • What does the new generation of Internet-based real-time applications and systems require of a framework to allow distributed clients to act as autonomous entities with individual control, while at the same time allowing these clients to share information and control to jointly work as one system? • How can real-time communication be made robust and adaptable to handle lossy and congested network conditions in general? How can an audio agent be designed to handle delay, delay variation and packet loss in particular? • How can agents (modules) be designed for reuse and adaptation when implementing aggregated clients and systems? • How should frameworks be designed to allow usage by mobile users, who inherently have limited terminal and network capacities?

11. mVCR • mMOD • mIndex • mEdit • mVideo • SlideBurster • Director Most tools start with a small m, thus the mStar (m*) name. • mDesk (WebDesk) • mWhiteBoard • mAudio • mWeb • mChat • mVote • mWave • mIR • mRadio • mTunnel • mGW • mSD • mSP • mPortal The mStar EnvironmentmStar -> Marratech ’Better-than-being-there’

12. Audio is the most important real-time media Packet Loss Mostly 1-5% Occasionally higher mAudio Audio Quality Tool 3 sample technique Up to 20% loss can be repaired using fairly simple techniques substitution silence noise self-similarity repetition mixing redundancy The mStar EnvironmentRobust Audio Transport

13. The phrase ‘Hello World!’ (40% loss) The three waveforms: original mixing redundancy/repetition Combining methods for recovery are very effective! The mStar EnvironmentRobust Audio Transport

14. Many networks lack support for multicast Tunneling! Some networks cannot support high bandwidths Bandwith reduction! mTunnel web interface inter-domain mGW mobile applications intra-domain The mStar EnvironmentLight-weight Application Level Tunneling

15. Saving bandwidth Transcoding of data Statistical compression by grouping packets together 5-14% gain depending on traffic type Advanced measures to save bandwidth mixing audio sources switching video source depending on current speaker lowering the frame rate The mStar EnvironmentLight-weight Application Level Tunneling Yields extra delay!

16. Table of Contents • Background • Publications • The mStar Environment • Robust Audio Transport • Light-weight Application Level Tunneling • Context-aware Applications • Privacy • Net-based Learning • Distributed Real-time Systems • Conclusions “ The best way to predict the future is to invent it. ” - Alan Kay, 1971

17. Context-aware ApplicationsResearch Issues • What distinguishes a mobile application from its ‘stationary’ counterpart? Are they identical, or is it rather a question of the client and terminal capacities? Is location the primary distinguishing factor, in other words using context information to enhance usability and other factors in mobile terminals? • How could seamless selection between alternative positioning techniques be achieved? Could alternative positioning techniques be used in conjunction and thus serve as additive techniques? Could position information be exchanged between nearby portable devices? • How can access to location and context information be restricted to ensure the privacy of a user of a framework for distributed real-time applications? What criteria are important and how could usability be considered from the very start?

18. SB Map-service Yellow Pages Location DB Web Search RFID IRDA Context-aware ApplicationsThe Alipes Platform Applications Privacy / Security Positioning platform Map Service Service Infobase GPS GSM/GPRS Bluetooth WaveLAN

19. Carol Dave Platform Platform MPS Bluetooth Bluetooth WaveLAN MPS ±ε WaveLAN ±ε Context-aware ApplicationsPeer-to-Peer Position Exchange

20. Context-aware ApplicationsTracker and GeoNotes

21. The user owns and controls his own position information! The user decides who is allowed to access his position by granting a contract Period in time Number of queries Granularity Delegation rights A contract can be set up between the user’s client and... A client application A location server The user can choose to delegate the right to set up contracts to a location server Context-aware ApplicationsPrivacy

22. Table of Contents • Background • Publications • The mStar Environment • Robust Audio Transport • Light-weight Application Level Tunneling • Context-aware Applications • Privacy • Net-based Learning • Distributed Real-time Systems • Conclusions “ The best way to predict the future is to invent it. ” - Alan Kay, 1971

23. Net-based LearningResearch Issues • What are the effects of using the frameworks for distributed teamwork in general and net-based learning in particular? • What services and applications could best exemplify the future use of an integrated framework for distributed real-time applications with support for context-awareness and conferencing for distributed teamwork and net-based learning?

24. Net-based Learning • One of the primary usages of the mStar environment • Lectures • Seminars • Groupwork • Presence (Corridor) • Interactivity! • Independence • Space & Time Tutored groupwork for language learning

25. Students view the recorded lectures Evenings Days before exams Partial replay, finding something in particular Possibility to pause and study base or in-depth material Side conversations chat Social clustering students help out group assignments changes the burden of the teacher, instead of a lecturer he will become a tutor Complexity! A/V hardware Tools (usability) Time (isolation) Net-based LearningLessons Learned

26. To support presence and improvisation in static lectures Electronic whiteboard Sensor-switched cameras Multiple projectors New pedagogical approaches is needed! Problem-based learning Project-based learning Bring the conferencing tools into a face-to-face situation Best of two worlds! Mobile learning - find learning objects while moving around Recorded media Assignments Tutors Context dependent Net-based LearningLessons Learned

27. Table of Contents • Background • Publications • The mStar Environment • Robust Audio Transport • Light-weight Application Level Tunneling • Context-aware Applications • Privacy • Net-based Learning • Distributed Real-time Systems • Conclusions “ The best way to predict the future is to invent it. ” - Alan Kay, 1971

28. Equal access and control of media involved Interactivity!! No centrally stored shared state Loosely coupled autonomous client applications mStar IP Multicast Alipes Peer-to-peer SIP server topology IP Multicast Mobile IP Distributed Real-time SystemsSymmetry

29. A symmetric design means few or no dependencies on central servers No bottleneck No shared state Handle freely leaving and joining clients ’Best-effort’ local states No guarantee that all users see an identical view at all times mStar IP Multicast Reliable/Unreliable Loss recovery Repair schemes Redundancy Bandwidth adaptation and management Alipes Peer-to-peer Multiple positioning sources Redundancy Distributed Real-time SystemsRobustness

30. Clients need to adapt to heterogeneous networks Other types of heterogeneity needs also to be considered Terminals Users Context/Usage mStar Bandwidth adaptation Tunnels and gateways Recoding Mixing Scaling Switching Alipes Multiple positioning sources Distributed Real-time SystemsAdaptiveness

31. Scalability is imortant from multiple perspectives Heterogenous Terminals Networks Number of simultaneous Users Sessions mStar IP Multicast (?) Alipes Peer-to-peer SIP server topology IP Multicast Mobile IP Distributed Real-time SystemsScalability

32. Standard components used in different ways A/synchronously Stand alone Integrated Different scenarios Net-based learning Collaborative teamwork Special purpose clients are quickly developed mStar Agent architecture Control Bus Alipes Modular positioning techniques Platform (Multiple techniques) Single technique Modular map services Distributed Real-time SystemsReusability

33. Distributed real-time systems may be quite complex to use Not an add-on, it needs to be considered from scratch Tight-loop iterative or cyclic design that involves users! mStar Reuse of agents Stand-alone/Integrated Different contexts Alipes Allows for differentiation of privacy management Distributed Real-time SystemsUsability

34. Table of Contents • Background • Publications • The mStar Environment • Robust Audio Transport • Light-weight Application Level Tunneling • Context-aware Applications • Privacy • Net-based Learning • Distributed Real-time Systems • Conclusions “ The best way to predict the future is to invent it. ” - Alan Kay, 1971

35. ConclusionsContributions The principal contributions made by the author of this thesis are: • The design and implementation of the mStar environment for distributed real-time applications based on IP-multicast and agent components. • The design, implementation and evaluation of the audio agent component in the mStar environment. • The design and implementation of the audio transcoding and video-by-audio switching mechanisms in the mTunnel application. • The application and study of net-based learning using the mStar environment. • The design and implementation of the Alipes architecture for location-aware applications, including the privacy management.

36. ConclusionsNovelty The novelty of this work lies in the ways in which the mStar environment and the Alipes architecture support the creation of symmetric, robust, modular and interactive real-time applications for distributed teamwork and net-based learning scenarios using heterogeneous networks and terminals.

37. ConclusionsBetter-than-(only-)being-there • The original vision was that remote users would find using the tools ’better-than-being-there’. • Now people are becoming highly mobile and are bringing the tools to face-to-face meetings... • ...what if context-awareness, wearable computing and enhanced reality is added to the equation? • Could it even be ’better-than-only-being-there’?

38. Questions? That’s all.Thank you for your attention! Discussion by Angela Sasse, PhD. Questions by the examination board and the audience.

39. Seminars today 13.15-13.55 Angela Sasse (30+10 minutes) 13.55-14.20 Coffee 14.20-15.00 Björn Pehrson (30+10 minutes)

41. BackgroundUnicast • The sender duplicates the traffic once per additional receiver • This leads to a waste of bandwidth! • An alternative is to use replication servers in the network. • Points of failure • Bottle-necks The sender sends 3 identical copies of the data (one for each receiver).

42. BackgroundMulticast • Traffic is duplicated when necessary • a single copy of the data will travel on each connection • the sender does not need to keep track of the receivers • not relying on a central server • MBone The sender sends 1 copy which is duplicated by the network.

43. BackgroundUnicast versus Multicast

44. BackgroundPeer-to-peer • Clients talk directly between each other without any servers in between • This is typically used by loosely-coupled autonomous clients using ad-hoc networking

45. mStar Ubiquitous multicast Unicast/multicast Tunnels/gateways Layered media (video) Reduce complexity Net-based learning Pedagogy Technology Special client Alipes Study the privacy mechanisms Prototype applications Sound positioning Non-users Context Combining mStar and Alipes Mobile Learning (Mole) Future Work

46. It all began in 1995 VIC, video VAT, audio The Tunable Multicast Platform (/TMP) a software library for developing and integrating distributed applications The main choice of technology: Java and IP Multicast The original usage: Electronic Education(Net-based Learning) Electronic Meetings(Collaborative Teamwork) Electronic Corridor(Virtual Corridor) The mStar EnvironmentOverview

47. There is a need to control the use of applications within an organization avoiding flooding remote support and maintenance Agent-based design Control Bus, CB Resource Discovery finding the agents in a group identify points of control mManager Also, bandwidth management in clients based on policies The mStar EnvironmentManagement and Control of Distributed Applications

48. Context-aware ApplicationsResearch Issues • What criteria should be used for seamless selection between alternative positioning techniques? • Could alternative positioning techniques be used in conjunction and serve as additive techniques? • Could these be different peoples portable devicees? • How could privacy be kept intact? • What services and applications could best examplify future usage of such a platform?

49. Query Deny access Check on/off rule Off On Check party id versus ban rules Deny Banned! Ok Verify party id and key Unknown key Ok Check general criteria Matching denycriterion Matching grantcriterion Ok Check contracts Valid contract found Ok Grant Negotiate contract Grant access No contract New contract Context-aware ApplicationsThePrivacy Mechanism

50. Context-aware ApplicationsLocating a user using peer-to-peer technologies 1. Locate Carol’s SIPd Carol@ipaq.homeip.net Dave 3. Query Position GPS WLAN BT MPS WLANd 2. Lookup Carol BTd SIPd MPSd