Lill Kristiansen, Prof. Dr. Scient Dept. of Telematics, NTNU, Norway

1. ENME: An ENriched MEdia application utilizing context for session mobility;technical and human issues Lill Kristiansen, Prof. Dr. ScientDept. of Telematics, NTNU, Norway lillk@item.ntnu.nowww.item.ntnu.no/~lillk Mainly based on Master Thesis of Egil C. Østhus

2. Content • Scenarios • Background material • From telecom and pervasive computing • Overview of the ENME service implementation • Discussion • Ideas and issues for the health care domain

3. Scenario 1: Business trip

4. Scenario 1: Business trip • Peter is on a business trip, and is currently at the airport express train station. Peter carries a laptop (packed away), as well as a mobile phone (ready for use). • His boss Carl tries to contact Peter to discuss a presentation and wants to use full multimedia for this discussion. I.e., he proposes: voice, (video) and shared data application. • At the train station there are some ’multimedia booths’. • When called, they start with voice telephony (due to capability limitations). • After 1 min. Peter finds and enters one of these booths. • The system realizes the new capabilities and propose to Peter to use the new screen for both video and shared application. • The session moves to the new device and new media flows are added. The old mediaflows are ended.

5. Scenario 2: In a hospital

6. Examples of possible terminals (devices / MDAs) in the hospital setting

7. Scenario 2: In the hospital • Eve is a doctor and is discussing a patient with a nurse who got worried about the situation. They use voice, as well as video showing the patients hearthbeat (ECG) • The multimedia conversation goes to an MDA (variant of PDA: Medical Digital Assistant) • Eve enters a multimedia booth • The system propose to move the video to a bigger screen • Variant: Eve passes a public bigger screen. • Shall the video be moved?

8. Focus in this paper • The border between human and technical issues • NOT: The details of context gathering • NOT: The details of the SIP protocol extensions • NOT: The GUI details • BUT: • How human issues add requirements to the technical solution • How technical possibilities may fit or not fit in with human and organizational requirements

9. Content • Scenarioes • Background material • From telecom and pervasive computing • Overview of the ENME service implementation • Discussion • Ideas and issues for the health care domain

10. SIP: Session Initiation Protocol • From IETF (Internet Engineering Task Force) • Also used by traditional telecom standardization bodies • 3GPP (with ETSI for UMTS) for use in IMS • IPMultimedia Subsystem in UMTS • 3GPP2 the US variant • Baseline SIP handles call set up • SIP Refer handles ’call transfer’ (used by us) • SIP allows for extensions (used by us) • SIP for presence • SIMPLE (not used here, but relevant for pervasive comp.)

11. Telecom and computer science • ’Computing’ has less focus on realtime than telecom • Computing has more focus on rich information sources • But only recently was the PDA capable of acting as a phone • In the telecom domain one (always) assumes the possibility to call another person in another operator domain, and using another terminal vendor • Without the need for installing common special software • This is the reason why we wanted to experiment with common SIP • This assumption is not always present in computer science / pervasive computing

12. Separate Services Servers Content Content Communication Control Access Gateways Access Gateways Backbone Network PLMN PSTN/ISDN Data/IP Networks CATV Access Access Access Users Separate users Recent changes in telecom From several monolithic systems (PSTN, GSM, IP,..) To open layered system components (see e.g. IMS)

13. Advantages of a layered approach • Using the same infrastructure for many applications and many services • Same core infrastructure for data (’chat’/IM) and phone, namely IMS (SIP Proxies etc.) • Same application may be used for non-real-time media and real-time media • This may however not always be wanted (more later) • Typical example: • Location and other context informaton may be used by many applications

14. Relevant work from pervasive computing • Some issues in pervasive computing [1]: • Effective use of smart spaces • Localized Scalability • Invisibility • Weiser’s ideal that the computer disappears from the user’s consiousness • Masking Uneven Conditioning • The deployment of pervasive computing into the environment is depended on non-technical factors such as organizational structure, business models and economics. • Our focus here is on the last two bullets

15. Definition of context • The definition proposed by Dey [2]: • “Context is any information that can be used to characterize the situation of an entity. An entity is a person, or object that is considered relevant to the interaction between a user and an application, including the user and application themselves.”

16. Context stack proposed by Li[4]: We base our work on such a model as this allows for the following the same context info may be used in many applications The same sensors may be used in many applications Many sensors may be used in the same application as well Basically the same advantages as for the layeres approach shown before Fusion Conversion Measurements Sensor Managing context information

17. Context model • Henricksen et al [5] introduce an object-oriented approach. They suggest dividing the context information into persons, devices and channels. Person Device Channel

18. Content • Scenarioes • Background material • From telecom and pervasive computing • Overview of the ENME service implementation • Discussion • Ideas and issues for the health care domain

19. Requirements of ENME • We base our implementation on SIP • We separate between the ENME subscriber and other users (i.e., the communicating partner(s)) • Users shall be able to communicate with the subscriber of the ENME service using his normal SIP client and well established extensions of SIP • The ENME subscriber shall use plain SIP or well established SIP extensions to the extent possible • The service shall be meaningfull to the subscribers and other users, and make sense in their daily life/work

20. The service logic • More formalized here • Assume there is an ongoing communication session between two or more users, the service maintain knowledge about users preferences when it comes to service description, e.g. voice only or video. • If a user moves to a new location, check for available devices. If there are devices that are better than the ones in use, and match user’s preferences, proceed to Step 3 . Otherwise goto to Step 1. • Request the user if he wants to move the session to a new device. Proceed to Step 4 if positive answer, goto to Step 1 otherwise. • Move (parts of) the session to new device. Then goto step 1 • Still this description allows for much flexibility in the design

21. Design of EMNE • We use the following entities in addition to the familiar SIP entities: • Context handler • Receiving info from the sensors • Talking SIP to the SIP-system (via SIP-proxies) • Context data • Data repository • Our implementation uses RFID readers as location sensors • Our implementation: • the user and his PDA is located on a model railroad system • RFID registeres 2 zones • (The context model is more general)

22. SIP and our context model • We base our work partly on [5], as well as on terminology from SIP Rough comparision with [5]: User - Person Session – Channel Device - Device

23. Our context model • User: A person with access to use the offered application • Zone: A geographical area, e.g. a room or inside a booth. • Device • A terminal, both public and nonpublic available. A device is described by its capabilities (ability to support video and voice, screen resolution, speakers, related codecs, etc). Sub-entities: High/low capability device. (In the implementation only available codecs are looked at.) • Session • A session is a communication between two or more parties. A session may consist of zero or more dialogs. • A dialog will comprise media transfer. • A zero-dialog session consists of only signaling, but no media transfer. This is according to SIP [7],

24. Voice Design continued

25. Voice Voice Video Design continued !

26. Content • Scenarioes • Background material • From telecom and pervasive computing • Overview of the ENME service implementation • Discussion • Ideas and issues for the health care domain

27. Voice Voice Voice Video ? Discussion of simple alternative • By sending the SIP REFER message to the A terminal the technical solution would be much simpler • But: This is meaningless, since A may never have heard of ENME, the question should go to B (the subscriber)

28. Virtual terminal • An other type of solution could use some type of virtual terminal • The new media flows goes via the handheld device • Terminal must forward it, (though not display it) • Timing/synchronization issues?

29. Human factors • What about the user –terminal relation? • Good to get a bigger screen? • Or confusing to relate to different layouts on different screens? • Medium size for less confusion? • Does one size fit all? • In our implementation the control part of the session follows the media flows to the new terminal. • Maybe instead the control part should at all times stay on the handheld device • Needs to look into details in SIP to see if this is possible

30. Media types (media richness theory) • User choose (wanted) media types when starting the service, • Sometimes quite consious choice at initiation • ….but may only get a subset • Continue at all if less media? • Probably yes, if wanted media can be established ’soon’ • It might be disturbing in the middle of the conversation to change media types • Different media types have different properties, and one might think that the ’communication strategy’ migh have changed due to less media. However this change in communication strategy in not explicit, • we cannot assume that the system can detect this easily/automatically. • Hence if the context handler keeps the wanted media types, it is not in accordance with the new communication strategy of the user. Adding media later is of no (good) use.

31. Content • Scenarioes • Background material • From telecom and pervasive computing • Overview of the ENME service implementation • Discussion • Ideas and issues for the health care domain

32. Issues in a hospital setting • Security: • many issues in general, even more in hospital setting(not looked into) • Organizational issues! • Shall the patient be able to call the nurse? (replacing the current ’button’ near the bed) • Shall a nurse be able to call a doctor? • Shall healthcare workers be able to see each other’s locations and status information? • Call eachother, or write messages in the patient’s journal? • Call a ’function’ /arbitrarily member in a group or call an individual human?

33. Related work in hospital setting • Mexico: IM (Instant Messaging) and message box in hospital setting • but not for real time voice communication • Others: A lot for PDA/MDA and patient journal research • Less with real time communication (’telephony’)

34. Questions? Contact: Lill Kristiansen, Prof. Dr. ScientDept. of Telematics, NTNU, Norway lillk@item.ntnu.nowww.item.ntnu.no/~lillk

35. M. Forthuns work H2004 • Using ServiceFrame for user context has been done by Forthun, • Using ServiceFrame also in this case combined with SIP might be looked into • Group communication in health care, implemented in ServiceFrame • Full information and report can be found at: http://www.item.ntnu.no/lab/nettint1/activities/prosjekter/host2003/ForthunPresentasjon.pdf • 2 next foils are based on this student’s presentation

36. SCENARIO 1 FROM ST.OLAV – HELP WITH PATIENT Keeping track of Context inform-ation Mary Dan Location: Room 333 Presence: Busy with patient GroupSession: 1 Location: Room 337 Presence: Free GroupSession: 0 Automatic handling /redirecting of the message,based on type and context Stroke Unit Stroke Unit Primary group - G1 Primary group - G1 Able to help with patient in Room 333? Help Patient Yes No Interface on the terminals

37. SCENARIO 2 FROM ST. OLAV – EMERGENCY Eve Benny Dan Location: Room 310 Presence: Emergency GroupSession: 1 Anna Location: Room 310 Presence: Emergency GroupSession: 1 Location: Room 333 Presence: Emergency GroupSession: 1 Location: Room 310 Presence: Emergency GroupSession: 1 Primary group - G1 Primary group - G1 Stroke Unit Stroke Unit EMERGENCY IN ROOM 333 Emergency Interface on the terminals The patient’s doctor Emergency team