Interactive Communications – voice, video, IM and presence-based services

1. Interactive Communications – voice, video, IM and presence-based services Henning Schulzrinne (with Jonathan Lennox, Mathew Mintz-Habib, Anshuman Rawat, Ron Shacham, Xiaotao Wu) Columbia University Lucent Bell Labs November 15, 2004 VoIP

2. Overview • The state of VoIP • Context & location-based services • Service creation in end systems • Service and session mobility • Emergency calling • Spam, spit and other unsavory things • What is identity? • What’s left to do? VoIP

3. (Early) Adulthood • “fully developed and mature” • Not quite yet, but no longer a teenager • probably need another 6 years to be grown up… • Responsibilities: • Dealing with elderly relatives  POTS • Financial issues  payments, RADIUS • Family emergencies  911 VoIP

4. Evolution of VoIP “how can I make it stop ringing?” long-distance calling, ca. 1930 “does it do call transfer?” going beyond the black phone “amazing – the phone rings” catching up with the digital PBX 1996-2000 2000-2003 2004- VoIP

5. The state of SIP VoIP

6. SIP is PBX/Centrex ready boss/admin features centrex-style features attendant features VoIP from Rohan Mahy’s VON Fall 2003 talk

7. A constellation of SIP RFCs Non-adjacent (3327) Symmetric resp. (3581) Service route (3608) User agent caps (3840) Caller prefs (3841) Request routing Resource mgt. (3312) Reliable prov. (3262) INFO (2976) UPDATE (3311) Reason (3326) SIP (3261) DNS for SIP (3263) Events (3265) REFER (3515) ISUP (3204) sipfrag (3240) Mostly PSTN Content types Core Digest AKA (3310) Privacy (3323) P-Asserted (3325) Agreement (3329) Media auth. (3313) AES (3853) DHCP (3361) DHCPv6 (3319) Configuration VoIP Security & privacy

8. SIP, SIPPING & SIMPLE –00 drafts includes draft-ietf-*-00 and draft-personal-*-00 VoIP

9. Protocol interoperability problems • Three core interoperability problems: • syntactic robustness • “You mean you could have a space there?” • often occurs when testing only against common reference implementations • need “stress test” (also for buffer overflows) • implementation by protocol example • limiting assumptions (e.g., user name format) • see “SIP Robustness Testing for Large-Scale Use”, First International Workshop on Software Quality (SOQA) • semantic assumptions • “I didn’t expect this error” • mutually incompatible extensions • expect extension to make something work VoIP

10. Protocol interoperability • Proprietary protocol • Example: Skype • Can only reverse-engineer  only backwards-compatibility problems • incentive to force upgrades (see Microsoft Word) • quicker evolution, but limited product selection • less Metcalfe’s law value • Open standard, but dominant vendor • Example: H.323, Asterix • doesn’t matter what the standard says • NetMeeting and H.323  test with Microsoft implementation • limits feature evolution to dominant vendor speed • Open standard, multiple large and many small vendors • Example: SIP • hardware vs. software, UA vs. proxy, phones vs. gateways • interoperability problems until product maturity • harder to test internally against all (competing) products • better long-term outcome, but slower VoIP

11. Context-aware and location-based services VoIP

12. Context • context = “the interrelated conditions in which something exists or occurs” • anything known about the participants in the (potential) communication relationship • both at caller and callee VoIP

13. “Legacy” IM & presence systems  SIP-based systems • centralized systems (single name space) • federated systems, similar to email • mostly instant text messages • media-agnostic – transmit any media object • separate from session-based services (VoIP, video conferencing) • integrated: • use IM as part of media sessions • use presence to facilitate session setup • limited presence status, mostly manually set • rich presence, with time information • imported from sensors, calendars, backend systems, … • proprietary systems (AOL, Yahoo!, MSN, ICQ, …) • standards-based systems VoIP

14. Presence = special case of event notification “user Alice is available for communication” Human users: multiple contacts per presentity device (cell, PDA, phone, …) service (“audio”) activities, current and planned surroundings (noise, privacy, vehicle, …) contact information composing (typing, recording audio/video IM, …) Multimedia systems: REFER (call transfer) message waiting indication conference floor control conference membership push-to-talk system configuration General events: emergency alert (“reverse 911”) industrial sensors (“boiler pressure too high”) business events (“more than 20 people waiting for service”) Presence and event notification VoIP

15. IETF efforts • SIP, SIPPING and SIMPLE working groups • but also XCON (conferencing) • Define SIP methods PUBLISH, SUBSCRIBE, NOTIFY • GEOPRIV: • geospatial privacy • location determination via DHCP • information delivery via SIP, HTTP, … • privacy policies • SIMPLE: • architecture for events and presence • configuration (XCAP) • session-oriented IM (↔ page mode) • filtering, rate limiting and authorization VoIP

16. Presence data model “calendar” “cell” “manual” person (views) alice@example.com audio, video, text r42@example.com video services devices VoIP

17. GEOPRIV and SIMPLE architectures rule maker DHCP XCAP (rules) target location server location recipient notification interface publication interface GEOPRIV SUBSCRIBE presentity presence agent watcher SIP presence PUBLISH NOTIFY caller callee SIP call INVITE INVITE VoIP

18. RPID: rich presence • Provide watchers with better information about the what, where, how of presentities • facilitate appropriate communications: • “wait until end of meeting” • “use text messaging instead of phone call” • “make quick call before flight takes off” • designed to be derivable from calendar information • or provided by sensors in the environment • allow filtering by “sphere” – the parts of our life • don’t show recreation details to colleagues VoIP

19. Classification: service class device, in-person, service, presentity class for labeling sphere “work”, “home”, … relationship “family”, “associate”, “assistant”, “supervisor” User activities: activities “on-the-phone”, “away”, “appointment”, … user-input last usage of device Mood Surroundings: time offset place-type “home”, “office”, “industrial”, … privacy “public”, “private” RPID: rich presence VoIP

20. CIPID: Contact Information • More long-term identification of contacts • Elements: • card – contact Information • home page • icon – to represent user • map – pointer to map for user • sound – presentity is available VoIP

21. Presence is about here & now but often only have (recent) past – e.g., calendar or future “will be traveling in two hours” “will be back shortly” allows watcher to plan communication loose synchronization of calendars <tuple id="7c8dqui"> <contact> sip:bob@example.com </contact> <status> <basic>open</basic> </status> <fs:timed-status from="2003-08-15T10:20:00.000-05:00“ until="2003-08-22T19:30:00.000-05:00"> <basic>closed</basic> </fs:timed-status> </tuple> <note>I'll be in Tokyo next week</note> Timed Status VoIP

22. Location-based services • Finding services based on location • physical services (stores, restaurants, ATMs, …) • electronic services (media I/O, printer, display, …) • not covered here • Using location to improve (network) services • communication • incoming communications changes based on where I am • proximity triggers communications • configuration • devices in room adapt to their current users • awareness • others are (selectively) made aware of my location • security • proximity grants temporary access to local resources VoIP

23. Location-based SIP services • Location-aware inbound routing • do not forward call if time at callee location is [11 pm, 8 am] • only forward time-for-lunch if destination is on campus • do not ring phone if I’m in a theater • outbound call routing • contact nearest emergency call center • send delivery@pizza.com to nearest branch • location-based events • subscribe to locations, not people • Alice has entered the meeting room • subscriber may be device in room  our lab stereo changes CDs for each person that enters the room VoIP

24. Location detection VoIP

25. DHCP for locations • modified dhcpd (ISC) to generate location information • use MAC address backtracing to get location information 8:0:20:ab:d5:d DHCP server CDP + SNMP 8:0:20:ab:d5:d 458/17 DHCP answer: sta=DC loc=Rm815 lat=38.89868 long=77.03723 458/17  Rm. 815 458/18  Rm. 816 VoIP

26. a@foo.com: 128.59.16.1 SIP URIs for locations location beacon • Identify confined locations by a SIP URI, e.g., sip:rm815@cs.columbia.edu • Register all users or devices in room • Allows geographic anycast: reach any party in the room sip:rm815 Contact: bob Contact: alice Room 815 VoIP

27. Describe a location • Geospatial coordinates • e.g., measured by GPS • expressed in PIDF-LO as GML: <gml:coordinates>40:48:34N 73:57:39W</gml:coordinates> • Civic address • National Emergency Number Association (NENA) • <location country="USA" A1="NY" A3="New York" A6="West 120th Street" HNO="500" LOC="450"> • Location attributes • place type, noise, media privacy, number of people, moving, … VoIP

28. Privacy • All presence data, particularly location, is highly sensitive • Basic location object (PIDF-LO) describes • distribution (binary) • retention duration • Policy rules for more detailed access control • who can subscribe to my presence • who can see what when <tuple id="sg89ae"> <status> <gp:geopriv> <gp:location-info> <gml:location> <gml:Point gml:id="point1“ srsName="epsg:4326"> <gml:coordinates>37:46:30N 122:25:10W </gml:coordinates> </gml:Point> </gml:location> </gp:location-info> <gp:usage-rules> <gp:retransmission-allowed>no </gp:retransmission-allowed> <gp:retention-expiry>2003-06-23T04:57:29Z </gp:retention-expiry> </gp:usage-rules> </gp:geopriv> </status> <timestamp>2003-06-22T20:57:29Z</timestamp> </tuple> VoIP

29. Privacy policy relationships common policy geopriv-specific presence-specific future RPID CIPID VoIP

30. Conditions identity, sphere, validity time of day current location identity as <uri> or <domain> + <except> Actions watcher confirmation Transformations include information reduced accuracy User gets maximum of permissions across all matching rules Extendable to new presence data rich presence biological sensors mood sensors Privacy rules VoIP

31. Location-based service language NOTIFY true false action alert IM alert incoming proximity message outgoing log conditions occupancy actions events notify call message time transfer subscription join VoIP

32. Program location-based services VoIP

33. VoIP

34. Tracking VoIP

35. Overview of SIPc functions emergency configuration handling multimedia session setup presence network multicasted session information appliance control Email clients Web browsers Instant message Network appliance control Third party call control Real time streaming audio video Emergency handling SIP Multimedia call control SIP CGI engine SAP white board location sensors Location sensing Floor control SIP for presence LESS/CPL engine Service Location Detection (SLP) desktop sharing Some IETF drafts CPL, SIP 3PCC, SIP Device Control GEOPRIV location format, SIP for IM SIP: RFC 3261 SAP: RFC 2974 RTSP: RFC 2326 SDP: RFC 2327 RTP: RFC 1889 SLP: RFC 2608 VoIP SIP Event Notification: RFC 3265

36. Evolving Conferencing floor control Device control Presence notification ir/x10 xcon Location tracking SIP event notification Service detection Message waiting indication Location sensing SIP SLP Voicemail handling Call Emergency handling RTP SDP MapLynx RTSP Instant messaging Session broadcasting SAP VoIP

37. Internet2 WG PIC trial SUBSCRIBE to my location NOTIFY myself and others’ locations PUBLISH presence status VoIP

38. Ubiquitous computing SA DA SA turn on projector Service Location Query send audio Resource to SIP Network Discovery UA2 Appliance (SLP UA) Control audio and Resource Script video Control engine streams (3pcc) SIP UA1 SIP VoIP call UA2

39. SIP Server Room 123 Hotel Front desk SLP Server Service scenario Media streams Bluetooth Authenticate Call Use Location Resources Visitor Call Info Resource Info Home domain Register SIP and AAA server VoIP

40. Example: user-adaptive device configuration “all devices that are in the building” RFC 3082? 802.11 signal strength  location SLP device controller HTTP PA REGISTER To: 815cepsr Contact: alice@cs tftp SUBSCRIBE to each room • discover room URI • REGISTER as contact for room URI SIP room 815 SUBSCRIBE to configuration for users currently in rooms VoIP

41. Service creation VoIP

42. Service creation • Tailor a shared infrastructure to individual users • traditionally, only vendors (and sometimes carriers) • learn from web models VoIP

43. Automating media interaction – service examples • If call from my boss, turn off the stereo  call handling with device control • As soon as Tom is online, call him  call handling with presence information • Vibrate instead of ring when I am in movie theatre  call handling with location information • At 9:00AM on 09/01/2005, find the multicast session titled “ABC keynote” and invite all the group members to watch  call handling with session information • When incoming call is rejected, send email to the callee  call handling with email VoIP

44. LESS: simplicity • Generality (few and simple concepts) • Uniformity (few and simple rules) • Trigger rule • Switch rule • Action rule • Modifier rule • Familiarity (easy for user to understand) • Analyzability (simple to analyze) modifiers trigger switches actions VoIP

45. LESS: Decision tree • No loops • Limited variables • Not necessarily Turing-complete VoIP

46. LESS: Safety • Type safety • Strong typing in XML schema • Static type checking • Control flow safety • No loop and recursion • One trigger appear only once, no feature interaction for a defined script • Memory access • No direct memory access • LESS engine safety • Ensure safe resource usage • Easy safety checking • Any valid LESS scripts can be converted into graphical representation of decision trees. VoIP

47. LESS snapshot incoming call <less> <incoming> <address-switch> <address is=“sip:myboss@abc.com"> <device:turnoff device=“sip:stereo_room1@abc.com”/> <media media=“audio”> <accept/> </media> </address> </address-switch> </incoming> </less> If the call from my boss Turn off the stereo Accept the call with only audio trigger, switch, modifier, action VoIP

48. SIP user agent SIP Device agent Presence agent Basic user agent presence Generic Media UI Event x10 vcr LESS packages • Use packages to group elements im email web calendar conference session location VoIP

49. When Tom is online, … <less> <EVENT:notification> <address-switch> <address is="sip:tom@example.com"> <EVENT:event-switch> <EVENT:event is="open"> <location url="sip:tom@example.com"> <IM:im message="Hi, Tom"/> </location> </EVENT:event> </EVENT:event-switch> ……… </less> VoIP

50. When I am in a movie theatre, … <less> <incoming> <location-switch> <location placetype=“quiet”> <alert sound=“none” vibrate=“yes”/> </location> </location-switch> </incoming> </less> VoIP