Reliable, Scalable and Interoperable Internet Telephony

Reliable, Scalable and Interoperable Internet Telephony PhD thesis presentation by Kundan Singh Advisor: Henning Schulzrinne Computer Science Department, Columbia University, New York June 21, 2006

My research background/timeline Conference evaluation P2P VoIP using SIP SIP Failover/load sharing Multimedia collaboration CINEMA user interface Interactive voice response Enterprise VoIP infrastructure Mobile NAT Libsip++ (SIP library) SIP conferencing SIP-RTSP voice mail SIP-H.323 translator Reliability & scalability PhD@cs.columbia H.323 client gateway VoIP infrastructure MS@cs.columbia Work@ Motorola Undergrad @India 1997199920002001 2002 2003 2004 2005 2006

Outline of the presentation • Introduction • What is the problem? Why important? • My contributions • Server redundancy • Load sharing and failover in SIP telephony • Comparison of thread models for SIP server • Peer-to-peer (P2P) • SIP servers using external P2P network • Additionally, P2P maintenance using SIP • Enterprise IP telephony • Multi-platform collaboration using SIP • Scalable centralized conferencing architecture • Interworking between SIP/SDP and H.323 • Conclusion 36 slides

database (SCP)10 million customers 2 million lookups/hour database (SCP)for freephone, calling card, … local telephone switch(class 5 switch)10,000 customers 20,000 calls/hour signaling router (STP)1 million customers 1.5 million calls/hour signaling network (SS7) signaling router(STP) regional telephone switch(class 4 switch)100,000 customers 150,000 calls/hour Telephone reliability & scalability (PSTN: Public Switched Telephone Network) “bearer” network telephone switch(SSP) Switches strive for 99.999% availability. Lucent’s 5E-XC supports 4 million BHCA.

REGISTER INVITE INVITE DNS Internet telephony(SIP: Session Initiation Protocol) alice@yahoo.com yahoo.com example.com bob@example.com 129.1.2.3 192.1.2.4 DB Can SIP server provide carrier grade reliability and scalability using commodity hardware

What are the problems? • Can SIP server provide carrier-grade reliability and scalability using commodity hardware? • What affects the server performance? • How can we build a server-less self-organizing peer-to-peer VoIP network? • Can this be done in standards compliant way? • Can communication be extended to multi-platform collaboration using existing protocols? • How well multi-party conferencing scales? • How to interoperate between SIP and H.323?

My contributions • Server redundancy • Implemented failover using database replication • Two-stage architecture for SIP load sharing • Comparison of thread models for SIP server • Peer-to-peer (P2P) • SIP servers using external P2P network • Additionally, P2P maintenance using SIP • Enterprise IP telephony • Multi-platform collaboration using SIP • Scalable centralized conferencing architecture • Interworking between SIP/SDP and H.323 New architecture, New algorithm or approach, Implementation, Evaluation

Outline of the presentation • Introduction • What is the problem? Why important? • My contributions • Server redundancy • Load sharing and failover in SIP telephony • Comparison of thread models for SIP server • Peer-to-peer (P2P) • SIP servers using external P2P network • Additionally, P2P maintenance using SIP • Enterprise IP telephony • Multi-platform collaboration using SIP • Scalable centralized conferencing architecture • Interworking between SIP/SDP and H.323 • Conclusion

INVITE REGISTER Server redundancyThe problem: failure or overload

Implementation Using MySQL replication System reliability individual component reliability Call setup latency DNS TTL, time-to-repair, retry timeout User unavailability Most registers are refreshes Retry timeout, replication interval, register refresh interval DNS Caller TR Callee D2 P2 P1 D1 Tc Td Tc A Tr TR A Tc High availabilityImplementation and analysis Master/ slave Slave/ master D2 D1 P2 P1

REGISTER INVITE ScalabilityLoad sharing: redundant proxies and databases • REGISTER • Write to D1 & D2 • INVITE • Read from D1 or D2 • Database write/ synchronization traffic becomes bottleneck P1 D1 P2 D2 P3

ScalabilityLoad sharing: divide the user space • Proxy and database on the same host • Stateless proxy can become overloaded • Use many P1 D1 a-h P2 D2 i-q P3 D3 r-z

P1 P1 a-h D1 D1 P2 P2 i-q D2 D2 P3 P3 D2 r-z ScalabilityComparison of the two designs Low Scalability High Reliability High Scalability Low Reliability

Master Slave Master Slave Scalability (and reliability)Two stage architecture I stage II stage a*@example.com a.example.com _sip._udp SRV 0 0 a1.example.com SRV 1 0 a2.example.com a1 s1 a2 sip:bob@example.com s2 sip:bob@b.example.com b*@example.com b.example.com _sip._udp SRV 0 0 b1.example.com SRV 1 0 b2.example.com s3 b1 b2 ex example.com _sip._udp SRV 0 40 s1.example.com SRV 0 40 s2.example.com SRV 0 20 s3.example.com SRV 1 0 ex.backup.com Capacity = f(#stateless, #groups)

Load SharingPerformance result: calls/second • Using 3+3 servers gives carrier-grade performance (10 million busy hour call attempts) • Registration test: supports 10 million subscribers On commodity hardware: 3 GHz, Pentium 4, 1 GB memory Test with UDP, stateless, no DNS and no mempool

recv File read send accept Match transaction Modify response stateful Stateless proxy Response Found Update DB recvfrom parse Redirect/reject sendto REGISTER Match transaction Build response Request Lookup DB other Stateless proxy Proxy Modify Request DNS (Blocking) I/O Critical section (lock) Critical section (r/w lock) Server performanceWhat happens inside a server? What thread/event models possible? Web server SIP server • Pure event-based (one thread) • Thread-per-msg or transaction • Pool-thread per msg (sipd) • Two stage thread pool

Server performanceResults of my measurement • Event-based performs 30% better than existing thread-pool architecture on single-CPU • Two stage thread-pool architecture gives better performance on multi-CPU • 60% more on 4xPentium • Both Pentium and Sparc took 2 MHz of CPU cycles per call/s on single-CPU

C C P P S C C P P C P Problem with servers • Server-based • Cost: maintenance, configuration • Central points of failures, catastrophic failures • Controlled infrastructure (e.g., DNS) • Peer-to-peer • Robust: no central dependency • Self organizing, no configuration • Inherent scalability

We built: P2P-SIP • Unlike proprietary Skype architecture • Robust and efficient lookup using DHT • Interoperability • DHT algorithm uses SIP communication • Hybrid architecture • Lookup in SIP+P2P • Inter-domain P2P-SIP • Unlike file-sharing applications • Data storage, caching, delay, reliability • Data model and service model • Disadvantages • Lookup delay and security

SIP-using-P2P Replace SIP location service by a P2P protocol P2P-over-SIP Additionally, implement P2P using SIP messaging How to combine SIP + P2P? P2P network REGISTER INVITE alice FIND P2P-SIP overlay INSERT Alice 128.59.19.194 INVITE sip:alice@128.59.19.194 Alice 128.59.19.194

SIP-using-P2PLogical Operations • Contact management • put (user id, signed contact) • Key storage • User certificates and private configurations • Presence • put (subscribee id, signed encrypted subscriber id) • Composition needs service model • Offline message • put (recipient, signed encrypted message) • NAT and firewall traversal • STUN and TURN server discovery needs service model XML-based data format

SIP-using-P2PImplementation in SIPc with the help of Xiaotao Wu • OpenDHT • Using Data model • Identity protection • Certificate-based • SIP id == email • P2P for Calls, IM, presence, offline message and name lookup

P2P-over-SIPArchitecture and implementation • DHT (Chord) algorithm using SIP messages with query and update semantics of REGISTER • Has SIP registrar, proxy, user agent • Other: discovery, NAT traversal, failover • Adaptor: allows existing SIP devices to become P2P

P2P-over-SIPAnalysis: scalability • Computed message load as function of • Refresh rate (keep-alive, finger table, user registration), call arrival rate, churn (join, leave, failure), scale (number of peer nodes and users) • Number of nodes = f(individual node capacity) • Measured performance: 800 register/s. • Assuming a conservative 10 reqister/s capacity, and aggressive refresh and call rate of 1/min, it gives more than 16 million peers (super nodes) in the network.

P2P-over-SIPAnalysis: availability and call setup latency • To increase user availability: • Fast failure detection: increase keep-alive rate • Reduce unavailability: frequent registration refresh • Replicate: user and node registrations • Call setup latency: • Same as DHT lookup latency: O(log(N)) • Calls to known locations (“buddies”) is direct • Chord: 10000 nodes => 6 hops • At most a few seconds • User availability and retransmission timers

Not SIP-specific, hence no implementation overhead for non-VoIP but P2P applications Low transport and transaction overhead No P2P security burden on SIP No dependency on single DHT implementation Reuse SIP naming, routing, security, NAT/firewall traversal Easily reuse existing SIP components without change voicemail, conference Single DHT implementation Readily supports service model SIP-using-P2P vs P2P-over-SIP

Server-based vs peer-to-peer

SIP VXML My work Web server Internet telephony infrastructureCINEMA: Columbia InterNet Extensible Multimedia Architecture CINEMA servers Telephone switch rtspd: media server Local/long distance 1-212-5551212 sipconf: Conference server Quicktime RTSP PSTN RTSP clients Department PBX sipum: Unified messaging sipd: Proxy, redirect, Registrar server Internal Telephone Extn: 7040 713x SQL database cgi SIP/PSTN Gateway vxml Web based configuration Built many components in a complete system for enterprise IP telephony and multimedia collaboration H.323 siph323: SIP-H.323 translator NetMeeting

My other work • Communication to collaboration • Comprehensive, multi-platform collaboration using SIP • Unified messaging: The gaps among different media (audio, video, text), devices (PC, phone) and means of communications (Email, SIP, IM) disappear for messaging • Novel SIP/RTSP based voicemail and answering machine • SIP interface to VoiceXML browser • Centralized conferencing • Audio mixing, video forwarding, IM, shared web browsing, screen sharing, web-based configuration and control, floor control • Performance evaluation; cascaded server architecture • SIP-H.323 translation

Conference serverPerformance evaluation of audio mixer • On commodity PC • About 480 participants in a single conference with one active speaker • About 80 four-party conferences, with one speaker each • Both Pentium and Sparc took 6 MHz per participant

N.(N-1) participants Higher delay N2/4 participants Lower delay Conference serverCascaded architecture SIP REFER message is used to create cascading       I measured the CPU usage for two cascaded servers: supports about 1000 participants

Revisiting the problems Developed a two stage scalable and reliable SIP server architecture: linear scaling. Use event-based. • Can SIP server provide carrier-grade reliability and scalability using commodity hardware? • What affects the server performance? • How can we build a server-less self-organizing peer-to-peer VoIP network? • Can this be done in standards compliant way? • Can communication be extended to multi-platform collaboration using existing protocols? • How well multi-party conferencing scales? • How to interoperate between SIP and H.323? Developed P2P-SIP architecture: SIP-using-P2P and P2P-over-SIP Multi-platform collaboration using existing protocols and tools, unified messaging, centralized conferencing (cascaded), SIP-H.323 interworking.

Conclusions • Impact: • Commercialized by SIPquest (now FirstHand) and sold to many customers. • CINEMA was deployed in our department for a brief period of time. • Used in various other projects at IRT: NG911, firewall controller, presence scalability, TCP/TLS measurements,… • P2P-SIP is a “hot” topic in industry and IETF now – client desktop, hardware phone as well as server vendors are pursuing this. • SIP-H.323 requirements eventually became an RFC • Plan to open source SIPc for large scale deployment experience of P2P-SIP • Started working on a P2P-based self organizing servers for 3GPP at Bell Labs • “So what” (Implications): • Replacing PSTN – better features, quality and performance at lower cost and maintenance; zero cost VoIP using P2P-SIP • Distributed, multi-provider, component architecture for telephony and collaboration

My publicationsConference, workshop, technical report, magazine/journal • K. Singh and H. Schulzrinne, “Using an external DHT as a SIP location service", Columbia University Technical Report CUCS-007-06, NY, Feb’06. • K. Singh and H. Schulzrinne, “Peer-to-peer Internet telephony using SIP", NOSSDAV, Skamania, Washington, Jun 2005.. K. Singh and H. Schulzrinne, "Peer-to-peer Internet Telephony using SIP", New York Metro Area Networking Workshop, CUNY, NY, Sep 2004. K. Singh and H. Schulzrinne, "Peer-to-peer Internet Telephony using SIP", Columbia University Technical Report CUCS-044-04, NY, Oct 2004. • K. Singh and H. Schulzrinne, “Failover, load sharing and server architecture in SIP telephony”, Elsevier Computer Communication Journal. To appear. Aug 2006. “K. Singh and H. Schulzrinne, “Failover and load sharing in SIP telephony", SPECTS (Symposium on performance evaluation of computer and telecommunication systems). Philadelphia, PA, Jul 2005. K. Singh and H. Schulzrinne, "Failover and Load Sharing in SIP Telephony", Columbia University Technical Report CUCS-011-04, NY, May 2004. • H. Schulzrinne, K. Singh and X. Wu, "Programmable Conference Server", Columbia University Technical Report CUCS-040-04, NY, Oct 2004. • K. Singh, Xiaotao Wu, J. Lennox and H. Schulzrinne, "Comprehensive Multi-platform Collaboration", MMCN 2004 - SPIE Conference on Multimedia Computing and Networking, Santa Clara, CA, Jan 2004. K. Singh, Xiaotao Wu, J. Lennox and H. Schulzrinne, "Comprehensive Multi-platform Collaboration", Columbia University Technical Report CUCS-027-03, NY, Nov 2003. • M. Buddhikot, A. Hari, K. Singh and S. Miller, "MobileNAT: A new Technique for Mobility across Heterogeneous Address Spaces", ACM MONET journal, March 2005. M. Buddhikot, A. Hari, K. Singh and S. Miller, "MobileNAT: A new Technique for Mobility across Heterogeneous Address Spaces", WMASH 2003 - ACM International Workshop on Wireless Mobile Applications and Services on WLAN Hotspots, San Diego, CA, Sep 2003. • K. Singh, A. Nambi and H. Schulzrinne, "Integrating VoiceXML with SIP services", ICC 2003 - Global Services and Infrastructure for Next Generation Networks, Anchorage, Alaska, May 2003. K. Singh, A. Nambi and H. Schulzrinne, "Integrating VoiceXML with SIP services", Second New York Metro Area Networking Workshop, Columbia University, NY, Sep 2002. • K. Singh, W. Jiang, J. Lennox, S. Narayanan and H. Schulzrinne, "CINEMA: Columbia InterNet Extensible Multimedia Architecture", Columbia University Technical Report CUCS-011-02, NY, May 2002. W. Jiang, J. Lennox, H. Schulzrinne and K. Singh, "Towards Junking the PBX: Deploying IP Telephony", NOSSDAV 2001. W. Jiang, J. Lennox, S. Narayanan, H. Schulzrinne, K. Singh and X. Wu, "Integrating Internet Telephony Services", IEEE Internet Computing (magazine), May/June 2002 (Vol. 6, No. 3). • K. Singh, Gautam Nair and H. Schulzrinne, "Centralized Conferencing using SIP", 2nd IP-Telephony Workshop (IPTel'2001), April 2001. • K. Singh and H. Schulzrinne, "Unified Messaging using SIP and RTSP", IP Telecom Services Workshop 2000, Atlanta, Georgia, U.S.A, Sept 2000. K. Singh and H. Schulzrinne, "Unified Messaging using SIP and RTSP", Columbia University Technical Report CUCS-020-00, NY, Oct 2000. • K. Singh, H.Schulzrinne, "Interworking Between SIP/SDP and H.323", 1st IP-Telephony Workshop (IPTel'2000), April 2000. K. Singh and H. Schulzrinne, "Interworking Between SIP/SDP and H.323", Columbia University Technical Report CUCS-015-00, NY, May 2000.

Reliable, Scalable and Interoperable Internet Telephony