Design and Implementation of a WAP Gateway

1. Design and Implementation of a WAP Gateway A Master’s thesis by Lars Wirzenius CSCI 5939.02 – Independent study Fall 2002 Presented by: Obaidullah Khan

2. Chapter 1: Introduction • Catching up with science fiction: Star treck, Imperial Earth • Evolution of mobile phone, laptop, and then palmtops • Vision of mobile phone as web browsers • WAP technology enable mobile phone acts as hypertext browsers

3. What is covered in this master’s thesis • Kannel – an open source WAP gateway • Problems • Design • Technical issues • Project management issues • Kannel project started in June 1999, by Wapit Ltd

4. Chapter 2: Problems in Implementing Services for Mobile Phones

5. Technical issues • Low battery capacity • Small screen and keyboard • Limited bandwidth and high error rate

6. Business issues • Leverage on existing mobile phone networks • Must be adaptable to future mobile networks • Standardized for interoperability and mass market • Allows easy user interface • Allow billing

7. Pre-WAP solution • Normal voice calls • Short textual messages (SMS) • These two works fine in current networks and billing infrastructure exists. • Fetching normal HTTP pages on GSM phone – simply not feasible.

8. Chapter 3: The Wireless Application Protocol • History of WAP • The goals of WAP • Introduction to WML and WMLScript • Current status of WAP

9. WAP and WAP Forum • History • WAP Forum founded in June 1997 by Nokia, Ericsson, Motorola, and Unwired Planet • 1.0 specification in April 1998, but not implemented • 1.1 specification in July 1999, first to be implemented in phones

10. WAP and WAP Forum (cont’d) • Goals • Develop an open, freely licensed specification • The specification not tied to any network technology or any specific device

11. The WAP architecture Figure 3.1. WAP architecture

12. The WAP architecture (cont’d) • WAP Protocol • In binary and compressed form • Reduces the protocol overheads • WAP Gateway • Protocol translation between phone and server • Compresses WML pages to save bandwidth

13. WML and WMLScript • WML • A simple mark up language defined in XML • A page is a deck of cards • WMLScript • Based on ECMAScript or JavaScript • Makes WAP more dynamic • Provides libraries for controlling phone functionalities

14. WML pages at content server textual form WAP Gateway binary Phone Source code form WML and WMLScript (cont’d)

15. The WAP protocol stack • Consist of three layers • Wireless Datagram Protocol (WDP) • Wireless Transaction Protocol (WTP) • Wireless Session Protocol (WSP)

16. The WAP protocol stack (cont’d) Figure 3.2. A representative WAP session

17. The duties of a WAP gateway • Implement WAP protocol stack • User authentication and billing • Optimize WAP usage – keep cost down while utilizing bandwidth wisely

18. Chapter 4: The Kannel Open Source WAP Gateway This section covers the design and implementation of Kannel WAP gateway

19. Introduction and status of the project • Why it was started? • Wapit needed a gateway • When? • July 1999 at WAP Forum in San Francisco

20. Introduction and status of the project (cont’d) • Goals? • A technically good enough gateway for small operator / service providers • Serve thousand of concurrent users at reasonable price • Kannel to be a WAP gateway as well as an SMS gateway • Be scalable • Crash or failure of one node should not affect others

21. Gateway architecture • Discusses requirements • Division of gateway duties to processes - bearer, wap, sms boxes • Duties of each process

22. Gateway architecture (cont’d) • External interface of the gateway • SMS center, using various protocols • HTTP servers, to fetch WAP / SMS content • WAP phones, using WAP protocol stack Figure 4.1. External interfaces of Kannel

23. Gateway architecture (cont’d) • SMS protocol essentials • Client logs into SMS center • SMS center sends a message as it arrives, client should acknowledge it • Client send a request for message and SMS center acknowledge it • When client is done, it logs out • Only one connection to an account at a time

24. Gateway architecture (cont’d) • HTTP protocol essentials • Client connect to server • Client sends a request • Server respond and complete transaction • Multiple request over same connection are possible

25. Gateway architecture (cont’d) • Achieve maximum throughput • By Multitasking internally • By using internal queues • Reliability problem • What if Kannel crashes when there are request in internal queues? • Solution • Kannel should keep the lists on disc, but it does not do so

26. Gateway architecture (cont’d) • Division of duties to processes: the boxes • The bearerbox: Implements the WDP layer, and connect to SMS center • The smsbox: Implements the SMS gateway • The wapbox: Implements the WAP stack

27. Gateway architecture (cont’d) Figure 4.2. Boxes of Kannel • Only one bearerbox but multiple smsboxses & wapboxes • Each box is internally multithreaded • Static thread structure

28. Gateway architecture (cont’d) • Heartbeats • Each box sends “I am still alive” messages • The bearerbox keeps track of heartbeats of all boxes • Bearerbox closes connection, if heartbeat is not receives for long time • Heartbeat serves as load factor, which helps bearerbox in routing packets among boxes

29. The Bearer Box • What it does? • Receives UDP messages from phones • Send these messages to wapboxes • Receives reply from wapboxes • Sends the UDP message back to phones • Only UDP bearer for WDP is supported

30. The Bearer Box (cont’d) Figure 4.3. Bearerbox architecture

31. The Bearer Box (cont’d) • Bearerbox routes the UDP packets to wapboxes • Phones are allocated IP dynamically • All messages from same IP are sent to same wapbox • The bearerbox must know when a session or transaction finishes • Multiple queues within the bearerbox • Bearerbox balance the load among wapboxes using load factor

32. The WAP Box • What it does? • Wapbox reads messages from bearerbox, maintains internal states for each client, and makes HTTP requests for clients • Only WTP and WSP are implemented • WTLS is not implemented in this thesis

33. The WAP Box (cont’d) Figure 4.4. Wapbox thread structure

34. The WAP Box (cont’d) • WAP protocol stack layer is implemented in its thread • Communication between threads is via message queues • Each layer exposes only event data structures: simplified code, execution is faster • Static thread structure: starts at program startup and keeps running • Each layer extracts the event, process it, and sends other events to other layers • Locking is needed only when accessing the event queue for each layer

35. Implementation of protocol state machine • WTP and WSP ( connection mode) are implemented in term of state machine • Macro language ( the C processor) is used to describe state machine in source code

36. Efficient implementation of HTTP request • WAP gateway must implement HTTP request efficiently at high usage level • Implementing each HTTP request in a separate thread would be expensive • Implement HTTP as a static number of threads – for performance

37. Efficient implementation of HTTP request (cont’d) • Basic steps in HTTP request: • IP number look-up for the HTTP server: delay in DNS query • Open a TCP connection: may takes some time if network is congested or due to slow server • Write the request: if the request is large it may takes more than one TCP packet • Read the reply: it can takes several seconds • Close the connection: this should be fast

38. Efficient implementation of HTTP request (cont’d) • More than one thread requires scheduling, and context switching at wake-up • Having one thread eliminate context switching overhead • Unix system call select and poll are used, in single thread mode, to wait for input from several sources

39. Efficient implementation of HTTP request (cont’d) • What a thread does? • Wait until there is something to read • Read it into a connection specific buffer • If the buffer has a complete HTTP reply, return it to whoever made the request • Repeat forever

40. Making concurrent domain name lookups • DNS maps textual domain name into IP number • Problem • The gethostbyname function in C does not support concurrency • Solution • Implement DNS protocol within gateway Or • Run sub-processes to do gethostbyname calls • Single calls at a time, but multiple sub-processes provides concurrency • Security risk

41. Converting WML and WMLScript to binary • Used two simple application of complier theory for conversion to binary form • Compiler gets as input the WML source code and character set from HTTP header • Return a binary form of WML deck or error message • WML complier utilizes a string table to compress the output • WMLScript compiler parse the input, form a parse tree, optimize the tree, then generate byte code, and optimize again

42. Chapter 5: Experiences From this Implementation • Subjective evaluation • First installation in Sep. 1999, with some bugs to fix • Fixing the bugs slowed the development but improved the quality • Sufficient quality in order to succeed • Setup a ‘nag’ script successfully: compile remotely and report errors to developers • Automatic test cases, added later

43. Experiences From this Implementation (cont’d) • Effect of open source project • Ever changing group of developers • More varied testing: more compatible • More people doing debugging • Simple source code and program structure • More time spent on email communication

44. Chapter 6: Plans for the Future • New features • WAP security layer (WTLS) • WAP push technology • Using SMS messages as a bearer for WAP

45. Plans for the Future (cont’d) • Better quality • Improved security, reliability, and speed • Over feeding of data causes it to run out of memory and crash • Store SMS messages in persistent memory • Should allow push feature • Migrating job between SMS box or WAP box might improve performance