Performance Evaluation of L3 Transport Protocols for IEEE 802.21 (2 nd round)

1. Performance Evaluation of L3 Transport Protocols for IEEE 802.21 (2nd round) Richard Rouil, Nada Golmie, and David Griffith National Institute of Standards and Technology http://www.antd.nist.gov/seamlessandsecure.shtml

2. Outline • MIH transport issues investigated • Simulation scenarios • Performance evaluation results • UDP without MIH ACK (for reference) • UDP with MIH ACK • TCP • SCTP • Conclusions

3. MIH Transport Issues Investigated • Transport protocol type • UDP • TCP • PoS Location • RTT between the MIH nodes • Message parameters • Size • Rate • Network conditions • Link congestion • Priority queuing • Retransmission timeout • Fragmentation • Congestion & rate control

4. Mobility scenarios Varying the RTT between the MN and the MoS addresses all 4 scenarios identified in draft-xxx-mipshop-mstp-solution-00: • S1: Home Network MoS, the MN and the services are located in the home network. • S2: Visited Network MoS, MN is in the visited network and mobility services are also provided by the visited network. • S3: Roaming MoS, MN is in the visited network and all services are provided by the home network. • S4: Third party MoS, MN is in home or visited network and services are provided by a 3rd party network.

5. Network Topology PoS (not co-located) CN Variable link delay, packet loss, and bandwidth AP (co-located PoS) MIH message exchange to PoS MN Background traffic to CN for congestion

6. Simulation parameters

7. Transport protocol performance evaluation Performance metrics: • Transaction success rate (i.e. indication or response received) • Delay to complete a transaction • Network load • Overhead created by the MIH acknowledgement mechanisms and the transport layer • Transport throughput

8. Impact of RTT • Impact of the RTT on the transport of MIH messages: • Vary the RTT between 0.1s and 0.5s • For different packet loss (0%, 10%, 20%) • The following transport protocol are used: • TCP • UDP without MIH ACK • UDP with MIH ACK

9. UDP (no ACK) Performance

10. Transaction delay for indications • The transaction delay equals half the RTT regardless of the packet loss.

11. Transaction delay for requests • The transaction delay equals the RTT regardless of the packet loss.

12. Transaction success rate for indications • Success rate is equal to 1-p with p the packet loss in the network.

13. Transaction success rate for requests • Success rate is equal to (1-p)2 with p the packet loss in the network.

14. MIH overhead

15. UDP (with ACK) Performance

16. Transaction delay for indications • When there is no loss, the delays are identical to the UDP without retransmission. • Packet loss causes retransmission thus increasing the delays. • Delay=

17. Transaction delay for requests RTT < ReTx RTT > 2*ReTx ReTx < RTT < 2*ReTx • The results can be sectioned in three parts: • RTT < ReTx: In this case, retransmissions are only due to packet loss. As the packet loss increases, the delay increases. • ReTx < RTT < 2*ReTx: We observe that the transaction delays are converging. This reason is that the second retransmission is ineffective. • RTT > 2*ReTx: Regardless of packet loss, the MIH will timeout twice sending and both retransmissions are ineffective as the ACK would arrive after the timeout of the second retransmission (Transaction state is invalid, and response is ignored).

18. Closer look at retransmission Sender Receiver Sender Receiver Sender Receiver Timeout 1 Timeout 1 Timeout 1 Timeout 2 Timeout 2 Timeout 2 Transaction fails Transaction fails Transaction fails Timeout 3 Timeout 3 Timeout 3 RTT < timeout timeout < RTT < 2* timeout RTT > 2*timeout All retransmissions are effective Second retransmission is ineffective. Two packets lost on the same transaction causes it to fail. All retransmissions are ineffective. Any loss will cause the transaction to fail.

19. Transaction success rate for indications • Success rate equals 1-p3 with p the packet loss in the network.

20. Transaction success rate for requests RTT < ReTx RTT > 2*ReTx ReTx < RTT < 2*ReTx • The results can be sectioned in three parts: • RTT < ReTx: success rate equals (1-p3)2. • ReTx < RTT < 2*ReTx: success rate equals (1-p2)2. The second retransmission is ineffective. • RTT > 2*ReTx: success rate equals (1-p)2.. Both retransmissions are ineffective.

21. MIH overhead for indications RTT < ReTx RTT > 2*ReTx ReTx < RTT < 2*ReTx • The results can be sectioned in three parts: • RTT < ReTx: The retransmissions are due to packet loss only. Overhead = • ReTx < RTT < 2*ReTx: There is always one retransmission due to late ACK and some additional one due to packet loss. Overhead = 2(2-p) + p(2-p)2. • RTT > 2*ReTx: success rate equals (1-p)2. There are 2 retransmissions due to late ACK. As the packet loss increases, late responses are ignored and therefore no ACK is being generated. Overhead = 3 (2-p).

22. MIH overhead for requests RTT < ReTx RTT > 2*ReTx ReTx < RTT < 2*ReTx The overhead for request/response is following a similar pattern as the overhead for indications. Note: the number of MIH Packets sent is divided by both the number of MIH requests and MIH responses sent.

23. UPD with MIH ACK summary • The location of the PoS (i.e. RTT) impacts the performance of the MIH ACK mechanisms. • If the retransmission timeout is not adequately set, the retransmission may become ineffective: • ReTx < RTT < 2* ReTx: the results are identical to an MIH using one retransmission. • RTT > 2*ReTx: the results are similar to UDP without MIH ACK.

24. TCP Performance

25. Transaction delays for indications and requests/responses • TCP retransmits the packets using an exponential backoff leading to high retransmission delays.

26. Transaction success rate for indications and requests/responses • TCP’s reliability allows for a success rate of 100%.

27. MIH overhead • MIH does not retransmit messages when using a reliable transport, therefore there is not MIH overhead.

28. SCTP Performance

29. Network Topology Variable link delay, packet loss, and bandwidth PoS (not co-located) CN Interface 1 Interface 2 BS IEEE 802.16 AP IEEE 802.11 (co-located PoS) Added Access Network for multi-homing support MIH message exchange to PoS MN Background traffic to CN for congestion Second interface added for multi-homing support

30. Simulation parameters

31. Transaction delay for indications • When multi-homing is not available, a packet loss increases the backoff time for retransmission and generates high delays. The behavior is similar to TCP. • Using multi-homing, retransmissions are done on an alternate path which may provide lower loss. This is can be favorable if the primary path is using an interface ready for handover.

32. Transaction delay for requests

33. Transaction success rate for indications • SCTP provides reliable transport of MIH messages by retransmitting on the same interface or using an alternate when available.

34. Transaction success rate for requests

35. MIH overhead for indications • MIH does not retransmit messages when using a reliable transport, therefore there is not MIH overhead.

36. MIH overhead for requests

37. Congestion control • One of the backbone network link between the MN and PoS provides limited capacity (1Mb/s). • MN sends messages at a rate between 50kb/s and 1500kb/s for 30s (defined as offered load). • Packet sizes are 100 bytes and 500 bytes. • The following congestion control are used: • TCP • UDP with no rate limiter • UDP with Token bucket (200 bucket size, 200 token/s, 200 queue size) • UDP with Token bucket (200 bucket size, 50 token/s, 200 queue size)

38. Transaction delay for Indications • While there is no congestion (Offered load less than 1Mb/s), TCP uses all the available bandwidth. During congestion, TCP queues the packets and the delays to receive the indication increase. When the offered load increases, the size of the TCP segments increases to carry multiple MIH messages thus reducing the overhead. • When using UDP without rate limiter, the packets delays are low when there is no congestion. After an offered load of 600kb/s, delays appear due to the overhead generated by UDP (see throughput graph). • The rate limiter generates additional delays even for low offered loads. We observe that it acts later for larger MIH messages as it generates less packets for the same offered load. Furthermore, we can note that the indication is considered successful for indications even with high delays since the receiver is not aware of when it was sent. The sender on the other side may not receive the ACK on time, may retransmit the packet, and may consider the operation has failed.

39. Transaction delay for Request/Response • Similar observation for TCP and UDP without rate limiter. • When using rate limiter, the delays incurred due to the queuing causes the responses to be ignored. Therefore only the initial messages succeeded with low transaction delay.

40. Transaction success rate for Indications • TCP’s reliability allows for a success rate of 100%. • When UDP is used without rate control, an offered load of 600kb/s or more shows decreasing success rate. This is due to the packets being drop at the congested link in the network. • When control rate is used, the smaller the token rate, the lower the success rate. This is due to the queue limit of the token bucket implementation.

41. Transaction success rate for Request/Response • Observation similar to indications but lesser success rate for UDP since delayed responses are ignored.

42. MIH overhead for Indications • There is no MIH overhead created when using TCP since there is no retransmission at the MIH level. • When using UDP, there are 2 MIH packets corresponding to the Indication and the ACK for low offered load. • When no rate control is in place and the load is more than 600kb/s, increased in packet delay causes retransmissions, thus increasing the overhead. • The overhead is decreasing when rate control is in place due to packets being drop at the MIH level. The smaller the token rate, the faster the queue fills up and MIH message are dropped without trying to be sent.

43. MIH overhead for Request/Response • We observe an increase of overhead before it decreases again. This is happens when the sending rate of MIH packets caused by retransmissions is close to the optimum sending rate allowed by the token configuration. After this rate is passed, the packets are queued and dropped without trying to be sent.

44. Network layer load for Indications • The network load measures the traffic generated by the MN’s transport layer to send MIH packets. • TCP’s congestion control can be observed as the sending rate saturates close to 1Mb/s. • When UDP is not using rate control, all the retransmissions increase the network load. This leads to additional congestion in the network. In our scenario, it could saturate the wireless network if other nodes are present. • The rate control allows to limit the load though adequate values of token rate must be configured. We observe for example that in the case of packet size of 500 Bytes and token rate of 200, the rate control performs almost identically to TCP.

45. Network layer throughput for Indications • The network throughput is the data rate measured by the receiver side, in this case the PoS. • For TCP and UDP using rate control, the throughput measured is identical to the load since the load is lesser than the available bandwidth on the congested link. • When no rate control is in place, the throughput saturates.

46. Summary of results on Congestion Control • Congestion control is needed congestion spreads in the network due to a significant number of retransmissions. • TCP’s embedded congestion control and packing provides less overhead for low congestion. When the link saturates, the exponential backoff mechanism and Head of Line situation leads to very high delays. • The token bucket parameters must be adjusted to consider the congestion level and the average MIH packet size.