Efficient & Robust TCP Stream Normalization

1. Efficient & Robust TCP Stream Normalization Mythili Vutukuru Joint work with Hari Balakrishnan and Vern Paxson

2. Network Intrusion Detection Systems attack attack IDS Evasion Attacks

3. at tack Evasion by Fragmentation at tack IDS must parse data stream in order.

4. Evasion by Ambiguity: Inconsistent Retransmissions Low TTL at junk tack Inconsistent TCP segments at junk tack Tools to create such attacks exist. Makes IDS ineffective.

5. TCP Stream Normalizer • Removes ambiguity from network traffic. • Detects inconsistent TCP segments. at junk tack at junk

6. Existing Normalizer Designs • Buffer all unacknowledged data. • Buffer content hashes of unacknowledged data. at junk tack H(at) H(junk) H(tack)

7. Problems With Existing Designs • Too much memory. • Partial overlaps. • 20–30% of retransmits in 5 real-world traces. • Caused by repacketization. • State exhaustion attacks on the normalizer. at junk tack H(at) H(junk) ? H(tack)

8. Related Work • Evasion attacks. [Ptacek and Newsham, 1998] • Concept of normalization by storing all unacked data. [Malan et al., 2000] [Handley et al., 2001] • Buffering hashes of data (without handling partial overlaps). [Sugawara et al., 2005] [Commercial normalizers] • Reassembling data streams robustly. [Dharmapurikar and Paxson, 2005]. • Normalization for signature matching only. [Varghese et al., 2006]

9. RoboNorm • Detects inconsistent TCP retransmissions. • Memory Efficiency: stores only hashes. • Robustness: • Handles partial overlaps correctly. • Withstands memory exhaustion attacks.

10. RoboNorm: Basic Mechanism 101-200 1-100 101-200 51-150 Hash store H(1-100) H(101-200) Not equal Equal 51-100 H(101-200) 101-150 Partial retransmits held back ... Fitting segments 1-50 ... until fitting segments arrive. 151- 200 H(1-100)

11. Will segments be held forever? ACK:1 ACK:101 Hash store H(1-100) H(101-200) 51-100 101-150 Partial retransmits held back. 1-50 Sender TCP stalls!! 101-150 ~2 in thousand connections prone.

12. TCP Stalling: Fixing The Problem ACK:101 ACK:151 Hash store H(1-100) H(101-200) 101-150 Partial retransmits held back. ACK promotion 151-200 Necessary to check partial overlaps.

13. Connection Tuple Ptr Connection Table Putting it all together... DATA ACK RoboNorm Suitable for hardware implementation. Hash Store Held retransmits

14. Connection Tuple Ptr Connection Table Hash Store Held retransmits Memory Footprint Segment arrival rate & holding time Max # concurrent connections Max concurrent partial overlaps

15. Connection Tuple Ptr Connection Table Hash Store Held retransmits Memory Footprint – Trace Analysis 2.5 MB on a Gbps link. 10 X less than storing all content. Up to 66 X in practice. 2 MB 375 KB 100 KB

16. RoboNorm • Detects inconsistent TCP retransmissions. • Memory efficiency: stores only hashes. • Robustness: • Handles partial overlaps correctly. • Withstands memory exhaustion attacks.

17. Connection Tuple Ptr Connection Table Memory Exhaustion Hash Store Held retransmits Goal: should not consume RoboNorm memory “cheaply”. No new vulnerability.

18. SYN Flood. Keep conns idle. Unterminated conns. Init state on first data. Reclaim space for inactive conns. Timeout Bloom Filter. Connection Tuple Ptr 1 1 0 0 1 1 1 1 0 0 Connection Table SYNACK Bloom Filter Inactive Connection Bloom Filter Connection Table 48 bytes 1 byte. SYN ACK Save ~50% space with 5 min inactivity timer. Inactive conn

19. Exhaust connection table memory only by: Opening large number of conns. Actively sending data on all of them. Connection Tuple Ptr 1 1 0 0 1 1 1 1 0 0 Connection Table SYNACK Bloom Filter Inactive Connection Bloom Filter Connection Table No new vulnerability.

20. Pick conn with largest Coalesce hashes. Or evict connection if avg segment size large. Small segments. Segments stored for long time. avg segment holding time avg segment size Hash Store H(X) H(XY) H(Y)

21. Exhaust hash store memory only by: Sending data in large packets. Clearing packets fast. Hash Store Fill hash store only by consuming link bandwidth.

22. Hash Function • Hn(X) = (an.X + bn) mod pn pn = n-bit prime an in {1,...,pn-1} bn in {0,...,pn-1} • Hn(XY) = { Hn(Y) + 2k [Hn(X) – bn] } mod pn • n = 64 provides sufficient security.

23. Conclusion • TCP Stream Normalizer design that: • Is memory efficient. • Detects all inconsistent retransmissions. • Is robust to state-exhaustion attacks.

24. Backup Slides

25. Memory Footprint: Trace Analysis • Connection table • # peak conns = 34,000 • Bytes per conn = 48 • Hash store • Avg sgmt hold time = 200 ms • Sgmt arrival rate = (1 Gbps / 1000 B) • Bytes per hash = 15 • Held retransmits = 100 KB 2 MB 375 KB 100 KB ~2.5 MB

26. Eviction Policy of Hash Store • λi avg rate of segment arrival • δi avg hold time of segments. • si avg segment size. • λiδi hash memory consumed (cost) • λi si bandwidth consumed (benefit) • δi / si cost-to-benefit ratio. • Evict conn with largest δi / si