Automated Worm Fingerprinting

1. Automated Worm Fingerprinting Sumeet Singh, Cristian Estan, George Varghese, and Stefan Savage

2. Introduction • Problem: how to react quickly to worms? • CodeRed 2001 • Infected ~360,000 hosts within 11 hours • Sapphire/Slammer (376 bytes) 2002 • Infected ~75,000 hosts within 10 minutes

3. Existing Approaches • Detection • Ad hoc intrusion detection • Characterization • Manual signature extraction • Isolates and decompiles a new worm • Look for and test unique signatures • Can take hours or days

4. Existing Approaches • Containment • Updates to anti-virus and network filtering products

5. Earlybird • Automatically detect and contain new worms • Two observations • Some portion of the content in existing worms is invariant • Rare to see the same string recurring from many sources to many destinations

6. Earlybird • Automatically extract the signature of all known worms • Also Blaster, MyDoom, and Kibuv.B hours or days before any public signatures were distributed • Few false positives

7. Background and Related Work • Almost all IPs were scanned by Slammer < 10 minutes • Limited only by bandwidth constraints

8. The SQL Slammer Worm:30 Minutes After “Release” - Infections doubled every 8.5 seconds - Spread 100X faster than Code Red - At peak, scanned 55 million hosts per second.

9. Network Effects Of The SQL Slammer Worm • At the height of infections • Several ISPs noted significant bandwidth consumption at peering points • Average packet loss approached 20% • South Korea lost almost all Internet service for period of time • Financial ATMs were affected • Some airline ticketing systems overwhelmed

10. Signature-Based Methods • Pretty effective if signatures can be generated quickly • For CodeRed, 60 minutes • For Slammer, 1 – 5 minutes

11. Worm Detection • Three classes of methods • Scan detection • Honeypots • Behavioral techniques

12. Scan Detection • Look for unusual frequency and distribution of address scanning • Limitations • Not suited to worms that spread in a non-random fashion (i.e. emails) • Detects infected sites • Does not produce a signature

13. Honeypots • Monitored idle hosts with untreated vulnerabilities • Used to isolate worms • Limitations • Manual extraction of signatures • Depend on quick infections

14. Behavioral Detection • Looks for unusual system call patterns • Sending a packet from the same buffer containing a received packet • Can detect slow moving worms • Limitations • Needs application-specific knowledge • Cannot infer a large-scale outbreak

15. Characterization • Process of analyzing and identifying a new worm • Current approaches • Use a priori vulnerability signatures • Automated signature extraction

16. Vulnerability Signatures • Example • Slammer Worm • UDP traffic on port 1434 that is longer than 100 bytes (buffer overflow) • Can be deployed before the outbreak • Can only be applied to well-known vulnerabilities

17. Some Automated Signature Extraction Techniques • Allows viruses to infect decoy progs • Extracts the modified regions of the decoy • Uses heuristics to identify invariant code strings across infected instances • Limitation • Assumes the presence of a virus in a controlled environment

18. Some Automated Signature Extraction Techniques • Honeycomb • Finds longest common subsequences among sets of strings found in messages • Autograph • Uses network-level data to infer worm signatures • Limitations • Scale and full distributed deployments

19. Containment • Mechanism used to deter the spread of an active worm • Host quarantine • Via IP ACLs on routers or firewalls • String-matching • Connection throttling • On all outgoing connections

20. Defining Worm Behavior • Content invariance • Portions of a worm are invariant (e.g. the decryption routine) • Content prevalence • Appears frequently on the network • Address dispersion • Distribution of destination addresses more uniform to spread fast

21. Finding Worm Signatures • Traffic pattern is sufficient for detecting worms • Relatively straightforward • Extract all possible substrings • Raise an alarm when • FrequencyCounter[substring] > threshold1 • SourceCounter[substring] > threshold2 • DestCounter[substring] > threshold3

22. Detecting Common Strings • Cannot afford to detect all substrings • Maybe can afford to detect all strings with a small fixed length

23. Detecting Common Strings • Cannot afford to detect all substrings • Maybe can afford to detect all strings with a small fixed length A horse is a horse, of course, of course F1 = (c1p4 + c2p3 + c3p2 + c4p1 + c5) mod M

24. Detecting Common Strings • Cannot afford to detect all substrings • Maybe can afford to detect all strings with a small fixed length A horse is a horse, of course, of course F2 = (c2p4 + c3p3 + c4p2 + c5p1 + c6) mod M F1 = (c1p4 + c2p3 + c3p2 + c4p1 + c5) mod M

25. Too CPU-Intensive • Each packet with payload of s bytes has s-β+1 strings of length β • A packet with 1,000 bytes of payload • Needs 960 fingerprints for string length of 40 • Still too expensive • Prone to Denial-of-Service attacks

26. CPU Scaling • Random sampling may miss many substrings • Solution: value sampling • Track only certain substrings • e.g. last 6 bits of fingerprint are 0 • P(not tracking a worm) = P(not tracking any of its substrings)

27. CPU Scaling • Example • Track only substrings with last 6 bits = 0s • String length = 40 • 1,000 char string • 960 substrings  960 fingerprints • ‘11100…101010’…‘10110…000000’… • Track only ‘xxxxx….000000’ substrings • Probably 960 / 26 = 15 substrings in total string that end in 6 0’s

28. CPU Scaling • P(finding a 100-byte signature) = 55% • P(finding a 200-byte signature) = 92% • P(finding a 400-byte signature) = 99.64%

29. Estimating Content Prevalence • Finding the packet payloads that appear at least x times among the N packets sent • During a given interval

30. Estimating Content Prevalence • Table[payload] • 1 GB table filled in 10 seconds • Table[hash[payload]] • 1 GB table filled in 4 minutes • Tracking millions of ants to track a few elephants • Collisions...false positives

31. Multistage Filters stream memory Array of counters Hash(Pink) [Singh et al. 2002]

32. Multistage Filters packet memory Array of counters Hash(Green)

33. Multistage Filters packet memory Array of counters Hash(Green)

34. Multistage Filters packet memory

35. Multistage Filters packet memory Collisions are OK

36. Multistage Filters Reached threshold packet memory packet1 1 Insert

37. Multistage Filters packet memory packet1 1

38. Multistage Filters packet memory packet1 1 packet2 1

39. Stage 2 Multistage Filters packet memory Stage 1 packet1 1 No false negatives! (guaranteed detection)

40. Conservative Updates Gray = all prior packets

41. Redundant Redundant Conservative Updates

42. Conservative Updates

43. Estimating Address Dispersion • Not sufficient to count the number of source and destination pairs • e.g. send a mail to a mailing list • Two sources—mail server and the sender • Many destinations • Need to count the unique source and destination traffic flows • For each substring

44. Bitmap counting – direct bitmap Set bits in the bitmap using hash of the flow ID of incoming packets HASH(green)=10001001 [Estan et al. 2003]

45. Bitmap counting – direct bitmap Different flows have different hash values HASH(blue)=00100100

46. Bitmap counting – direct bitmap Packets from the same flow always hash to the same bit HASH(green)=10001001

47. Bitmap counting – direct bitmap Collisions OK, estimates compensate for them HASH(violet)=10010101

48. Bitmap counting – direct bitmap HASH(orange)=11110011

49. Bitmap counting – direct bitmap HASH(pink)=11100000

50. Bitmap counting – direct bitmap As the bitmap fills up, estimates get inaccurate HASH(yellow)=01100011