Fast Port Scan Detection Using Sequential Hypotheses Testing *

1. Fast Port Scan Detection Using Sequential Hypotheses Testing* Authors: Jaeyeon Jung, Vern Paxson, Arthur W. Berger, and Hari Balakrishnan IEEE Symposium on Security and Privacy 2004. Presenter: Tai Do CAP 6938 Jan. 18,2007

2. Introduction • Problem: Random portscans of IP addresses is a popular method for attackers to find vulnerable machines in the reconnaissance phase. • Threshold Random Walk: an online detection algorithm. • Motivation: Early detection allows some form of protective response to mitigate or fully prevent damage. • Three quantities of interest for a detection problem: • Detection accuracy • False alarm rate (false positive) • Misdetection rate (false negative) • Detection delay time

3. Challenges • No crisp definition of the activity: • An attempted HTTP connection to the site’s main Web server is OK. • A sweep through the entire address space looking for HTTP servers is NOT OK. • But how about connections to a few addresses, some of which succeed and some of which fail??? • The granularity of identity: • Probes from adjacent remote addresses as part of a single reconnaissance activity? • Probes from nearby addresses which together form a clear “coverage” pattern. • The locality of the addresses to which the probes are directed might be tight or scattered. • Temporal vs. spatial considerations: • how much time do we track activity? Do we factor in the rate at which connections are made. • Intent: • Not all scans are necessarily hostile (search engine crawlers, p2p applications).

4. Assumptions • Focus only on TCP scanners • “Identity”- Single remote IP addresses. No distributed scans. No vertical scans of a single host. • Does not assume a particular scanning rate from a remote host.

5. Outline • Existing Works • Data Analysis • Online Detection Algorithm: Threshold Random Walk • Performance Evaluation • Concluding Remarks

6. Exiting Works • Counting Models: Network security Monitor, Snort, and Bro. • Probabilistic Models: [LeckieK00], and SPICE.

7. Counting Models • Network security Monitor, Snort: “detect N events within a time interval of T seconds.” • Bro: “treats connections differently depending on their services”. Services in a configurable list (only count failed attempts) vs. others. Raise flags if the number of distinct destination addresses reaches a configurable parameter. • Disadvantages: threshold selection.

8. Probabilistic Models • [LeckieK02]: • An access probability distribution for each local IP address, computed across all remote source IP addresses that access that destination. • Also consider the number of distinct local IP addresses that a given remote source has accessed so far. • Scanners: are modeled as accessing each destination address with equal probability. • Flaws: • Many false positives • No confidence levels to assess whether the difference is large enough. • How to assign an a priori probability to destination addresses that have never been accessed.

9. Probabilistic Models • SPICE [StanifordHM00] • Detect stealthy scans (very low rates, and spread across multiple source addresses) • Assign anomaly scores to packets based on conditional probabilities derived from the source and destination addresses and ports. • Collect packets over long intervals (days or weeks) and then cluster them using simulated annealing to find correlations that are then reported as anomalous events. • Disadvantages: • Significantly more run-time processing • More complex. • Off-line method

10. Outline • Existing Works • Data Analysis • Online Detection Algorithm: Threshold Random Walk • Performance Evaluation • Concluding Remarks

11. Initial Data Sets • HTTP worms: Code Red or Nimda. • Other_bad: send packets to 135/tcp, 139/tcp, 445/tcp, or 1433/tcp corresponding to Windows RPC, NetBios, SMB, and SQL-Snaket attacks. • Two Research Labs: LBL, and ICSI • Bro NIDS is used. • 8 data sets (6 + 2). • 24-hour period. known_bad = scanner + HTTP worms + other_bad

12. A Better Ground Truth • “Ground Truth”: the available data sets is a good start, but not strong enough. • There may be undetected scanners among remainder entries. • How to determine likely, but undetected scanners? • Ideal situation: using a method that is wholly separate from the subsequently developed detection algorithm. The paper fails to find such a method. • Use the same properties to 1) distinguish likely scanners from non-scanners in the remainder hosts, and 2) incorporate in the detection algorithm. • Soundness of the method: show that the likely scanners do indeed have characteristics in common with known malicious hosts.

13. Key Observation • inactive_pct: the percentage of the local hosts that a given remote host has accessed for which the connection attempt failed (rejected or unanswered).

14. Key Observation • inactive_pct: the percentage of the local hosts that a given remote host has accessed for which the connection attempt failed (rejected or unanswered).

15. Separating Possible Scanners • inactive_pct: the percentage of the local hosts that a given remote host has accessed for which the connection attempt failed. • inactive_pct < 80%: benign remote host. • inactive_pct >= 80%: possible scanner (suspect).

16. Final Data Sets • Additional Supporting Evidence: Suspect hosts exhibit distribution quite similar to those for known-bad hosts.

17. Outline • Existing Works • Data Analysis • Online Detection Algorithm: Threshold Random Walk • Performance Evaluation • Concluding Remarks

18. Hypothesis testing formulation • A remote host R attempts to connect a local host at time i let Yi = 0 if the connection attempt is a success, 1 if failed connection • As outcomes Y1, Y2,… are observed we wish to determine whether R is a scanner or not • Two competing hypotheses: • H0: R is benign • H1: R is a scanner The distribution of the Bernoulli random variable Yi:

19. An off-line approach • Collect sequence of data Y for one day (wait for a day) 2. Compute the likelihood ratio accumulated over a day This is related to the proportion of inactive local hosts that R tries to connect (resulting in failed connections) 3. Raise a flag if this statistic exceeds some threshold

20. Stopping time A sequential (on-line) solution • Update accumulative likelihood ratio statistic in an online fashion 2. Raise a flag if this exceeds some threshold Acc. Likelihood ratio Threshold 1 Threshold 2 0 24 hour

22. The second equality follows from the i.i.d assumption of the random variables Yi|Hj. Likelihood Ratio

23. Threshold Selection Performance Criteria: Detection Probability, PD: the algorithm selects H1 when H1 is in fact true. False Positive Probability, PF: the algorithm selects H1 when H0 is in fact true. Threshold Selection: or similarly Errors: differences between actual bounds and desired bounds

24. The number of observations N until the test terminates. Detection Delay Time Log likelihood Ratio: Wald’s equation What is E[N]?

25. Outline • Existing Works • Data Analysis • Online Detection Algorithm: Threshold Random Walk • Performance Evaluation • Concluding Remarks

26. Evaluation Methodology • Used the data from the two labs • Knowledge of whether each connection is established, rejected, or unanswered • Maintains 3 variables for each remote host • D_s, the set of distinct hosts previously connected to • S_s, the decision state (pending, H_0, or H_1) • L_s, the likelihood ratio

27. Evaluation Methodology (cont.) • For each line in dataset • Skip if not pending • Determine if connection is successful • Check whether is already in connection set; if so, proceed to next line • Update D_s and L_s • If L_s goes beyond either threshold, update state accordingly

28. Comparison with other existing intrusion detection systems (Bro & Snort) 0.963 0.040 4.08 1.000 0.008 4.06 • Efficiency: 1 - #false positives / #true positives • Effectiveness: #false negatives/ #all samples • N: # of samples used (i.e., detection delay time)

29. Comparison with other existing intrusion detection systems (Bro & Snort)(cont.) • TRW is far more effective than the other two • TRW is almost as efficient as Bro • TRW detects scanners in far less time

30. Outline • Existing Works • Data Analysis • Online Detection Algorithm: Threshold Random Walk • Performance Evaluation • Concluding Remarks

31. Strengths of the paper • Good observation: • inactive_pct provides a strong modality to differentiate benign hosts from suspicious hosts. • Sequential analysis is well-suited: • Provide mathematical bounds on the expected performance of the algorithm (PD, PF, and N) • minimize the detection time given fixed false alarm and misdetection rates • balance the tradeoff between these three quantities (false alarm, misdetection rate, detection time) effectively

32. Limitations and Possible Improvements • Nearly circular argument between ground truth, and the developed detection algorithm. Both use the same key observation. • Oscillation problem in the detection algorithm. • Leveraging Additional Information: • Managing State: • How to Respond: • Evasion and Gaming: • Distributed Scans:

33. References • [LeckieK02] C. Leckie and R. Kotagiri. A probabilistic approach to detecting network scans. In Proceedings of the Eighth IEEE Network Operations and Management Symposium (NOMS 2002), pages 359–372, Florence, Italy, Apr. 2002. • [StanifordHM00] S. Staniford, J. A. Hoagland, and J. M. McAlerney. Practical automated detection of stealthy portscans. In Proceedings of the 7th ACM Conference on Computer and Communications Security, Athens, Greece, 2000. • XuanLong Nguyen. Sequential analysis:balancing the tradeoff between detection accuracy and detection delay. Presentation, Radlab, UCB, 11/06/06.