Enterprise Security: A Community of Interest Based Approach

1. Enterprise Security:A Community of Interest Based Approach Patrick McDaniel (psu), Subhabrata Sen, Oliver Spatscheck, Jacobus Van der Merwe, Charles Kalmanek (at&t), Bill Aiello (ubc) NDSS’06

2. Outline • Introduction • Dataset • Securing the End Host • COI Profiles • Throttling Disciplines • Usability Analysis • Security Analysis • Conclusion and Comments Speaker: Li-Ming Chen

3. Enterprise Networks • Enterprise networks have certain properties which make it easier to protect them • Known network topology • Have knowledge of all end hosts allowed • Manageable end hosts • Controllable routers and switches • Traditional perimeter defense – firewalls • Using rules to protect internal hosts from potentially malicious external hosts Speaker: Li-Ming Chen

4. Motivation and Goal • (vs. Internet) Corporate enterprise networks carry the vast majority of mission critical communications • A successful worm attack within it will be substantially more devastating to most companies than attacks on the Internet • Firewalls are not enough • worms might be introduced by laptops or by unauthorized software installations • These attacks are exacerbated by the size of enterprise networks • (Goal) improve the protection against active malware within enterprise networks • Protect internal-to-internal communications! Speaker: Li-Ming Chen

5. Dataset • 11 weeks flow records are collected from a single site in a large enterprise environment (at&t..?) • This environment consists of more than 400 distributed site and serves more than 50,000 users • The flow records contain all traffic for more than 300 hosts • Take 150 hosts that communicated during the entire 11 week period as the focal point of the analysis • Data preprocessing: • Exclude the communication with the external hosts • Only focus on TCP and UDP traffic • Remove weekend data • Tag data with client/server designations Speaker: Li-Ming Chen

6. Problem Settings • Defining rules for dropping or allowing packets where both the source and destination are internal hosts • Rules could be any arbitrary subset of the 4-tuple: • source IP、destination IP and port、protocol • A brownfield approach • Target in existing large, complex enterprise network • The design space of rules should follow 3 principle: • Security、usability、manageability Speaker: Li-Ming Chen

7. Methodology • Premise • If future communication patterns are constrained to historical “normal” communication patterns, then the ability of malware to exploit vulnerabilities in the enterprise is severely curtailed • This premise might hinder both usability and security • Approaches: • Develop a COI (Community of Interest) profile of each end host to capture what communication is normal • Define TDs (Throttling Disciplines) to handle out-of-profile communications Speaker: Li-Ming Chen

8. Simple COI Profiles • Pure history-based profiles for a given set of clients • 1. PCSPP {Proto, Client, Server, Server Port} • Most closely represents past communication • Suffer the problems of applications using ephemeral port • 2. PCSP {Proto, Client, Server} • Wild cards the Server Port • 3. PCP {Proto, Server} • Only contains all {Proto, Server} tuples for the given set of clients To compensate for the presence of ephemeral port communication. (promote usability) But with weak security Speaker: Li-Ming Chen

9. Extended COI Profile • Identify the ephemeral communications and define ephemeral rules to assist the PCSPP • Use an automated data clustering approach to accurately partition the training data • 4-step approaches: • Non-ephemeral Global • Non-ephemeral Per-Server • Ephemeral (generate ephemeral rules) • Non-ephemeral Unclassified generate PCSPP rules Speaker: Li-Ming Chen

10. Extended COI Profile (4-step Approaches) • Step 2: Non-ephemeral Per-Server identify the • significant (server, port) pairs. • Also use K(2)-means algo. • PCSPP rules: (prot, c, s, p) 445 80 Popular service ports 66 Ephemeral -like 55 21 # of servers using that port 44 33 # of connections of port Unclassified Ephemeral • Step 1: Non-ephemeral Global • use K(2)-menas to separate the • heavy-hitter ports. • The ports are then selected to • build rules for PCSPP, (prot, c, • s, p) • Step 3: Ephemeral, identify those • (client, server) pairs comm. on • many ports ! • Add ephemeral (range) rules. • Step 4: add unclassified comm. to the PCSPP ! Speaker: Li-Ming Chen

11. 3 Throttling Disciplines • n-r-Strict、n-r-Relaxed、n-r-Open • Miss: every out-of-profile communication attempt by a host is deemed a miss • n-r is the allowable rate of out-of-profile communication • means: “if number of misses exceed a threshold n within a time period r ” • Event: an event is triggered when the TD threshold n is reached Before trigger event The event Speaker: Li-Ming Chen

12. Usability Analysis (profile size) • The profile size will impact the complexity required to implement such a profile to network device (switch/router/firewall) • Profile size = number of rules needed to be specified • A rule has slightly different definitions for the profiles • E.g., PCSPP rules defined as (prot, c, s, p) • E.g., Extended COI Profile includes: (1) non-ephemeral PCSPP rules (2) ephemeral communication rules Speaker: Li-Ming Chen

13. Usability Analysis (profile size) (cont’d) (Conclude: the profile sizes are quite manageable !!) (both UDP & TCP) Require less than 400 ephemeral rules for the client set TCP server ports are more stable than UDP server ports Rules increase by adding client IP address Speaker: Li-Ming Chen

14. Usability Analysis (the prediction) 20% of the clients miss at least 100 connections per week. (Unusable PCSPP..) (This highlights the need for a policy that allows for some level of out-of- profile comm.) (Missed connections per client in PCSPP) (Total connections per client) The 4 test weeks has a comparable mix of client traffic. Speaker: Li-Ming Chen

15. Usability Analysis (Impact of 3 TDs) • Parameters of TDs simulation: • Profile: PSP, PCSP, PCSPP, and extended COI • TD: STRICT, RELAXED, OPEN • c: the out-of-profile counter • n: the allowed threshold, {0, 1, 5, 10, 15, 20} • r: the counter-reset-time (reset to 0), {1 hr, 1 day} • Block Time: the event execution time (after a client is unblocked c is reset to 0), {1 min, 10 min, 1 hr} • The simulation measures blocked events, blocked connections and blocked time. Speaker: Li-Ming Chen

16. Usability (Impact of 3 TDs)(Number of Blocking Events using 10 min. block time) 90%tile clients’ avg. TDs and # of events is Independent ! 50%tile clients’ avg. Speaker: Li-Ming Chen

17. Usability (Impact of 3 TDs)(Blocked Connections for 3TDs using 10 min. block time) 90%tile clients’ avg. OPEN RELAXED • OPEN TD performs best in usability. • (but cannot provide security..) • 0-r-RELAXED = 0-r-STRICT. • STRICT TD always blocks out-of-profile • comm. even if no event occurs. • Simple COI based profiles are becoming • less usable as additional IP header fields • are considered. • r seems to impact the usability sub-linearly. STRICT Speaker: Li-Ming Chen

18. RELAXED TD, r = 1 day, n = 10. Usability (Impact of 3 TDs)(Blocked Connections vs. Block duration) • The block time is determined • by how quickly network • operators react. • Blocked connections increase • sub-linearly with increasing • block time. • The result is acceptable.. 10 min. 50%tile clients’ avg. 10 min. 90%tile clients’ avg. Speaker: Li-Ming Chen

19. Usability (Impact of 3 TDs)(The Impact of Extended COI) • A substantial part of the out-of-profile connections in the PCSPP are due to ephemeral ports • Use extended COI profile to more accurately predict such ephemeral comm. • The table shows the relative improvement of the events and blocked connections of extended COI profile Speaker: Li-Ming Chen

20. Security Analysis • A simulation based security evaluation • Perform in discrete time (round) within a modeled enterprise network • The vulnerability (target port) is fixed • Each infected host has a fixed probability s of successfully comprising one another host in a round • But depends on the policy • The infected hosts will attempt to infect other hosts in subsequent rounds • The experiment terminates when all hosts are compromised or there are no hosts that can compromise any remaining uninfected hosts • Assume all hosts that have the target port in their profile are vulnerable Speaker: Li-Ming Chen

21. Security Analysis (# of ) SMTP HTTP DNS DCE endpoint resolution NETBIOS name service NETBIOS name service NETBIOS session service HTTPS Microsoft-DS (RPC) • The number of infectable hosts by protocols • By construction, all hosts will be modeled as • vulnerable in the PCSP and PSP Speaker: Li-Ming Chen

22. Security Analysis (Worst-case Scenario) • worst-case, all hosts are • vulnerable and no counter • -measure in place to detect • and mitigate the worm. • the curve demonstrate why • worms are so dangerous. • Hit-list worm takes only 14 • rounds to infect the entire • network. • Goal: slow the rate of • infection.. (hard to “stop • a worm”) 14 round Speaker: Li-Ming Chen

23. Security Analysis (Worm Containment, # of infected hosts) Worm infections on port 137, UDP, n=10, s=1%, 4 Profiles, 3 TDs. 98% 47% around 30% • After 10 misses, the host is prevented • from communicating over the network • The STRICT almost never goes • beyond a single host ! • The OPEN lead to more polar results. • and the profile types begin to exhibit • different levels of effectiveness ! Speaker: Li-Ming Chen

24. Security Analysis (Worm Moderation, time to terminate) Worm infections on port 137, UDP, n=10, s=1%, 4 Profiles, 3 TDs. • 10 round lower bound occurs when the • worm stays alive while it consumes its • n=10 out-of-profile grace connections. • The OPEN leads to polar behavior • Notice that the time to saturation is • significantly longer than the baseline • simulation • (allow more time to enact effectively) Speaker: Li-Ming Chen

25. Conclusion • This paper presented a brownfield approach to hardening an enterprise network against internally spreading malware. • Can automatically generate 4 different individual host profiles to capture historical COI • Define 3 security TDs. • The results validate the key premise of the approach • Examine the tradeoff between usability and security • Suggestion: • Extended COI profile + n-r-Relaxed TDs • Future work: • The profiles update ! Speaker: Li-Ming Chen

26. My Comments • The Environment • The COI-like approaches are suitable for well managed network environments • Compare to our work: • It also relies on the historical normal dataset and mentions that the profiles need to be updated as communication patterns change over longer time period. • It focus on the 4-tuple, especially the DP when building the Extended COI profiles • As a detection mechanism, it emphasizes the tradeoff between security, usability and manageability • We are focus on a scalable forensics mechanism and the tradeoff between the accuracy and scalability • FP (usability) is not that important in our case Speaker: Li-Ming Chen