FORENSIC BOTNET DETECTION

1. FORENSIC BOTNET DETECTION PROF. NASIR MEMON, POLYTECHNIC UNIVERSITY, BROOKLYN, NEW YORK DR. ELLIOT FISCHER, BELL LABS INTERNET RESEARCH DEPT., WHIPPANY, N.J. ARO-DARPA –DHS SPECIAL WORKSHOP ON BOTNETS JUNE 22-23, 2006

2. OUTLINE • MOTIVATION – WHY A FORENSIC APPROACH • THE ForNet SYSTEM AS INFRASTRUCTURE • PROPOSED BOTNET DETECTION SYSTEM • ILLUSTRATIVE EXAMPLE Lucent Technologies – Polytechnic University

3. MOTIVATION – WHY A FORENSIC APPROACH? • Botnet evidence is subtle and spread over different “channels” • Scanning behavior over time • Sporadic use of irc or other comm channels to communicate with master • Changes to windows registry or other host activities • Detection requires collecting evidence over time • Behavior found in network traffic • Behavior found in infected hosts • Need capability to reach back in time to search for additional evidence in network traffic infected hosts and build up detection confidence Lucent Technologies – Polytechnic University

4. ForNet – FORENSIC NETWORK ForNet Domain: A domain covered by single monitoring and privacy policies. Forensic Server: Responsible for archiving synopses, query processing & routing, enforcing monitoring, security policies, for the domain. SynApp:equipped routers or hosts. Primary function is to create synopses of network traffic. May have limited query processing and storage component as well. Lucent Technologies – Polytechnic University

5. ForNet COMPONENTS • SynApps • Collect and Synopsize Data • Either standalone devices or embedded into networking components, interconnected with forensic servers to form a hierarchy • All synapps within a domain form a network and are associated with the forensic server for the domain • Data collected / summarized • Links/connections between the nodes • Content traversing the links • Various protocol mappings • Data can be collected and stored for months and archived and analyzed for even longer periods Lucent Technologies – Polytechnic University

6. DATA SYNOPSES • Use of Bloom Filters and hierarchical bloom filters (HBFs) for packet content querying • Which flows contained packet content “xyz” ? • Only store the filter, not the packet content • Can span packets • Can be used to detect existence of bot passwords or other packet content • Flow content characterization • Encrypted, compressed, text, audio, video, or jpeg • Flow records Lucent Technologies – Polytechnic University

7. Synopses in ForNet Lucent Technologies – Polytechnic University

8. FORENSIC SERVER AND QUERIES • Forensic server stores data and processes queries • Archiver for data collected by the synapps • Advertises monitoring and privacy policies of the domain • Receives queries from outside the domain boundaries, authenticates them and either responds to them itself or passes them along to the appropriate synapps • Queries are a collection of one or more events in a set of networks within a time interval • May partially describe an event and request that the details be filled in by ForNet • May be sent to the forensic server of a domain or can be propagated to forensic servers in neighboring networks for gathering additional information Lucent Technologies – Polytechnic University

9. ForNet Deployed in an Intranet • Investigations based on payload characteristics • Determine victims of worms, trojans and other malware • Trace spread of mydoom • Detection of potential victims of phishing and spyware • Source of intellectual property theft • Investigations based on connection characteristics • Detection of zombies in a network • Detection of malware (bd) based on connection pattern • Detection of emerging threats (proactive) • Determination of “host roles” (proactive) • Investigations based on aggregates • Insider abuse • Downloading too much or too little but consistent • Network troubleshooting Lucent Technologies – Polytechnic University

10. STORAGE AND MEMORY • 1.3TB server stores over 3 months of data from edge and 2 subnets • Few thousand nodes • Bandwidth consumption of network is about a 1TB/day • Synopses reduces this traffic to about 25GB/day • 4 TB server can store over 9 months of data Lucent Technologies – Polytechnic University

11. BOTFINDER SYSTEM ARCHITECTURE Lucent Technologies – Polytechnic University

12. BOTSIG SIGNATURE DATABASE • Signature language • Forensic capability • Detection and corroboration from both network and host data Lucent Technologies – Polytechnic University

13. BOTSIG SIGNATURE DATABASE (CONT’D) • Signature may include corroborating patterns for a subset of botnet phases • Each corroborating pattern may require mechanisms from the NTA, HTA or both • Examples: • Connect: NTA queries ForNet to detect if any of a set of suspicious hosts sent or received a particular byte pattern according to the stored synopsized data ·Server password “gringle”, ircbot.Gt • Trigger satisfied by NTA detecting traffic on known irc channel • HTA detects specific library call on host • Connect: NTA queries ForNet for set of hosts that communicated with one of the known servers for a triggered irc channel in the last two weeks • Setup: detects periodic process over time · Checking for connectivity every 5 minutes, sdbot.Ag • Propagate: trigger satisfied by NTA detecting scans for specific exploitable vulnerabilities ·dcom rpc, PHATBOT • HTA checks if host is in promiscuous mode (PHATBOT) Lucent Technologies – Polytechnic University

14. NETWORK TRACE ANALYZER • Bridge between ForNet and BOTFINDER • Combine • Information about network events from ForNet • Signature information from BOTSIG • Construct and analyze evidence of potential botnets • Can transform BOTSIGs into appropriate ForNet queries and interpret the results • Supplies ForNet with a set of triggers from BOTSIG that are first signs of a potential botnet • Look for particular bit-string in network traffic associated with bot • Threshold function of packet size and inter-arrival time distribution per connection over a period of days Lucent Technologies – Polytechnic University

15. HOST TRACE ANALYZER • Allows BOTFINDER to look on end host for evidence of bots • Remote operations–actions the HTA can execute automatically: • Reading Windows registry entries using Remote Registry Service , provides authorized users remote access to the registry on Windows XP, Windows 2000, and Windows Server 2003 • Examining file contents and directory structure on a remote host using tools such as Windows File Sharing and PsTools • Local operations–actions executed by the Sys Admin on the suspected host: • Detect • Known vulnerabilities, rootkits, and backdoors. presence of vulnerabilities and malicious code • Open files or network ports by running utility program( Foundstone FPort ) • Hidden files • Host-resident application operations–executed by programs running on each host (such as commercial anti-virus software) • Detect changes to the content of key OS files using file integrity checkers, such as Tripwire • Monitoring of system and event logs for anomalous events, such as the addition of new users accounts on a desktop • Detecting anomalous activity on a host system such as intrusion detection systems Lucent Technologies – Polytechnic University

16. MITIGATION RECOMMENDER • All mitigations are presented as recommendations to the Systems Administrator (SA) • Makes use of information gathered during detection and corroboration. • Constructs recommendation by extracting strategy from the corresponding BOTSIG signature and automatically composing specific recommendation • List of addresses and ports to block • files to delete • Tools that can be run automatically on network devices and hosts (with SA approval) to mitigate the bots • Additional defensive recommendations • cleanup vulnerabilities or backdoors associated with the botnet. Lucent Technologies – Polytechnic University

17. BOTFINDER CONTROLLER • Provides coordination between various components • Network trace analyzer • Host trace analyzer • Mitigation recommender • Responsible for coordinating these actions and their results • Determines when to apply each BOTSIG Lucent Technologies – Polytechnic University

18. ILLUSTRATIVE EXAMPLE – BOT DESCRIPTION • Hypothetical strain of AGOBOT, from which PHATBOT was derived • Behavior is similar to that of PHATBOT • (1) Scans the network for vulnerable hosts to infect and uses the irc protocol on a non-standard port for command and control. • (2) After installation, the bot configures an irc client and connects to a rogue server, scans the network for three backdoors (port 2745 for bagle, 3127 mydoom, and 3410 optix trojan), sends the scan results to an irc server, goes dormant except for • (3) periodic irc ping messages • (4) waits for commands to launch new attacks • (5) during installation, the bot updates a windows registry value to rerun the bot application after reboot. Lucent Technologies – Polytechnic University

19. ILLUSTRATIVE EXAMPLE- DETECTION • (6) BOTSIG includes AGOBOT signature that specifies NTA should construct a ForNet query to detect the byte pattern corresponding to the specific irc ping message in network traffic • Query returns a set of potentially infected hosts • Further corroboration needed to confirm the existence of bots because legitimate irc traffic may also contain the same byte pattern and the query might have missed some bots • (7)BOTSIG signature specifies second query to NTA for scanning pattern that bot uses to locate backdoors • Query checks historical connection records in the synapps to find any hosts that • contacted the same server as the potentially infected hosts and • scanned the network on ports 2745, 3127, or 3410, which the bot uses for backdoors. • (8) For further corroboration, BFC requests HTA to check potentially infected hosts for further evidence • HTA looks in BOTSIG for the particular registry key that bot uses to register itself as a service that starts at boot-time Lucent Technologies – Polytechnic University

20. MITIGATION RECOMMENDATIONS • (9) If AGOBOT is confirmed to be on host, then HTA responds to BFC that it has detected AGOBOT on the host • Systems administrator is alerted with a list of infected hosts and mitigation recommendations. • (10) Present Systems Administrator with • List of suspected host addresses to block at access switch • List of suspected server addresses and ports to block at the firewall • (11) Provide instructions on how to clean the infected host • Removing bot, registry keys, and the backdoor(s) used Lucent Technologies – Polytechnic University

21. ILLUSTRATIVE EXAMPLE Lucent Technologies – Polytechnic University

22. CONCLUSIONS • Forensic detection is needed to find subtle attacks like botnets and low and slow attacks • Need to develop evidence over time and go back in time to find corroborating evidence in network traffic and host behavior • ForNet can serve as the forensic infrastructure needed to facilitate detection • Synopses of flows and packet contents over long periods of time (months) needed to detect these subtle attacks • Packet synopses can be used to detect traffic with particular keywords or other evidence • Connection histories can be queried to find other evidence in network traffic • Botnet detection using ForNet could lead to earlier and more accurate detection Lucent Technologies – Polytechnic University