Bro: A System for Detecting Network Intruders in Real-Time

1. Bro: A System for Detecting Network Intruders in Real-Time • Presented by • Abdulmajid , Ameen, Yaseen ,mohammed • Lecturer : • DR.Nur Izura

2. Bro Overview • Bro is an open-source, Unix-based Network Intrusion Detection System (NIDS) that passively monitors network traffic and looks for suspicious activity. • Bro was developed by Vern Paxson in the Network Research Group at Lawrence Berkley National Lab, and by the International Computer Science Institute. • Bro == Big Brother (“watching and taking care of you” and all your actions).

3. Original Goals • High-speed, large volume monitoring: • No packet filter drops (Low packet loss rate) Would allow intruder to Avoid the NIDS. • Real-time notification • The monitor will be attacked • Mechanism separate from policy:

4. Bro Features • Rich Application-Layer Analysis • Custom Scripting Language • Pre-written Policy Scripts • Powerful Signature Matching Facility • Network Traffic Analysis • Detection Followed by Action • Snort Compatibility Support (snort2bro)

5. Target Environments • Bro is intended for use by sites requiring flexible, highly customizable intrusion detection. • Development of bro is primarily driven by research for intrusion detection and traffic analysis • Bro is specifically well-suited for scientific environments • Extremely useful in networks with liberal (“default allow”) policies • Supports intrusion prevention schemes • High-performance on commodity hardware • Runs on Unix-based systems (e.g., Linux, FreeBSD, MacOS) • Open-source (BSD license)

6. Target Environments(con`t) • Bro is designed for use by Unix experts who place a premium on the ability to extend an intrusion detection system with new functionality as needed, which can greatly aid with tracking evolving attacker techniques as well as inevitable changes to a site's environment and security policy requirements. • Some sites may wish to run a commercial IDS as their front-line of defense, and then also run Bro as a way to: • Verify the results of the commercial IDS / defense-in-depth • Attain richer forensics capabilities • Provide policy-checking capabilities not facilitated by the commercial IDS • Experiment with new approaches and incorporate leading-edge research

7. Bro Deployment

8. Structure of the system • Bro is layered into three major components . It uses the now-standard libpcap packet-capture library developed by LBNL's Network Research Group to filter the packet stream in the kernel for high performance. • Bro is conceptually divided into an “event engine” that reduces a stream of (filtered) packets to a stream of higher-level network events. • An interpreter for a specialized language that is used to express a site's security policy. • The lower-most layers process the greatest volume of data, and hence must limit the work performed to a minimum. • As we go higher up through the layers, the data stream diminishes, allowing for more processing per data item. • This basic design reflects the need to conserve processing as much as possible, in order to meet the goals of monitoring high-speed, large volume traffic flows without dropping packets

9. Bro Architecture • “Policy Scripts” in Bro Policy language processes event stream ... Policy Interpreter • ... and takes action! • Distills filtered stream into high-level events. Event Engine • Kernel filters down high-volume stream via packet capture library. libpcap • Tap network link passively, send up a copy of all traffic. Network

10. Architecture • Taps network link passively, sends up a copy of all network traffic. Network

11. Architecture • Kernel filters down high-volume stream via standard libpcap packet capture library. Tcpdump Filter Filtered Packet Stream libpcap Packet Stream Network

12. Architecture • “Event engine” distills filtered stream into high-level, policy-neutral events reflecting underlying network activity • E.g., connection_attempt, http_reply, user_logged_in • These span a range of semantic levels • Currently about 300 different types Event Control Event Stream Event Engine Tcpdump Filter Filtered Packet Stream libpcap Packet Stream Network

13. Architecture Policy Script Real-time Notification Record To Disk • “Policy script” processes event stream, incorporates: • Context from past events • Site’s particular policies Policy Script Interpreter Event Control Event Stream Event Engine Tcpdump Filter Filtered Packet Stream libpcap Packet Stream Network

14. Architecture Policy Script Real-time Notification Record To Disk • “Policy script” processes event stream, incorporates: • Context from past events • Site’s particular policies Policy Script Interpreter Event Control Event Stream Event Engine Tcpdump Filter Filtered Packet Stream • … and takes action: • Records to disk • Generates alerts via syslog or paging • Sends events to other Bro’s • Executes programs as a form of response libpcap Packet Stream Network

15. Packet Filtering • capture via pcap API (libpcap) • Filters expressed in same format as tcpdump(8).Also support off-line traces • port finger or port ftp or (tcp[13] & 7 != 0) • Using libpcap gains significant advantages: it isolates Bro from details of the network link technology (Ethernet, FDDI, SLIP, etc.Network . • it greatly aids in porting Bro to different Unix variants (which also makes it easier to upgrade to faster hardware as it becomes available). Berkeley Packet Filter (BPF), found on some Unix, it allows libpcap to bring Bro’s network filter into the kernel (filters aren’t in userland!). • FreeBSD has a Zero-copy Buffer Extension, which makes it Bro’s OS of choice!

16. Event Engine • Written in C++. • Distills filtered stream into high-level, policy-neutral events reflecting underlying network activity! • E.g., connection_attempt, http_reply, user_logged_in, etc. • Protocols supported: 30 protocols • ARP, IP, ICMP, TCP, UDP • DCE-RPC, DNS, FTP, Finger, Gnutella, HTTP, IRC, Ident, NCP, NFS, NTP, NetBIOS, POP3, Portmapper, RPC, Rsh, Rlogin, SMB, SMTP, SSH, SSLCiphers, SunRPC, Telenet

17. Event Engine, cont. • Those protocol analyzers generate over 300 events! • new_connection(c: connection), new_packet(c: connection, p: pkt_hdr), connection_established(c: connection), tcp_contents(c: connection, is_orig: bool, seq: count, contents: string), icmp_echo_reply(c: connection, icmp: icmp_conn, id: count, seq: count, payload: string), ... • Originally, event engine analyzers were written in C++ Class libraries. • Now BinPAC (Binary Protocol Analyzer Compiler) is used. • binpac generates the C++ code.

18. Event Engine, cont. • When the packets arrive, the Event Engine checks if the headers of these packets are well formed or not. Packages with not well formed headers are discarded and events of this problem are created. • The packets with well formed headers gets a connection state which can be connection_attempt,connection_established, connection_rejected or connection_finished for TCP and for UDP the state is defined as udp_request or udp_reply. • The packets are then collected to their corresponding connection. These collections use different protocols e.g. HTTP, FTP, SMTP and is therefore handled by the analyzers specialized for each protocol. • Whether or not the connection are handled by any analyzer depends on which analyzers that are enabled in the system.

19. Event Engine, cont. • The analyzer for each protocol then examines the content of the connection. • When this procedure is done, the Event Engine has created high-level events describing the activity and these are now ready for the Policy Script Interpreter. • As analyzers observe activity, generate events • Event = name + typed parameter list • Queued for asynchronous execution • All events triggered by a given packet executed prior to processing next packet (Along with timers that expire at that point).

20. Policy Script Interpreter • The Policy Script Interpreter takes the high-level events generated by the Event Engine and compares these with the policy scripts in the system. • The events are sorted in a FIFO list which means the first that comes along are the first that is processes. • Policy Script Interpreter takes action if it detects any suspicious and dangerous actions or it discards other events not defined in the policy scripts. • Traffic that seems like attacks but aren’t (false negatives), can be detected at this point, but if the policy scripts are good, this will be minimal.

21. Policy Language • Policy analysis scripts which track network activity written in Bro’s language • Procedural You write event handlers = functions asynchronously executed that don’t return a value). • Strongly typed (to detect inconsistencies at compile-time). • Rich in types: • USUAL : bool, int, count, double, string, time, interval, port, addr, hostname, pattern. • Domain-specific types, such as addresses, ports, subnets • Aggregates: record, table, set, file. • Real-time communication with other Bro instances and Extensive state management support (Timers; automatic expiration; persistence

22. Policy Language • Policy analysis scripts written in Bro’s language: • Procedural (you provide event-handlers that are executed asynchronously and that don’t return a value). • Strongly typed (to detect inconsistencies at compile-time). • Rich in types: • bool, int, count, double, string, time, interval, port, addr, hostname, pattern. • Aggregates: record, table, set, file. • Flex-like regular expressions built into language (pattern)!

23. Language, cont. • Operators (+, -, *, /, %, !, &&, ||, ?:, relationals, infix, --, ++). • Static, global & const scoping of variables. • const allowed_services: set[addr, port] = { • [ftp_serv, [ftp, smtp, 20/tcp]], • [nttp.lbl.gov, nttp] • }; • Statements (if-then-else, when, return, print, add, delete, and function). • No for loops! But recursion through functions is allowed.

24. Notices & Alarms Convention is to express much detection in terms of “notices” • I.e., “I’ve noticed the following activity” • Notices do not necessarily convey a problem • Distributed scripts detect 78 different types of notices • Scripts provide number of ways to decide when to promote a notice to an “alarm”These can include syslog, sending email, paging • When you see in a script something like NOTICE([$note=OutboundTFTP, $conn=u, $msg=fmt("outbound TFTP: %s -> %s", src, dst)]);it’s just a call to a function named “NOTICE”, using the language’s mechanism for constructing a record • Note: Entire mechanism is add-on to language, not built in

25. Sample Bro Policy • Using the Bro language, sites can write custom policy scripts to generate alarms on any policy violation. • For example, if a site only allows external http and mail to a small, controlled lists of hosts, they could do this: const web_servers = { www.lbl.gov, www.bro-ids.org, }; const mail_servers = { smtp.lbl.gov, smtp2.lbl.gov, }; redef allow_services_to: set[addr, port] += { [mail_servers, smtp], [web_servers, http], }; • Bro can then generate an Alarm or even terminate the connection for policy violations: if ( service !in allow_services) NOTICE([$note=SensitiveConnection, $conn=c,]); if ( inbound && service in terminate_successful_inbound_service ) terminate_connection(c);

26. Sample Bro Alarms

27. FTP buffer overflow attack

28. Bro in Practical Use • Primary IDS for LBNL/NERSC since 1996 • Primary IDS for SC00-03 conferences • No specialized hardware needed • Low cost allows for multiple deployment • Requirements • FreeBSD • Intel platform • Fiber tap • Disk space to archive data

29. Sites using Bro (LBNL) • Use multiple Bros across a 10Gb/s link, where each Bro is only executing a minimal set of polices. • Operational 247 since 1996. Monitors traffic for suspicious behavior or policy violations: incoming/outgoing/internal • In conjunction with blocking routers, Bro acts as a dynamic and intelligent firewall • Blocks access from offending IP addresses • Blocks high risk ports • Blocks known high-risk activity • Terminates connections and/or sends alarms • Locates site policy violations (e.g.: Kazaa and gnutella)

30. Sites using Bro • IDS controls a blocking router • IDS blocks dynamically when an intrusion attempt is detected or alerts upon suspicious activity • Router blocks statically like a firewall “Intrusion Prevention”

31. SC2003, Poenix, AZ Use of Bro Within NERSC ESNet Multiple Bro Systems • Real Time Analysis • Redundant Backup • Test Box • Bulk Traffic Recorder Tapped Traffic Network Traffic Filtering Border Router ACL Insertion Multiple IDS • Snort • Bro Heavyweight Protocol Analysis • Bro GRID / SSL Analysis Tapped Traffic Tapped Traffic • Internal Traffic Bro Monitor • Wireless Network Bro Monitor NERSC Wireless Network

32. Bro at (NERSC) • Tied into a paging system for on-call security person • Bro check pointed at set intervals (Reports generated whenever checkpointed) • Clears out ‘orphaned’ sessions • Allows for offline data analysis Data archiving • Maintain traffic data for about 3 months • Maintain network • Automatic ACL injection has very low false positive rate • At NERSC average about 1 every 6 months

33. NERSC Network Traffic3 Week Period

34. Hardware Support • Significant Bro research effort these days focuses on hardware to accelerate network analysis • Note, quite different problem than fast signature matching! • Approach #1: Bro Cluster • Set of PCs use inter-Bro coordination for joint analysis • Scales to x23 performance for 24 nodes (then flattens) • Approach #2: Shunting • Custom (simple) FPGA provides intelligent front-end • Per-flow decisions on forward/block/divert through IDS/IPS • 1 Gbps prototype working, undergoing upgrading • Future: Multicore Bro • Will require extensive analysis to understand/tune memory locality & intra-analysis messaging

35. Bro Future Directions • Grid related technologies • Ability to detect Grid related protocols • X.509 Certificate Analyzer • SSL Analyzer • Verify certificates are legitimate • Router Shunting • Primary bottleneck in moving packets into user space • Leverage router based hardware filtering to analyze “packets of interest” • Proof of concept demo at SC01-03 1)Utilizing Bro and Juniper router 2) Hardware based BPF to filter traffic

36. Port Mirroring External Network Mirrored Traffic Juniper GigE Interface Bro Internal Network

37. Filter-based Forwarding External Network Filtered Traffic Bro GigE Interface Filter Juniper Internal Network

38. Pros • Wow, where do we begin? • Separation of event generation and event handling (mechanism vs. policy). • Code is/was written for a hostile environment. • Bro’s language (to write policy scripts) is quite tractable. • Resilience in the presence of flooding(Via connection compressor) • An active community (mailing list: Bro@bro-ids.org). • Reaffirms that open-source solution are possible! • Support for plugging in external analyses

39. Pros : Bro Noteworthy Features • Broccoli = Bro Client Communications Library • C interface for external programs to transmit & receive values and events • E.g., syslog reader sends log messages to Bro • Dynamic adaptation to load • Via policy/load-levels.bro • Changes current packet filter to activate more/less analysis • User-level state management • Entries in tables, sets automatically expire T sec. after Creation / last read / last write • Advanced detection algorithms (Backdoors: protocols running off-port (obviated by DPD) • )

40. Cons • Encrypted Traffic is Bro’s bane: • Act like a SSL proxy (keep *all* necessary certificates?) • IPv6 ... I think it (finally) made it into development code and asymmetry . • Pretty complex, script-based system • • Requires understanding of the network • • No GUI, just ASCII logs • Only partially documented • • Lacking resources to fully polish the system

41. The End Thank You