Maximizing Network Security: Internet Traffic Analysis Essentials

1. Internet Traffic Analysis for Threat Detection Joshua Thomas, CISSP Thomas Conley, CISSP Ohio University Communication Network Services

2. Abstract • Useful logs may already exist at your institution. • Network transaction logging is a very useful, flexible, and inexpensive tool for network security. • Comprehensive network security relies on log collection and analysis. • Analysis of log files can be automated, and can provide information that can be the basis for prevention and response procedures.

3. Start with what you have • The collection and analysis of network transaction data is useful for a wide range of tasks • Security management • Network billing and accounting • Network operations management • Performance analysis • As a result, some form of network transaction logs may already exist within your institution, even if not specifically implemented for network security reasons.

4. “Pointed stick” • Low cost, high returns • Simple to implement • Nonspecific, flexible • Non-restrictive

5. Fundamental need • Network transaction logs are arguably the most basic, necessary countermeasure in network security. • Logs should form the basis for decisions regarding other security initiatives. • Traffic analysis will be necessary to validate the performance of other security countermeasures.

6. Needs pyramid: Maslow’s Hierarchy Self-actualization Esteem needs Belongingness and Love needs Safety needs Biological and Physiological needs

7. Needs pyramid: Network Security IDS/IPS Firewalls Host Security Security Staff Network Transaction Logs

8. Transparent monitor • Acts as a passive device, gathering traffic and performance statistics at appropriate places in networks (server or client locations) • Is not necessarily a point of failure in your network • Cannot alter network traffic, as active devices such as firewalls or IDS/IPS systems. • However, monitoring can co-exist with other network security devices, such as IPS/IDS

9. Transparent monitor: Simple setup Upstream Provider Hub Network Network Monitor

10. Scalable • Mirroring traffic is relatively inexpensive. • Institutions may choose to capture as much data as possible and only perform limited analysis as needed. • There are appropriate solutions for implementing network transaction monitoring at just about every level of a network. • Small lab environment • Single department • University border

11. Transparent monitor: Large-scale ISP 1 ISP 2 Network Monitor

12. Selective memory • In order to be able to store and analyze high volumes of traffic, the memory demands must be reduced in some way.

13. Selective memory: Depth ! ! • IPS/IDS systems generally select certain transactions (via signature matching, etc.) for storage and analysis. In other words, only communications that match a selection criteria are recorded, and all other data is ignored.

14. Selective memory: Breadth • Flow monitoring accounts for every transaction, but does not retain the content of the transactions. • Transactions contain both routing information and content. Only routing information is retained. • Applications that can capture this sort of transaction data include Argus, tcpdump, Ethereal, cflowd, etc.

15. Flow metrics • Metrics generally captured in network transaction logs include: • Source, destination IP addresses (for IP traffic) • Beginning, end times • Packet count • Byte count • TTL (for IP traffic) • TCP flags (for TCP/IP traffic) • TCP state progression (for TCP/IP traffic) • Base sequence numbers (for TCP/IP traffic)

16. Inference • Certain traffic characteristics are very useful in making inferences about the nature of the traffic. • Examples: • Amount of bandwidth consumed • Number of connection attempts • Connections to unused address ranges

17. Automation • Identifying problems through inference can be automated. • Once the criteria has been clearly defined, then the tasks that were once done by humans can be performed by simple programs. • Once the identification of problems is automated, then those results can be fed into response procedures.

18. Examples • Compare logs with blacklists, such as known- spyware or spam source IP lists • Examine traffic destined for non-populated subnets • Noise-floor analysis • TCP port usage

19. Endless possibilities • We are constantly discovering new uses for network transaction logs

20. About our institution • 4,820 employees (1,069 full-time faculty) • 20,143 students (18,497 full-time students) • 90+ Mbps Internet bandwidth (2 ISP’s) • 6,000,000,000+ packets per day • 3,000,000,000+ source packets • 3,000,000,000+ destination packets • 2,400+ GB per day (500+ DVD-ROMs) • 727 source GB per day • 1,675 destination GB per day • ~12 GB Argus log files generated per day, on average (0.6% of the total bytes represented)

21. References/Resources • RFC 2724, “RTFM: New Attributes for Traffic Flow Measurement.” (http://www.rfc-editor.org/rfc/rfc2724.txt) • Argus: http://www.qosient.com/argus