Collaborative Online Passive Monitoring for Internet Quarantine

1. Collaborative Online Passive Monitoring for Internet Quarantine Weidong Cui wdc@EECS.Berkeley.EDU SAHARA Winter Retreat, 2004

2. Motivation • Threats to Today’s Internet • Internet worms • Code-Red, Nimda, MS-SQL (Slammer/Sapphire), Blaster • DDoS attacks • Email spams • Disaster caused by these threats • Millions of PCs cannot work properly • Automatic reboot • Disconnected by network admins • Critical servers stopped working • SQL servers • DDoS attacked servers • Network outages • Links congested • Routers down

3. Internet Quarantine • Containing self-propagating malicious code is very important • Internet worms propagation caused huge problems • DDoS attacks rely on a large number of compromised zombies • Email spammers start exploiting compromised machines to forward spam emails • To contain worms successfully, we need to [moore03internet] • Automatically detect and activate filtering mechanisms within minutes, • Generate signatures for content filtering • deploy content filtering in a large number of coordinated ISPs

4. Can We Protect Our Own Network against Intruders? • Yes, but limited… • Network intrusion detection • Misuse detection (signature-based) • Detect known malicious attacks very well • Cannot detect new attacks without signatures • Anomaly detection • Can detect new attacks • high false alarm rates due to high variance of incoming traffic • Firewalls • Not flexible, usually require human intervention • movable points (laptops) • Distributed firewall is still a research problem

5. Our Idea • Why is it hard to detect intruders? • So many of them… • Large variance of behaviors • Can we monitor local hosts? • Limited number of them • Network behavior follows some pattern • Basic idea • Monitor network behavior of local hosts • Prevent compromised local hosts from infecting others • Generate signatures based on traffic from those hosts

6. Our Approach • Detect compromised local hosts in an edge network • Online passively monitor all traffic into/from an edge network • Train a network behavior profile for each host inside the edge network and online update it • Alarm when an end host behaves anomalously • Assumption: the period of normal behavior of end hosts is long enough for this training purpose • Generate signatures of malicious code • Redirect traffic from an anomalous host to a honeypot • Create signatures in the honeypot • Distribute signatures to other networks • Can leverage on overlay multicast

7. Design Choices • Why support the proposed monitoring? • Compromised hosts may infect other hosts inside the edge network • Why monitor at gateways of edge networks? • Single monitoring point for inbound and outbound traffic • Moderate traffic load • More information than end hosts • More reliable and harder to be compromised than end hosts

8. Network BehaviorProfile (I) • Network behavior of an end host can be abstracted as a series of connections to/from that host • TCP connection; each UDP packet is a connection • Each connection can be represented by a vector of one-dimension variables: X=(X1, X2,… Xn) • Duration, transport protocol, service, outgoing/incoming packet/data size, time since last connection, if the remote host is visited before, etc • Aggregated features of connections • # connections/minute • Model of network behavior • a multivariate distribution P(X) • describes how likely a connection may happen

9. Network BehaviorProfile (II) • A network behavior profile is an approximation of the multivariate distribution P(X) • Quantify the resolution of each variable • Time-of-Day: day time/night; Day-of-Week: weekday/weekend • Select a subset of one-dimensional marginal and conditional distributions for approximating the multivariate distribution • P(X)=P(X1)P(X2)P(X3|X2) • Use a set of histograms to model one-dimensional distributions • Histograms: nonparametric, each to update

10. Proof-of-Concepts • We do not have concrete results for anomaly detection. • We need to find features which can be used to differentiate normal and anomalous network behavior. • Outgoing connections • New targets • Different services • Data: 2 weeks (11/09/03-11/25/03) tcpdump traces of our group (40 active hosts) • We will show network behavior of 4 end hosts which indicate some possible ways to do network anomaly detection.

11. Network Behavior: TCP Connection Speed

12. Network Behavior: New Targets

13. Network Behavior: Services

14. Discussion • Is it possible to differentiate between normal and anomalous network behavior of end hosts? • Network behavior of most end hosts are relatively stable? • Client vs. Server • New service release • Planet lab hosts • Coordination among edge networks • What information to share? • How to make decision based on shared information? • Statistical learning theory for anomaly detection • Most data is normal behavior • Online update/detection • Trace collection • Departmental/campus network • Commercial ISPs?

15. Related Work • Virus Throttle [williamson03implementing] • Limit/Watch the speed of connection made by an end host to detect if it’s compromised • Static: 1 connection/second • Only look at connection speed • Implemented at end hosts: maybe removed by malicious code • Online Fraud Detection [lambert00detecting] • Online data mining of a stream of transactions for customer patterns • fraud detection applied to cell phones and credit cards • Honeycomb [kreibich03honeycomb] • Honeypots: Decoy computing resources set up for monitoring and logging malicious activities • String-based pattern detection

16. Summary • Problem • Self-propagating malicious code is big threat to Today’s Internet • Idea • Monitor network behavior of local hosts • Prevent compromised local hosts from infecting others • Generate signatures based on traffic from those hosts • Approach • Collaborative online passive monitoring at edge networks • Redirect traffic to honeypots to create signatures • Future work • Investigate anomaly detection algorithms on real world data • Study coordinated analysis algorithms • Efficient passive monitoring mechanism

17. References • [moore03internet] • Internet Quarantine: Requirements for Containing Self-Propagating Code • http://www.caida.org/outreach/papers/2003/quarantine/worm-infocom03.pdf • [williamson03implementing] • Implementing and Testing a Virus Throttle • http://www.hpl.hp.com/techreports/2003/HPL-2003-103.pdf • [lambert00detecting] • Detecting Fraud in the Real World • http://cm.bell-labs.com/stat/doc/hmds.pdf • [kreibich03honeycomb] • Honeycomb – Creating Intrusion Detection Signatures Using Honeypots • http://nms.lcs.mit.edu/HotNets-II/papers/honeycomb.pdf