Entropy Characteristics of Propagating Internet Phenomena

1. Entropy Characteristics of Propagating Internet Phenomena Alfonso Valdes SRI International Acknowledgement This research was partially sponsored by DARPA under Contract Number N66001-00-C-8058. The views expressed are those of the authors and do not necessarily reflect the views of the supporting agency.

2. Outline • Background • Detection • Efficient Iterative Algorithm for Entropy • Initial Results for Slammer Worm • Summary and Future Directions

3. Background • There have been numerous destructive Internet attacks that infect a vulnerable host and propagate from there to new targets (worms) • These have potential to saturate the entire vulnerable population in a brief time • Even sites without vulnerability suffer reduced QOS as worm traffic consumes bandwidth • Timely detection a the ISP or higher level may enable containment and control damage

4. Detection • Detection relies on enterprise-level IDS • Does the IDS have a signature? • Difficult to distinguish local from global • Administrators rely on phone net to get big picture • Exchanging IDS alert content may compromise confidential information

5. Detection (2): ISP Level Issues • Can we use conventional IDS? • Probably not,traffic rate to high • Cross-site alert aggregation? • Possibly, if the enterprise-level alerts are generated in the first place • Typically limited to a subscriber base • Confidentiality?

6. Detection (3): Worms and Entropy • Hypothesis: Propagating phenomena affect the entropy of Internet traffic • More diverse client (source IP) set • More concentrated service (dest port) set • Effect does not depend on conventional IDS signature • This is visible at the enterprise level. • We conjecture it is visible at higher levels • Side Benefit: Detecting worms this way raises no confidentiality issues • Can we compute entropy in real time? • Expensive log calls • State space explosion

7. Efficient Iterative Algorithm • “It can be shown” the entropy change due to a new observation can be computed from the current entropy value with 1 or 2 log calls • Many of these have a very good Taylor Series approximation

8. Efficient Iterative Algorithm (2)

9. Algorithm (3): State Space Management • A periodic update cycle prunes and ages the state space • Max state space size can be configured • Aging keeps most recent and active states • It is hoped these are the more interesting states

10. Results for Slammer Worm • As conjectured, source IP entropy increases and dest port entropy decreases • Data is firewall log entries for rejected e2i UDP requests • Down spikes in source IP trace and coincident up spikes in dest port trace are scans (serendipitous discovery) • Port 137 dominates non-Slammer accesses

11. Results for Slammer Worm

12. Results for Slammer Worm, Port 137 Requests Removed

13. Summary • Conjectured impact of worms on Internet process entropy holds for Slammer • Higher source IP entropy • Will this be true at ISP view? • Lower dest port entropy • Likely to remain true at ISP level • Scans from a single source appear as spike anomalies (discovered but not anticipated) • Defined fast algorithm with bounded state space • Feasible at ISP?

14. Future Directions • Examine large ISP level repository • Real-time feasibility • Does the hypothesis still hold • Other data streams? • Return codes • IDS alert mix • Packet content