Network-based and Attack-resilient Length Signature Generation for Zero-day Polymorphic Worms

1. Network-based and Attack-resilient Length Signature Generation for Zero-day Polymorphic Worms Zhichun Li1, Lanjia Wang2, Yan Chen1and Judy Fu3 1 Lab for Internet and Security Technology (LIST), Northwestern Univ. 2 Tsinghua University, China 3 Motorola Labs, USA

2. The Spread of Sapphire/Slammer Worms

3. 1010101 10111101 11111100 00010111 Limitations of Exploit Based Signature Signature: 10.*01 Traffic Filtering Internet Our network X X Polymorphism! Polymorphic worm might not have exact exploit based signature

4. Vulnerability Signature Work for polymorphic worms Work for all the worms which target the same vulnerability Vulnerability signature trafficfiltering Internet X X Our network X X Unknown Vulnerability Better!

5. Benefits of Network Based Detection • At the early stage of the worm, only limited worm samples. • Host based sensors can only cover limited IP space, which might have scalability issues. Internet Gateway routers Our network Host based detection Early Detection!

6. Design Space and Related Work Network Based Host Based • Most host approaches depend on lots of host information, such as source/binary code of the vulnerable program, vulnerability condition, execution traces, etc. Exploit Based Vulnerability Based

7. Outline Motivation and Related Work Design of LESG Problem Statement Three Stage Algorithm Attack Resilience Analysis Evaluation Conclusions

8. Basic Ideas • At least 75% vulnerabilities are due to buffer overflow • Intrinsic to buffer overflow vulnerability and hard to evade • However, there could be thousands of fields to select the optimal field set is hard Overflow! Protocol message Vulnerable buffer

9. Framework ICDCS06, INFOCOM06, TON

10. LESG Signature Generator

11. Outline Motivation and Related Work Design of LESG Problem Statement Three Stage Algorithm Attack Resilience Analysis Evaluation Conclusions

12. Field Hierarchies DNS PDU

13. Length-based Signature Definition 100 Length Signature (Name,100) Name Type Class TTL RDlength RDATA Length Signature RDATA Vulnerable Signature Set {(Name,100), (Class,50), (RDATA,300)} “OR” relationship Ground truth signature (RDATA,315) Buffer length!

14. Problem Formulation Worms which are not covered in the suspicious pool are at most  Suspicious pool LESG Signature Normal pool  Minimize the false positives in the normal pool With noise NP-Hard!

15. Outline Motivation and Related Work Design of LESG Problem Statement Three Stage Algorithm Attack Resilience Analysis Evaluation Conclusions

16. Stages I and II Trade off between specificity and sensitivityScore function Score(COV,FP) COV≥1%FP≤0.1% Stage I: Field Filtering Stage II: Length Optimization

17. Stage III • Find the optimal set of fields as the signature with high coverage and low false positive • Separate the fields to two sets, FP=0 and FP>0 • Opportunistic step (FP=0) • Attack Resilience step (FP>0) • The similar greedy algorithm for each step

18. Stage III (cont.) Name Type Class TTL Comments RDATA Stage ICOV0≥1%FP0≤0.1% Residual coverage≥5% 50% 0.05% (RDATA,300) [50%,0.05%] (Name,100) [40%,0.03%] (Class,50) [35%,0.09%] (Comments,2000) [10%,0.1%] suspicious normal

19. Stage III (cont.) Name Type Class TTL Comments RDATA Stage ICOV0≥1%FP0≤0.1% Residual coverage≥5% 50% 0.05% {(RDATA,300)} (Class,50) [25%,0.02%] (Name,100) [3%,0.08%] (Comments,2000) [1%,0.05%] suspicious normal

20. Stage III (cont.) Name Type Class TTL Comments RDATA Stage ICOV0≥1%FP0≤0.1% Residual coverageγ≥5% (50+25)% (0.05+0.02)% {(RDATA,300),(Class,50)} (Class,50) [25%,0.02%] (Name,100) [3%,0.08%] (Comments,2000) [1%,0.05%] suspicious normal

21. Attack Resilience Bounds • Depend on whether deliberated noise injection (DNI) exists, we get different bounds • With 50% noise in the suspicious pool, we can get the worse case bound FN<2% and FP<1% • In practice, the DNI attack can only achieve FP<0.2% • Resilient to most proposed attacks (proposed in other papers)

22. Outline Motivation and Related Work Design of LESG Problem Statement Three Stage Algorithm Attack Resilience Analysis Evaluation Conclusions

23. Methodology • Protocol parsing with Bro and BINPAC (IMC2006) • Worm workload • Eight polymorphic worms created based on real world vulnerabilities including CodeRed II and Lion worms. • DNS, SNMP, FTP, SMTP • Normal traffic data • 27GB from a university gateway and 123GB email log

24. Results • Single/Multiple worms with noise • Noise ratio: 0~80% • False negative: 0~1% (mostly 0) • False positive: 0~0.01% (mostly 0) • Pool size requirement • 10 or 20 flows are enough even with 20% noises • Speed results • With 500 samples in suspicious pool and 320K samples in normal pool, For DNS, parsing 58 secs, LESG 18 secs

25. Conclusions A novel network-based automated worm signature generation approach • Work for zero day polymorphic worms with unknown vulnerabilities • First work which is both Vulnerability based and Network based using length signature for buffer overflow vulnerabilities • Provable attack resilience • Fast and accurate through experiments