Worms and Worm Mitigation

1. Worms and Worm Mitigation Saman Amarasinghe Associate Professor, MIT EECS/CSAIL CTO, Determina Inc.

2. Outline • Worm Basics • Program Shepherding • Intrusion Prevention Systems • Anatomy of an Attack • The Determina Story

3. Outline • Worm Basics • Program Shepherding • Intrusion Prevention Systems • Anatomy of an Attack • The Determina Story

4. What is a worm? • Virus - a code segment which replicates by attaching copies to existing executables. • Self-replication • Requires a host program as a carrier • Activated by external action • Worm - a program which replicates itself and causes execution of the new copy. • Self-replication • Self-contained; does not require a host • Activated by hijacking or creating a process • Virus - a code segment which replicates by attaching copies to existing executables. • Worm - a program which replicates itself and causes execution of the new copy.

5. Types of Worms • E-mail based worm • The payload arrives as an e-mail attachment • Payload gets executed • Social Engineering or • Program vulnerability • Self propagate • Memory based worm • The payload arrive as packets on an open port • Takes advantage of a program vulnerability • Hijacks the program • Self propagate

6. History of Worms 1982 PARC “Worms as an administrative tool” • Perform legitimate tasks on a distributed system 1987 “Christmas Tree Exec” • attacked IBM mainframes • E-mail with a program that displayed a x-mas tree, but also send it to everyone in the user’s address book 1988 “The Morris Worm” • First true internet worm • Attacked both Sun and DEC unix systems • Used TCP/IP, attacked known OS bugs and admin flows • Automatic penetration, activation and propagation • Brought down the Internet for the first time ………. 2001 Code Red Worm 2003 Slammer Worm 2003 Blaster Worm 2004 Sasser Worm 2004 Witty Worm

7. Today’s Worm Threat • “One of the biggest threats to the modern information infrastructure” • Characteristics of Modern Worm Threat • Relatively rare (two a year) • Extreme rapid propagation • human vs. machine speed • Infects millions of systems • May or may not cause real damage • Catastrophic situations for many enterprises • Example: Blaster and CSX

8. Worldwide Code Red Infections 700,000 machines infected $2-2.9 billion in damage (Computer Economics) $200 million in damage per day during attacks

9. Outline • Worm Basics • Program Shepherding • Intrusion Prevention Systems • Anatomy of an Attack • The Determina Story

10. Program Shepherding Outline • Memory based attacks • Motivating Example: A buffer overflow attack • Dissecting the lifecycle in the penetration phase • Overview of Program Shepherding • DynamoRIO basics • Technique 1: Restricted Code Origins • Technique 2: Restricted Control Transfer • Technique 3: Uncircumventable Sandboxing • Self Protection

11. Memory Based Attacks • Attack that that directly enters an application, corrupts the application’s memory and hijack the application • Types of memory based attacks • Buffer Overflow • Heap Manipulation • Format String Vulnerabilities • Shatter Attacks • Return to Libc Attacks • Danger of Memory-based attacks • Once the application is hijacked, the attack can do anything that the application can do • Modern server applications run with a lot of privileges

12. Network Arguments: … Return Address Local Variables: … Application Data Argument: h Return Address Local Variables: tmp Local Variables: URL Instructions Operating System An Example: Buffer Overflow Attack Code Stack handle_URL(handle * h) { char url[64]; … char * tmp =geturl(h) strcpy(url, tmp); … }

13. An Example: Buffer Overflow Attack Network Program Counter: Executes the Program Instruction by Instruction Application Data Instructions Operating System

14. An Example: Buffer Overflow Attack Code Stack handle_URL(handle * h) { char url[64]; … char * tmp =geturl(h) strcpy(url, tmp); … } Arguments: … Return Address Local Variables: … Argument: h Return Address Local Variables: tmp Local Variables: URL URL: http://cag.lcs.mit.edu/~saman/tmp/doitnow.pl?helper=“foobar”myid=123453;junk=23#42@$@FFNLQ!_@##*RFNL!~@#+

15. An Example: Buffer Overflow Attack Code Stack handle_URL(handle * h) { char url[64]; … char * tmp =geturl(h) strcpy(url, tmp); … } Arguments: … Return Address Local Variables: … Argument: h Return Address Local Variables: tmp Local Variables: URL URL: http://cag.lcs.mit.edu/~saman/tmp/doitnow.pl?helper=“foobar”myid=123453;junk=23#42@$@FFNLQ!_@##*RFNL!~@#+

16. Enter An Example: Buffer Overflow Attack Network Application Data Instructions Operating System

17. An Example: Buffer Overflow Attack Code Stack handle_URL(handle * h) { char url[64]; … char * tmp =geturl(h) strcpy(url, tmp); … } Arguments: … 0x12FA2 lea %ecx%eax sub 0x1c(%edi) %eax movzx 0x8(%edi) %ecx shl $0x07 %ecx cmp %eax %ecx ………… Return Address Local Variables: … Malicious code segment Argument: h Return Address 0x12FA2 0x12FA2 Local Variables: tmp Local Variables: URL URL: http://0011101101101110110100010101101011010101101101 10110110110110101011010101010110101011010101010110...

18. Enter Hijack Attack An Example: Buffer Overflow Attack Network Application Data Instructions Operating System

19. Restricted Execution Environment Stop Applications From Being Hijacked When to stop an attack? • Enter • Monitoring is simple • Port monitoring or system call monitoring • Don’t know good guy from bad guy • only known criminals can be identified • Even known bad guys are hard to detect • encrypted channels • Attack • Monitoring can be done • System call monitoring • Hard to distinguish between actions of a normal program vs. a compromised program • Leads to false positives ABI • Hijack • “Catch in the act of criminal behavior” • All programs follow strict conventions • ABI (Application Binary Interface) • The Calling Convention • Currently no enforcement • All attacks violate some of these conventions • Enforcing conventions • Systematically catch an entire class of attacks • No false positives • Catch them before they do ANY bad activity  no attack code is ever run • Conventional Wisdom: Impossible to do without a large performance penalty • Need to be inside the application • Need to monitor activity at a very fine-grain – each instruction at a time • Overhead will be overwhelming • Program Shepherding lets you do just that! • Able to amortize the cost of enforcement, eliminating the overhead Processor Execution Environment

20. call call jmp jmp br br ret How Program Shepherding Work? Program Counter: Executes the Program Instruction by Instruction Run-time System Never Let go of the Program Counter Code Cache Program Restricted Code Origins: Is this code came from a code page? Restricted Code Origins: Is this code came from a code page? Restricted Code Origins: Is this code came from a code page? Restricted Control Transfer: Is it legal to go from here to there? Restricted Control Transfer: Is it legal to go from here to there?

21. A Basic Interpreter next VPC • Never give up the Program Counter (PC) • PC is in the interpreter fetch next instruction update VPC decode execute Validate exception handling Instruction Interpreter

22. Performance of the Basic Interpreter Slowdown factor

23. Trick I: Adding a Code Cache next VPC lookup VPC exception handling fetch block at VPC execute block emit block context switch BASIC BLOCK CACHE non-control-flow instructions

24. Adding a Basic Block CachePerformance Effect Slowdown factor Slowdown factor • Performance Problem: High cost of frequent context switches

25. Trick II: Linking next VPC lookup VPC exception handling fetch block at VPC link block emit block execute until cache miss context switch BASIC BLOCK CACHE non-control-flow instructions

26. Performance Effect of Basic Block Cache with direct branch linking Slowdown factor • Performance Problem: High cost of mispredicted indirect branches

27. Trick III: Efficient Indirect Branch Handling next VPC basic block builder context switch miss BASIC BLOCK CACHE miss indirect branch lookup non-control-flow instructions

28. Performance Effect of indirect branch linking • Performance Problem: inefficient code layout in code cache Slowdown factor

29. Trick IV: Picking Traces trace selector basic block builder START dispatch context switch BASIC BLOCK CACHE TRACE CACHE indirect branch lookup non-control-flow instructions non-control-flow instructions

30. Performance Effect of Picking Traces 300 Slowdown factor

31. Application ntdll.dll / *32.dll Monitor all Control Transfers • Must intercept all windows initiated control transfer events • Exceptions • Call backs in Windows • Asynchronous procedure calls • Setjmp/longjmp • Set thread context Code Cache Kernel Windows

32. Transparent to the Application • Application behavior cannot change! • No assumptions beyond the ISA and the OS interface • Actions of DynamoRIO is transparent to the application • Data Transparency • I/O Transparency • Transparent Exception Handling • Program Address Transparency

33. As code is copied to the code cache, check where it’s coming from Check the security policy only once Technique 1: Restricted Code Origins Code Cache Unmodified code pages D A Modified pages B E

34. Technique 1: Restricted Code Origins • Catches all the injected code attacks • Most of the popular attacks are of this type • What is left? • Malicious reuse of existing code • Change addresses used by return and indirect jump and indirect branch instructions • Much more difficult

35. An Example: Chained Call Attack Code Stack handle_URL(handle * h) { char url[64]; … char * tmp =geturl(h) strcpy(url, tmp); … } Arguments: … Fake arguments Return Address Local Variables: … 0x8A234 Fake arguments Argument: h Return Address 0x7F8B0 Local Variables: tmp Local Variables: URL Libraries 0x7F8B0 setuid() … 0x8A234 unlink() … URL: http://001110110110111011010001010110101101010110 10110110110110101011010101010110101011010101...

36. Technique 2:Restricted Control Transfers context switch BASIC BLOCK CACHE TRACE CACHE trace branch taken? indirect branch lookup non-control-flow instructions non-control-flow instructions Restrict based on source address, destination address, and/or transfer type

37. Technique 2:Restricted Control Transfers • Inter-Segment Indirect Calls and Jumps • Only to known function entry points • Only if the function is exported by the destination segment • Only if the function is imported by the source segment • Intra-Segment Jumps • Only within a known function or to a known function entry point • Intra-Segment Indirect Calls • Only to known function entry points

38. Technique 2:Restricted Control Transfers • Returns • Only to after a call instruction • If a direct call, called function should be the same as the function returning from

39. Technique 3: Un-circumventable Sandboxing • Typical problem with sandboxing: • If attacker gains control, can bypass checks • DynamoRIO-inserted sandboxing isun-circumventable • DynamoRIO enforces unique entry points

40. Protecting DynamoRIO Itself • DynamoRIO runs in the application’s address space • Must not allow application to manipulate DynamoRIO data or code cache • How? • Protect Dynamo data structures and the code cache • Sandbox system calls that can change protection and thread behavior

41. Memory protection

42. Memory protection

43. Outline • Worm Basics • Program Shepherding • Intrusion Prevention Systems • Anatomy of an Attack • The Determina Story

44. Evaluating an Intrusion Prevention Solution • Accuracy • Maintainability • Scalability • Coverage • Proactivity • Uncircumventability • Containment

45. 1. Accuracy • How much false positives • Frequency of False positives • Impact of a false positive • Why you need and IPS? • Availability vs. Privacy/Integrity

46. 2. Maintainability • What is a typical enterprise like? • How many machines, how many IT people? • Cost of operations… • How do you manage a large enterprise? • What impact maintainability? • Shelfware vs. deployed software

47. 3. Scalability • Requirements to run enterprise-wide… • Critical bottlenecks • Deployment / maintenance • Performance

48. 4. Coverage • What attacks are stopped? • What is the attack landscape? • Is there a map anywhere? • “I have N products, what do they stop and where are the holes?” • How to define the attack landscape? • Look at the past and extrapolate • Look at the vulnerabilities

49. Most Critical Vulnerabilities % of vulnerabilities Source: CVE, Microsoft Security Bulletins, 2003-2004

50. Good guys Patch like crazy Application Released With a bug Vulnerability announced Patch released Attack Released Bad guys Analyze patch & create attack 5. Proactivity • What information do you need to stop an attack? • Know the application • Know the vulnerability • Know the attack • Day-zero attacks. • Traditional Timeline of an Attack