Process Coloring: an Information Flow-Preserving Approach to Malware Investigation

1. NICIAR PI Meeting, Boston, MA, September 19, 2007 Process Coloring: an Information Flow-Preserving Approach to Malware Investigation Eugene Spafford, Dongyan Xu(Presenter) Department of Computer Science and Center for Education and Research in Information Assurance and Security (CERIAS) Purdue University Xuxian Jiang Department of Information and Software Engineering George Mason University

2. Motivation • Internet malware remains a top threat • Malware: virus, worms, rootkits, spyware, bots…

3. Motivation

4. The Challenge: Enabling Timely, Efficient Malware Investigation • Raising timely alerts to trigger a malware investigation • Identifying the break-in point of the malware • Reconstructing all contaminations by the malware Break-in point Break-in point trace-back Contamination reconstruction Log Log External detection point Infection Detection Time State-of-the-art log-based intrusion investigation tools

5. Limitations of Today’s Tools • Long “infection-to-detection” interval • Entire log needed for both trace-back and reconstruction • Questionable trustworthiness of log data Break-in point Break-in point trace-back Contamination reconstruction Log ? Log ? External detection point Infection Detection Time State-of-the-art log-based intrusion investigation tools

6. Log Technical Approach: Process Coloring • Key idea: propagating and logging malware break-in provenance information (“colors”) along OS-level information flows • Existing tools only consider direct causality relations without preserving and exploiting break-in provenance information Virtual Machine … Log Monitor Apache MySQL DNS Sendmail Attacker Guest OS Logger Virtual Machine Monitor (VMM) Runtime alert triggered by log color anomalies

7. New Capabilities Enabled by Process Coloring Capability 1: Color-based malware warning Initial coloring s30sendmail s30sendmail s55sshd s55sshd Syscall Log s45named s45named init rc s80httpd s80httpd • /etc/shadow • Confidential Info httpd netcat Capability 3: Color-based log partition for contamination analysis Local files /bin/sh Capability 2: Color-based identification of malware break-in point Coloring diffusion wget Rootkit

8. Color-Based Malware Warning ... BLUE: 673["sendmail"]: 5_open("/proc/loadavg", 0, 438) = 5 BLUE: 673["sendmail"]: 192_mmap2(0, 4096, 3, 34, 4294967295, 0) = 1073868800 BLUE: 673["sendmail"]: 3_read(5, "0.26 0.10 0.03 2...", 4096) = 25 BLUE: 673["sendmail"]: 6_close(5) = 0 BLUE: 673["sendmail"]: 91_munmap(1073868800, 4096) = 0 ... RED: 2568["httpd"]: 102_accept(16, sockaddr{2, cbbdff3a}, cbbdff38) = 5 RED: 2568["httpd"]: 3_read(5, "\1281\1\0\2\0\24...", 11) = 11 RED: 2568["httpd"]: 3_read(5, "\7\0À\5\0\128\3\...", 40) = 40 RED: 2568["httpd"]: 4_write(5, "\132@\4\0\1\0\2\...", 1090) = 1090 … RED: 2568["httpd"]: 4_write(5, "\128\19Ê\136\18\...", 21) = 21 RED: 2568["httpd"]: 63_dup2(5, 2) = 2 RED: 2568["httpd"]: 63_dup2(5, 1) = 1 RED: 2568["httpd"]: 63_dup2(5, 0) = 0 RED: 2568["httpd"]: 11_execve("/bin//sh", bffff4e8, 00000000) RED: 2568["sh"]: 5_open("/etc/ld.so.prelo...", 0, 8) = −2 RED: 2568["sh"]: 5_open("/etc/ld.so.cache", 0, 0) = 6 Capability 1: Color-based malware warning: “unusual color inheritance”

9. httpd index.html index.html Color-Based Malware Warning • Another example: “color mixing” RED: 1234 ["httpd"]: … RED: 1234 ["httpd"]: … RED: 1234 ["httpd"]: … RED+BLUE: 1234 ["httpd"]: system call to read file index.html httpd bind cp defaced.html index.html

10. Efficiency through Process Coloring Capability 2: Color-based break-in point identification Capability 3: Color-based log partitioning

11. Impact of Success • How will it benefit the NIC? • Accountability of NIC cyber infrastructure • Readiness against current and emerging malware threats (e.g., botnets, rootkits, spyware) to NIC • Protection of NIC critical data, information, and computation activities • Reduction of NIC human labor in malware investigation

12. Evaluation Metrics • Timeliness • Malware infection-to-warning interval • Efficiency • Percentage of log reduction for malware contamination reconstruction • Accuracy • False positive rate of malware warning • False negative rate of malware warning • Correctness of malware action graphs

13. Object and process relationships in Linux analyzed. create, mkdir, link create <s1, o1> color(o1) = color(s1) CREATE fork, vfork, clone create <s1, s2> color(s2) = color(s1) color(s1) = color(s1)υcolor(o1) read <s1, o1> read, readv, recv READ read <s1, s2> ptrace color(s1) = color(s1)υcolor(s2) color(o1) = color(s1)υcolor(o1) write <s1, o1> write, writev, send WRITE write <s1, s2> Ptrace, wait, signal color(s2) = color(s1)υcolor(s2) destroy <s1, o1> unlink, rmdir, close DESTROY destroy <s1, s2> exit, kill Work in Progress: Color Diffusion Modeling (Month 1-6) • Color Diffusion Model syscalls Operation Diffusion

14. Color diffusion and logging implemented on Xen. A demo is on-line at: http://cairo.cs.purdue.edu/projects/pc/pc-demo.html Work in Progress: Process Coloring for Client and Server Side Malware Investigation (Month 2-18) • Server-side malware investigation • Consolidated server environment with independent server applications • “Clustered” information flows partitioned by server applications • Color mixing highly unlikely between applications • Client-side malware investigation • Inter-dependent client applications (e.g., text editor  compiler; latex  dvips  ps2pdf) • More inter-application information flows • Legal color mixing exists

15. A number of client-side applications are being tested (e.g., Skype, Firefox). + FTP FTP Quick Tax Quick Tax Work in Progress: Process Coloring for Client and Server Side Malware Investigation (Month 2-18) • A motivating example of client-side process coloring Time

16. Technology Transfer Plan • Potential adopters • Computer forensics/malware investigators and researchers • System administrators • Anti-malware software companies • Open source communities (e.g., XenSource) • Software release and documentation • Presentations and demos to potential NIC adopters • Presentations and demos to anti-malware software companies (Symantec, Microsoft, VMware)

17. Thank you! For more information about the ProcessColoring project: http://cairo.cs.purdue.edu/projects/pc PC@cs.purdue.edu