GuestGuard: Dynamic Kernel Tampering Prevention Using a Processor-Assisted Virtual Machine

1. Information & Computer Sciences University of Hawaii at Manoa Yoshiaki Iinuma GuestGuard:Dynamic Kernel Tampering PreventionUsing a Processor-Assisted Virtual Machine

2. Outline • Problems (Malware, OS, & Anti-Malware)‏ • GuestGuard (Solution)‏ • Conclusions

3. I. Problems • Malware • Kernel Tampering Malware (KTM)‏ • KTM Technologies • KTM Classification • OS Problems • Anti-Malware Problems

4. 1.1 Kernel Tampering Malware (KTM)‏ • Malware trying to manipulate kernel code and data • Compared with user-land malware • More Artful, Powerful, and Stealthy • More difficult to detect • Kernel-mode Rootkits: 46% increased in 2008(SOPHOS)‏ • Many security vendors indicate the rise of KTM. • The Only Target of GuestGuard

5. 1.2 KTM Technologies • Hardware Facility Perversion(SMM, APIC, Fast System Call)‏ • OS Facility Perversion (Device driver, Windows API, Registry)‏ • Hooking (Inline, Function table)‏ • Direct Kernel Object Manipulation[DKOM](PsActiveProcessHead, PsLoadedModuleList, EPROCESS)‏ More difficult to detect

6. 1.3 KTM Classification • Type I (HW facility perversion): modifies system resisters or other system components (BIOS). • Type II (OS facility perversion): modifies the kernel memory in a legitimate way. • Type III (Hooking): modifies the kernel memory that is not supposedto be changed (code and tables). • Type IV (DKOM): modifies the kernel memory that is supposed to be changed (data structures dynamically allocated) in an illegitimate way.

7. 2. OS Problems • Difficulty in preventing kernel space intrusion • OS facilities (e.g. Device Driver)‏ • Hardware Facilities (e.g. SMM, APIC)‏ • Software (OS) Vulnerability • Human Involvement ← Social Engineering • No restriction on kernel space processes • Malware can compromise the security system • Too much flexibility for processes • No distinction between malware and benign software

8. 3. Anti-Malware Problems • Limitation of dynamic prevention • Sometimes, only for detection • Possible circumventions

9. II. GuestGuard (Solution)‏ • KTM Characteristics • Design Goals • GuestGuard Overview • Implementation Details • Virtual Memory Virtualization • Evaluation • Performance • Functional Test

10. 1. KTM Characteristics • Modifying a code segment • Executing code in a data segment • Illegally accessing a kernel object or different process's address space Preventing them a strong Deterrence against KTM

11. 2. GuestGuard Design Goals • Kernel Tampering Prevention (Against KTM)‏ • Dynamic Prevention • Unknown Malware • Tamper Resistance • Low Overhead • Protection for Current Home Computing(Windows XP and Intel X86)‏

12. 3. Protection Mechanism Overview • Guest OS: Windows XP • Host OS: Linux • ISA: Intel x86 • VMM: KVM • CPU:Intel Core 2 Duo(Intel-VT)‏

13. 4.1. Virtual MemoryVirtualization • X86 Paging Mechanism • MMU • Configuration • CR0 (PE, PG, WP)‏ • CR3 (Page Directory Base)‏ • CR4 (PAE, PSE)‏ • Page Tables • read/write (R/W)‏ • user/supervisor (U/S)‏ • present/access/dirty

14. 4.2 Virtual Memory VirtualizationShadow Page Table • Guest Virtual → Host Physical • The processor does not refer to the guest page tables. • Write-protect guest page tables • Dynamic Detection • Tamper Resistance

15. 5.1 EvaluationPerformance Overhead • Futuremark PCMark05 (for home PC usage)‏ • CPU, Memory, HDD benchmark suites • PCMarks (Score) is calculated from a geometric mean of the individual test results • Sample PCMarks in 2005: 1,200 (low) ~ 5,500 (high)

16. 5.2 EvaluationFunctional Test Result • 11 test samples from www.rootkit.com • GuestGuard detected 6 samples • Currently not support Type I, II, IV • Worked well against Type III (Hooking)‏ • However, circumventable with memory mapping

17. III. Conclusions • Kernel Tampering Malware Prevention • Dynamic prevention • Tamper resistance • Low overhead • Overcame OS and Anti-malware problems • Works without any modifications to Windows • Worked very well for Type III (Hooking)‏ • Can overcome the memory mapping problem • Extensible to Type I, II, IV

18. Question?

19. Why Virtual Machine (KVM)? • Introduce a new security layer to the current existing computing environment • Tamper Resistance – provide isolation for a security system • Dynamic Detection – change the execution path of the guest • KVM allows the guest OS to run on the native processor. • Low overhead • Windows XP • Intel-VT or AMD-V(processor virtualization)‏ KVM QEMU

20. Windows Introspection • Protects: • Table: IDT, GDT, SSDT • Code: Interrupt vectors, System services, Loaded modules • Automatically detects their locations. • Extracts information directly from the guest registers and memory data structures. (no guest portions)‏ • Examples: • IDTR → IDT base address and size • Each IDT entry → Interrupt vector base address and size • FS → KPCR → KdVersionBlock → PsLoadedModuleList → All the loaded modules

21. System Shutdown • By Injecting Triple Faults. • The safest way (← malware is already running)‏ • Might lose user data, but recoverable. • Backup and snapshots • Damage from data loss < Damage from malware • Possible different reactions in the future.

22. Improvements:Against Memory Mapping • GuestGuard can be easily subverted. • X86 page protection is based on virtual memory • Solution: write-protecting a newly mapped page based on physical memory • #5 and #9 uses the mapping circumvention technique.

23. Improvements:Against Type II • Easy to detect by hooking OS legitimate functions • Difficult to decide whether a usage of a function is acceptable. (← Too much process flexibility)‏ • Should be dealt with by the OS • Solution for the filter driver perversion • Define preferable information flows for each I/O • Track the information flow of each I/O • Policies & Policy enforcement mechanism → GuestGuard • #3 perverts a filter driver.

24. Improvements:Against Type I • Not difficult to detect Type I. • Monitor a specific privileged instruction or procedure. • SMM rootkits: • SMI handler in SMRAM ← write-protection • SMI generation through Local APCI register ← write-protection • #4 is an SMM rootkit

25. Improvements:Against Type IV (DKOM)‏ • Hook functions to create and delete a kernel object. • Set write-protection on a kernel object. • Check the range of the current IP (object manager). • #6 uses DKOM

26. Bibliography • Fu rootkit. http://www.rootkit.com/board_project_fused.php?did=proj12. • N. A. Quynh and Y. Takefuji. A novel approach for a file system integrity monitor tool of xen virtual machine. • Keith Adams and Ole Agesen. A comparison of software and hardware techniques for x86 virtualization. • Starr Andersen. Microsoft technet: Part 3: Memory protection technologies. http://technet.microsoft.com/en-us/library/bb457155.aspx, September 2004. Technical Report • Alex Ho, Michael Fetterman, Christopher Clark, Andrew Warfield, and Steven Hand. Practical taint-based protection using demand emulation, April 2006 • Intel Corporation. Intel Virtualization Technology Specification for the IA-32 Intel Architecture, April 2005. • Intel Corporation. Intel 64 and IA-32 Architectures Software Developer’s Manual Volume 3A & 3B System Programming Guide, 2008.

27. Bibliography • Arati Baliga, Pandurang Kamat, and Liviu Iftode. Lurking in the shadows: Identifying systemic threats to kernel data. 2007 IEEE. • Paul Barham, Boris Dragovic, Keir Fraser, Steven Hand, Tim Harris, Alex Ho, Rolf Neugebauer, Ian Pratt, and Andrew Warfield. Xen and the art of virtualization. • Fabrice Bellard. Qemu, a fast and portable dynamic translator. In Proceedings of the 2005 USENIX Annual Technical Conference, 2005. • Jamie Butler and Greg hoglund. VICE catch the hookers! Black Hat USA, 2004. • Futuremark Corporation. Futuremark corporation PCMark05. http://www.futuremark.com/products/pcmark05/, 2005. • IBM Corporation. Ibm internet security systems x-force 2007 trend statistics. • IBM Corporation. Ibm internet security systems x-force 2008 mid-year trend statistics.

28. Bibliography • Symantec Corporation. Symantec internet security threat report trends for 2008. • B. D. Payne, M. Carbone, M. Sharif, and W. Lee. Lares: An architecture for secure active monitoring using virtualization. In SP ’08 • Mila Dalla Preda, Mihai Christodorescu, Somesh Jha, and Saumya Debray. A semantics-based approach to malware detection. • Mark E. Russinovich and David A. Solomon. Microsoft WINDOWS INTERNALS (4th Edition): Microsoft Windows Server 2003, Windows XP, and Windows 2000. Microsoft Press, 4th edition, January 2005. • F-Secure. Blacklight. http://www.f-secure.com/blacklight/. • T. Garnkel and M. Rosenblum. A virtual machine introspection based architecture for intrusion detection. 2003. • Hoglund Greg. ROOTKIT. http://rootkit.com/. • Flavio Lombardi and Roberto Di Pietro. Kvmsec: A security extension for linux kernel virtual machines. 2009. ACM.

29. Bibliography • Greg Hoglund and James Butler. Rootkits: Subverting the Windows Kernel. Addison-Wesley Professional, August 2005. • Markus Jakobsson and Zulfikar Ramzan. Crimeware: Understanding New Attacks and Defenses. Addison Wesley Professional, April 2008. • Bernhard Jansen, Hari-Govind. V. Ramasamy, and Matthias Schunter. Policy enforcement and compliance proofs for xen virtual machines. 2008. ACM. • Xuxian Jiang, Xinyuan Wang, and Dongyan Xu. Stealthy malware detection through vmm-based ”out-of-the-box” semantic view reconstruction. 2007. • A. Joshi, S. King, G. Dunlap, and P. Chen. Past and present intrusions through vulnerability specic predicates. October 2005. • Kaspersky Lab. Malware evolution 2008 kaspersky security bulletin. Technical report, 2009. • Jr. N. L. Petroni, T. Fraser, J. Molina, and W. A. Arbaugh. Copilot - a coprocessor-based kernel runtime integrity monitor. 2004.

30. Bibliography • Koichi Onoue, Yoshihiro Oyama, and Akinori Yonezawa. Control of system calls from outside of virtual machines. 2008. ACM. • Opc0de. Bypassing vice 2. http://rootkit.com/newsread.php?newsid=197, June 2004. • Sophos Plc. Security threat report: 2009. Technical report, 2009. • Qumranet. Main page: KVM - Kernel Based Virtual Machine. http://www.linux-kvm.org/page/Main_Page. • J. Rutkowska. Subverting vista kernel for fun and prot, August 2006. Blackhat. • A. Seshadri, M. Luk, N. Qu, and A. Perrig. Secvisor: a tiny hypervisor to provide lifetime kernel code integrity for commodity OSes. 2007.

31. Bibliography • w3schools. OS platform statistics: What is the trend in operating systems usage? http://www.w3schools.com/browsers/browsers_os.asp, Semptember 2009. • Yanfang Ye, Dingding Wang, Tao Li, and Dongyi Ye. Imds: Intelligent malware detection system. 2007. ACM. • Heng Yin, Zhenkai Liang, and Dawn Song. HookFinder: Identifying and understanding malware hooking behaviors. 2008. • Heng Yin, Dawn Song, Manuel Egele, Christopher Kruegel, and Engin Kirda. Panorama: Capturing system-wide information flow for malware detection and analysis. 2007. • Jeremy Z. Kolter and Marcus A. Maloof. Learning to detect malicious executables in the wild. 2004. ACM. • Qinghua Zhang and Doublas S. Reeves. Metaaware: Identifying metamorphic malware. ACSAC. 2007.

32. Windows Architecture

33. Windows IntrospectionDetails • IDT, GDT – IDTR, GDTR (base, limit)‏ • SSDT – fs → KPRCB → KdVersionBlock → KeServiceDescriptorTable • Interrupt Service Routines – IDT → each ISR code segment (base, limit)‏ • System Services – SSDT → each SS base address • Loaded Modules - fs → KPRCB → KdVersionBlock → PsLoadedModuleList • System service dispatcher – MSR: IA32_SYSENTER_CS→ SYSENTER code segment

34. kernel32::LoadLibrary()‏ Code Injection using WinAPI Target Process Malware Loader • OpenProcess()‏ • VirutalAllocEx()‏ • WriteProcessMemory()‏ • CreateRemoteThread()‏ Process ID Malicious code Start Dispatcher Target Process DLL loaderKernel32::LoadLibrary()‏ Malware Loader • CreateRemoteThread() or • SetWindowsHookEx()‏ Load DLL loader Malicious DLL Malware Loader Target Process • GetThreadContext()‏ • SetThreadContext()‏ Thread Thread Info New context Malicious Code

35. Malware Techniques IIILayered Device Driver Device Driver • A device driver comprises several layered drivers. • I/O request packets to communicate with each other • Any number of filter drivers can be added between the layers. • Can modify the behavior of an existing driver. • Keylogger, network sniffer Request Handler Filter Driver Class Driver Filter Driver Port Driver Filter Driver Bus Driver Hardware

36. Hiding Processes using Hooking • GetSystemInfo → NtQuerySystemInformation • Returns the linked list of the process information • Malware hooking this function could change the result. User Application (Taskmgr.exe)‏ Kernel32.DLLNtdll.DLL process process System ServiceDispatcher NtQuerySystemInformation SSDT process process malware Fake NtQuerySystemInformation

37. Malware Technique 5Inline Hook (Runtime Patching)‏ • Copy the target's function preamble to the trampoline function. • Write JMP destination address of the trampoline function. • Write JMP destination address of the malicious function. • Replace the original preamble with the far JMP instruction to the trampoline function. Trampoline Function Original code Copied Preamble Preamble JMP to Malicious Malicious Function Malicious Body JMP to original

38. Hooking Detection • Search for branches that fall outside of an acceptable range. (VICE, Blacklight)‏ • IAT: each loaded module containing imported functions has a defined start address in memory and a size. • IRP handler TBL: functions are within a given driver's address range. • SSDT: all the system services are contained in Ntoskernel.exe.

39. Other Hooking Detection • Integrity based detection • Keep hash values calculated from each protected executable image and function table • Periodically recalculate those hash values and compare them with their originals. • Find extra instructions executed by hooks. (PatchFinder)‏ • Keep the number of instructions of each function. • Periodically call each function and compare the results. • Using the x86 single step mode.

40. Malware Technique VDKOM • Direct Kernel Object Manipulation. • Windows manages all the kernel objects through the Object Manager. • Bypasses the Object Manager, thereby bypassing all the access checks on the object. • Extreamely hard to detect. • difficult to implement (must understand how, where, when a kernel object is created, deleted and modified as well as the object format)‏

41. Hiding Processes using DKOM FS KPRCB MaliciousETHREAD PsActiveProcessHead EPROCESS MaliciousEPROCESS EPROCESS Head Before Process List EPROCESS EPROCESS Head After MaliciousEProcess

42. DKOM DetectionCross-View Based • Find a system discrepancy through multiple views of the same system information. • Compare the result from a Windows API with the information extracted through other low level methods (e.g. directly check the underlying kernel objects)‏ • Disadvantages • Complexity to support all hardware • Duplication of some parts of OS • Possible bypassing techniques