VMM Based Rootkit Detection on Android

1. VMM Based Rootkit Detection on Android Midterm Meeting Pete Bohman, Adam Kunk, Erik Shaw

2. Outline • The problem and why it is important • Our solution and why it is better • System Threat Defined • Kernel-mode Rootkitsyscall table hook • Preliminary Design • Defensive syscall integrity LKM • Android VMM • Preliminary Results

3. The Problem • Detecting rootkits on Android smart phones • This is important because: • Smart phone use is tremendously growing (with Android becoming the market share leader in 4Q 2010) • Phones are starting to be used like mini computers • Phones carry lots of sensitive data (more than a computer at times) • GPS location, contacts, text messages, call data, • People make purchases on their phones (billing info)

4. The Problem (cont.) • Rootkits are a major problem on any traditional monolithic operating system on our desktop computers • Android OS is built upon the Linux kernel • This means that many of the attack methods (LKM rootkits) that are targeted for the Linux OS may be applicable to Android • Currently, high power consumption is a major flaw in the existing prevention methods

5. Our Solution • Two part solution: • VMM layer to live below the guest Android OS • Layer below approach to ensure integrity of the LKM that lives as an extension to the kernel • This is necessary in the to avoid malicious rootkit countermeasures, such as corrupting or disabling a protection mechanism • Minimal execution in the VMM to preserve power • LKM that monitors the integrity of the syscall table and corresponding functions • Integrity checks at regular intervals

6. System Threat Defined • The syscall table can be hooked on Android • Hooking the syscall table is one of the most common actions performed by kernel-mode rootkits, and thus a prime place to look for rootkit activity

7. Preliminary Design: VMM • Android VMM lives a layer below the guest operating system, the Android kernel • Android VMM will check integrity of the LKM that monitors the syscall table

8. Preliminary Design: VMM (cont.) • Reproduce VMM design described in “Embedded VMM for Portable Virtual Machines” • Booting and Initialization • VMM image contains guest OSes as binary data • Enable Cache and MMU • Guest OS Loading • Load each OS at a separate physical address • Individual virtual machine state structure • Memory Management • Manage VMM page tables • Shadow page tables for guest OS

9. Preliminary Design: VMM (cont.) • Full virtualization through hardware virtualization extensions. • Modified QEMU ARMv7 CPU Emulator to trap to VMM upon privileged instruction execution. • Bhardwaj et al. A Choices Hypervisor on the ARM Architecture. Bhardwaj et al. • Kalla et al. Embedded VMM for Portable Virtual Machines

10. Preliminary Design (cont.) • LKM periodically checks integrity of syscall table and functions pointed to • Root of trust is placed within the VMM • The VMM checks integrity of this LKM from a layer below

11. Preliminary Results (boot time) • Boot times of normal Android (zImage) image versus the VMM (zVmm) image were measured. • The results on the next slide demonstrate the average of three boots for each image. • The Linux ‘time’ utility was used to obtain the ‘real’, ‘user’, and ‘sys’ running times of each boot. • The ‘boot time’ was measured as the time from booting the image in the Android emulator to the time it took for the emulator to boot up and unlock the initial screen.

12. Preliminary Results (boot time)

13. Preliminary Results (cont.) • Faux Rootkit LKM is currently hooking open(), close(), read(), and write() on load • This functionality will be used to obtain the GPS coordinates or the phone contacts list as an example of malicious software in action • Further investigation required to complete this behavior

14. Preliminary Results (cont.) • Protection LKM is still in design phase • Can leverage same build environment as Faux Rootkit LKM • Integrity checking options: • Must be loaded first and archive syscall table pointer on load to ensure no tampering (hard to guarantee load order without putting more responsibility on VMM, and this is still hard) • Compare syscall table in memory to syscall table on disk • Capabilities table with white-listed processes enforced • Would prevent user-mode access of GPS, call log database, and phone contacts database based on observed data from normal operation enforced as a rule