Secure In-VM Monitoring Using Hardware Virtualization

1. Secure In-VM Monitoring Using Hardware Virtualization Monirul Sharif, Wenke Lee, Weidong Cui, and Andrea Lanzi Presented by Tyler Bletsch

2. Introduction • Problem: Kernel level rootkits can kill security monitoring software • Solution: Virtualization allows the monitor to live outside the VM • Problem: Out-of-VM monitoring can be very expensive • Why?

3. Out-of-VM monitoring Inspection DP Program data CP Program code K Hook DK Hook data A Adversary program H Handler CM Monitor code DM Monitor data R Response

4. Problem with out-of-VM monitoring • Significant performance impact • VM context switch is expensive • Introspection is hypercall-driven: expensive • We want a monitor without these problems • Performance guarantees: • (P1) Fast invocation: No privilege change for handlers • (P2) Data read/write at native speed: No penalty to access DM or DP

5. Solution: Secure In-VM Monitoring DP Program data CP Program code K Hook DK Hook data A Adversary program H Handler CM Monitor code DM Monitor data R Response

6. Security guarantees • Run the monitor inside the VM context, but provide out-of-VM-style security guarantees: • (S1) Isolation of monitor code CM and data DM: Adversary can’t access the monitor at all • (S2) Designated point for switching into CM: There’s only point entry point (tiny attack surface) • (S3) The handler is called iff the hook is triggered: The hook calls the handler; nobody else can • (S4) Behavior of M is not maliciously alterable: No dependency on monitored data or code

7. How to achieve this? • Separate address space with a one-way page table for the monitor • Small entry/exit gates into this address space • Invocation checker verifies that the call is legit

8. The SIM address space SIM Data/Code • The monitor itself • Visible only within SIM address space Invocation checker • Verifies call chain is legit • Visible only in SIM space Entry/exit gates • Visible in both • Writable only in SIM space • Tiny, well crafted Kernel code/data • Not executable in SIM space(can't accidentally run insecure code)

9. Changing the address space • CR3 register indicates page table directory • Hypervisor controlled • To avoid hypercall, use Intel CPU's "CR3_TARGET_LIST" to freely switch between: • P_SHADOW: Process's normal space • SIM_SHADOW: Restricted SIM space

10. Entry/exit gates • Only point of entry for SIM address space • Entry: • Disable interrupts • Save CPU state to the stack • Switch address space • Re-disable interrupts • Switch stack to a SIM-restricted one • Run invocation checker • Exit: • Restore stack, page table, CPU state • Re-enable interrupts • Jump to return point (usually the point of origin)

11. Invocation checker • Obviously can't rely on stack • Use CPU debug registers to track jump history • Lock these registers from modification by VM

12. Security monitor • Can't rely on untrusted code • No kernel functions, dynamic libraries, etc. • Need to traverse kernel data structuresjust like the out-of-VM technique (VMI) • Hardware access granted by hypervisor API

13. Performance requirements met? • (P1) Fast invocation: Yes: No privilege change needed; just a page table switch • (P2) Data read/write at native speed: Yes: Pages are mapped, accessed natively

14. Security requirements met? • (S1) Isolation of monitor code CM and data DM: Yes: Monitor memory isn't mapped to the process space • (S2) Designated point for switching into CM: Yes: Entry/exit points are the only way in • (S3) The handler is called iff the hook is triggered: Yes: Hook calls handler; calls pass the invocation checker • (S4) Behavior of M is not maliciously alterable: Yes: Untrusted code is non-executable in SIM space

15. Ah, but what if they... • Jump into the entry gate after the interrupt disable instruction (CLI)? • It gets re-disabled again after the page table switch • Switch to SIM_SHADOW directly? • The next instruction is no longer executable • But what if they do it right before a SIM page? • Any page after an unallocated address is made non-executable

16. Implementation • Linux KVM host, Windows XP guest • To initialize from a trusted boot state, a driver: • Reserves virtual address space for entry/exit gates • Has the hypervisor construct a SIM page table • Loads the security monitor into kernel space andhas the hypervisor remap it into SIM space • Informs the hypervisor of each hook and handler;hypervisor generates entry/exit routines accordingly • Memory protection: • Don't check every time CR3 is changed,check new page table entries as they are created

17. Evaluation: Monitor invocation • Time to enter and exit a no-op handler • SIM: 0.469 μs • Out-of-VM: 5.055 μs • Factor of 10.7 improvement

18. Evaluation: Process creation monitor • Hook process creation • Traverse kernel data to find process name • Process must be on whitelist to proceed • If not, skip and return failure • Average time for process creation: • Traditional In-VM approach: 3.487 μs • SIM approach: 3.967 μs (+13.7%) • Out-of-VM approach: 28.039 μs (+690.5%)

19. Evaluation: System call tracing • Make note of every system call, emit trace every 10,000 calls • Macrobenchmarks from PCMark 05 suite

20. Conclusion • SIM: The best of both worlds • Security of out-of-VM solution • Performance of traditional in-VM monitoring

21. Concerns • Very difficult to develop monitors • No libraries • No system calls • Porting a full-fledge anti-malware package? • Interrupt race condition in entry code? Normal entry CLI PUSHA MOV EAX, SIM_SHADOW MOV CR3, EAX CLI MOV [P_ESP] ESP MOV ESP, [SIM_ESP] JMP INVOKATION_CHECK Address space switch Attacker entry Interrupt now?

22. Discussion