1 / 28

In-Band Detection of Virtual Machines

In-Band Detection of Virtual Machines. Estefan Ortiz & Cory Hayes Computer Science and Engineering Graduate Operating Systems December 16, 2011. Introduction.

toshi
Download Presentation

In-Band Detection of Virtual Machines

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. In-Band Detection of Virtual Machines Estefan Ortiz & Cory Hayes Computer Science and Engineering Graduate Operating Systems December 16, 2011

  2. Introduction • Malicious programs (malware) need to know if they are in a virtual environment so they can modify their behavior and avoid detection • Related work • Red Pill Tests: Examine byte-level behavior of instructions for physical and emulated CPUs. If any disagreements in output, create one or more “red pills” that can avoid detection • SubVirt: Virtual machine-based rootkit installed underneath host OS that runs OS as a guest to remain nearly undetectable

  3. Our Approach • Similar to Red Pill and SubVirt, but client-server based • Idea: Instead of monitoring system call discrepancies, analyze network data sent to/from physical and virtual machines • Goal: Determine if there are sufficient differences in network traffic to detect if aclient/server is being run on a virtual machine

  4. Goal Byte 0 Byte k1 Difference Found Byte k2 Byte n Client <-> Native TCP/IP Packet Client <-> Virtual Machine TCP/IP Packet

  5. General Setup

  6. Actual Setup Host Server (Apache) Port Switch Client Functions as the “MITM” Port Network output saved for analysis Wireshark

  7. Experiment Setup • Using Wireshark, capture and compare the raw info of TCP/IP packets sent back and forth between a client and a physical/virtual server running Apache • Bits 1-160: IP • Remainder: TCP • Virtual machine OS matches the OS of the host (Ubuntu-Ubuntu, Vista-Vista) • Use a small set of Matlab commands to send regular and malformed packets • Dynex 5-port 10/100/1000 Gigabit Ethernet Switch

  8. Sample Captured Wireshark Output 8th Packet sent between Client & VM running Apache VM Host Client 8th Packet sent between Client & Host running Apache

  9. Metrics Bit Difference Comparison: Fractional Hamming distance between two packets

  10. Metrics (cont.) Round trip time: Time from SYN request sent by client to received ACK from server

  11. Metrics (cont.)* Pairwise Packet Length Comparison: Number of concurrent packet pairs that differ in length

  12. Experiment #1 • Client: Windows Vista (4GB RAM, 2.6GHz) • Server: Ubuntu 11.04 32-bit w/ Apache Web Server 2.2 • Server: Host OS Ubuntu: VirtualBoxw/ Ubuntu running Apache • On isolated switch network (no other traffic)

  13. Exp. #1: Frac. Hamming Distance

  14. Exp. #1: Round-trip Timing

  15. Example: Packet #9 These bits correspond to the header length & flags in the TCP header

  16. Experiment #2 • Client: Mac (4GB RAM, 2.4GHz, MacOSX 10.6.8) • Server: Windows Vista 32-bit w/ Apache Web Server 2.2 • Server: Host OS Windows Vista: VirtualBoxw/ Windows Vista running Apache • On isolated switch network (no other traffic)

  17. Exp. #2: Frac. Hamming Distance

  18. Exp. #2: Round-trip Timing

  19. Example: Packet #4 Destination Address in IP header Flags in TCP header

  20. Experiment #3 • Client:Windows Vista (4GB RAM, 2.6GHz) • Server: Ubuntu 11.04 32-bit w/ Apache Web Server 2.2 • Server: Host OS Ubuntu: VirtualBoxw/ Ubuntu running Apache • Both client and server on CVRL subnet (at ~3:00 am)

  21. Exp. #3: Frac. Hamming Distance

  22. Exp. #3: Round-trip Timing

  23. Example: Packet #3 Destination Address in IP header

  24. Experiment #4 Client Host Server (Apache) F I R E W A L L Port Internet Port Sprint Mobile Hotspot ND/CVRL subnet

  25. Experiment #4 • Client: Windows Vista (4GB RAM, 2.6GHz) • Server: Ubuntu 11.04 32-bit w/ Apache Web Server 2.2 • Server: Host OS Ubuntu: VirtualBoxw/ Ubuntu running Apache • Could not monitor packet information; only ping tests • Varied number of bytes sent using ping • Performed 100 per fixed byte amount • Calculated avg. & std. dev • Executed at ~3:30 am

  26. Exp. #4: Ping Timing

  27. Conclusion • Examined packet information from a high level (packet-length) down to specific bit difference comparisons • Packet length provided no insight • Timing tests didn’t provide conclusive evidence of a connection to a virtual machine • Fractional hamming dist. provided first level of insight • Further analysis of differences at the bit level provided clues where to look for VM traces

  28. Future Direction • Experiments 1-3 were conducted under somewhat “ideal” scenarios • More realistic approach would be packet analysis on multi-hop connections with knowledge of which sections of the TCP/IP packets to monitor

More Related