Implementing an Untrusted Operating System on Trusted Hardware

1. Implementing an Untrusted Operating System on Trusted Hardware

2. Introduction • Tamper-resistant software • Combat piracy • Mobile code • However, SW-based solutions are easier to attack than HW-based solutions • To implement tamper-resistant hardware • Challenge: use untrusted OS to manage trusted HW

3. XOM: eXecute Only Memory • HW support for tamper-resistant software • Does not trust OS and memory content • Many design implications • Time-slicing • Process forking • User-level signal handling • Prototyped via SimOS

4. The XOM Trust Model • Protect against attackers who have physical access to the HW • Does not trust OS • OS performs only resource management • Only can perform denial-of-service attacks against its applications • Does not trust main memory • Values stored in memory are encrypted, along with their hashes of values and virtual addresses

5. Implications of the XOM Trust Model • XOM prevents programs from tampering with each other by placing them in separate compartments • Enforced by HW data tagging and cryptography • OS runs in a separate compartment • OS should be able to manage resources without having to interpret data values in transit • OS cannot read or modify process data

6. Related Work • Secure booting • CPU has a tamper-resistant secret • Used to authenticate the next layer (e.g., firmware) • Which in turn authenticates the next layer • XOM trusts neither OS nor memory • OS bugs cannot undermine the security of applications running on it • XOM can detect tampering of code or data at all times

7. The Original XOM Architecture • XOM uses tamper-resistant HW to protect a master secret that is unique for each processor • Which is used to encrypt keys that encrypt various software • All operations that use the master secret must be implemented on processor

8. The Original XOM Architecture • Master secret is used to support compartments • Each is immune to modification and observation

9. Implementing Compartments (1) • Use cryptography to implement compartments • XOM HW holds the private key • Each compartment uses XOM’s public key to encrypt its compartment key • Which is used to encrypt the compartment content • Compartment keys are stored in an encrypted program

10. Implementing Compartments (2) • Implications on SW distributions • A distributor must encrypt its software key with the public key of a XOM processor • Occurs when a user registers the software • Data generated by a program are isolated in its compartment • Data are encrypted when they leave the CPU chip • Cached data are stored in plain text

11. Implementing Compartments (3) • XOM Key Table • Maps compartment keys and ownerships • If a piece of data is encrypted, it’s in a compartment • NULL compartment • Not encrypted • Insecure data sharing among programs

12. Implementing Compartments (4) • To protect against tampering of data in memory • Use cryptographic hash to verify data integrity • A hash is generated each time a cache line is written to memory (a function of the cache line and its address) • Both hash and the cache lines are encrypted

13. Implementing Compartments (5) • New XOM HW instructions are introduced • enter xom • Decrypt the compartment key • Enter the key into the XOM Key Table • All instructions following enter xom must be decrypted before execution • exit xom • Compartment key to be unloaded from the XOM Key Table • Return to the NULL compartment

14. Implementing Compartments (6) • secure store, secure load • For non-NULL compartments • move from NULL, move to NULL

15. Symmetric Decryption Module Executable Code Asymmetric Decryption Module Currently executing XOM ID and Key XOM Key Table Implementing Compartments (7) Unencrypted Code Encrypted Code (sym) Encrypted Symmetric Key Private Key Main Memory XOM Processor Enter XOM Exit XOM

16. Handling Program State (1) • Since OS cannot access registers tagged by other compartments • Need a new way to save and restore contexts • New XOM instructions • save register • Encrypt the register • Create a hash of the register • Store both to memory

17. Hash Data Store encrypted data and hash to memory Encrypt Data Look up program key based on Tag OS XOM ID XOM Key Table Handling Program State (2) Data Tag

18. Handling Program State (3) • restore register • Takes an encrypted register value and hash • Verifies the hash • Restores them back to the original register, setting their ownership tags appropriately • To prevent the OS from replaying encrypted register values/hashes • Revoke the register key used to encrypt and hash register values each time a XOM compartment is interrupted

19. Handling Program State (4) • To protect memory from replay attacks • Store a hash of memory in a register • XOM currently rely on applications to track memory hashes • Otherwise, too expensive in terms of performance (25%) • Each virtual address needs a hash tree • Virtual to physical address mapping maintained by OS • XOM needs to interact with OS closely

20. Hash Data Tag Data Secure Store: Tag is copied from register to cache Writeback: Look up Tag, Encrypt and Hash XOM Key Table Handling Program State (5) Memory Registers Caches Data Tag Relocate data and hashes together

21. Handling Program State (6) • Incorrectly written program can still leak secrets • XOM assures that correct programs can secure their secrets

22. Supporting an Operating System (1) • Requirements • Good OS should be able to manage resources efficiently • Bad OS cannot violate the isolation of a compartment • An OS must adhere to XOM compartments when moving resources • Need to use XOM instructions

23. Supporting an Operating System (2) • XOM limitations • External debugging impossible • Shared memory and IPCs have to go through the NULL compartment • They need to be secured through separate mechanisms

24. XOM Key Table System Calls (1) • enter xom and exit xom insufficient when frequently called • Encryption/decryption everytime • If enter xom is privileged, a program has to cross the kernel boundary every time • If enter xom is not privileged, a malicious application can allocate all key entries to deny services

25. XOM Key Table System Calls (2) • enter xom is splitted into multiple calls • xalloc • Privileged • Allocate a register key • xentr • Not privileged • Enter a compartment

26. XOM Key Table System Calls (2) • Similarly for exit xom • xinval • Privileged • Mark the register key entry invalid • xexit • Return to the NULL compartment

27. Saving and Restoring Context • An OS needs xgetid to decrypt the ownership of a cache line • Exception frame must be enlarged to accommodate additional XOM and encrypted states • Certain things are revealed to the OS • TLB misses • Status bits that indicate whether a thread is in kernel mode

28. Paging Encrypted Pages • Hashes are stored separately • A bad OS needs to forge both data and the corresponding hash • Hashes are paged in and out with the corresponding data

29. Shared Libraries • Unencrypted for dynamic linking by various applications • Libraries that perform security sensitive routines should be statically linked and encrypted • OpenSSL

30. Process Creation • Need xom_fork() • Create two sets of register keys • Share the same compartment key

31. Child XOM ID Process Fork: Child uses one XOM ID, the Parent uses other XOM ID XOM ID Pre-fork: Allocate a new XOM Key Table Entry Parent XOM ID Process Creation Process XOM ID

32. User Defined Signal Handlers • Need to enlarge the signal context field for encrypted data • Signal key must be refreshed at each call to avoid replay attacks by an OS

33. Implementation Effort • Added 1,900 lines to the IRIX 6.5 • Ran applications in the SimOS simulator

34. Basic Processor Model Parameters • Encryption added 15 cycles to the memory access time • On cache misses and flushes • xalloc required 400,000 cycles to perform public key decryption

35. Operating System Overhead • Larger memory footprint • Additional I/O for storing and retrieving hashes

36. End-to-End Application Overhead • Turns out that granularity has little impact on performance overhead! • Overheads for most applications turn out to be less than 5% • If cache behavior is similar

37. Questions…. • What if the CPU is dead? • And you want to move a hard drive to another machine? • Backups are useless unless the original private key is also duplicated.…in hardware…