Enforcing Security Policies using Transactional Memory Introspection

1. Enforcing Security Policies using Transactional Memory Introspection Vinod Ganapathy Rutgers University

2. Take-home slide We can utilize the mechanisms of Software Transactional Memory to greatly improve security policy enforcement Transactional Memory Introspection

3. REMOTE LOCAL X server with multiple X clients Transactional Memory Introspection

4. REMOTE LOCAL Malicious remote X client Transactional Memory Introspection

5. REMOTE LOCAL Undesirable information flow Transactional Memory Introspection

6. Desirable information flow REMOTE LOCAL Transactional Memory Introspection

7. Operation request Response Allowed? YES/NO X server with authorization X client Security enforcement crosscuts application functionality X server Reference monitor Authorization policy Transactional Memory Introspection

8. Outline • Enforcing authorization policies • Problems with existing techniques • Transactional Memory Introspection • Implementation and experiments Transactional Memory Introspection

9. Existing enforcement interface dispatch_request ( ) { ... perform_request ( ); } perform_request ( ) { ... perform_access (resource); ... perform_access’(resource’); } Transactional Memory Introspection

10. Existing enforcement interface dispatch_request ( ) { ... perform_request ( ); } perform_request ( ) { ... if (allowed(principal,resource,access)){ perform_access (resource); } else { handle_auth_failure1(); }; ... if (allowed(principal,resource’,access’)){ perform_access’(resource’); } else { handle_auth_failure2(); }; } Transactional Memory Introspection

11. Three problems • Violation of complete mediation • Time-of-check to Time-of-use bugs • Handing authorization failures Transactional Memory Introspection

12. I. Incomplete mediation dispatch_request ( ) { … perform_request ( ); } Must guard each resource access to ensure complete mediation perform_request ( ) { ... if (allowed(principal,resource,access)){ perform_access (resource); } else { handle_auth_failure1(); }; ... if (allowed(principal,resource’,access’)){ perform_access’(resource’); } else { handle_auth_failure2(); }; } Transactional Memory Introspection

13. I. Incomplete mediation [Zhang et al., USENIX Security ‘02] ssize_t vfs_read (struct file *file, ...) { ... if (check_permission(file, MAY_READ)) { file->f_op->read(file, ...); } ... } int page_cache_read (struct file *file, ...) { struct address_space *mapping = file->f_dentry->d_inode->i_mapping; ... mapping->a_ops->readpage(file, ...); } Transactional Memory Introspection

14. II. TOCTTOU bugs perform_request ( ) { ... if (allowed(principal,resource,access)){ perform_access (resource); } else { handle_auth_failure1() }; ... if (allowed(principal,resource’,access’)){ perform_access’(resource’); } else { handle_auth_failure2() }; } Transactional Memory Introspection

15. II. TOCTTOU bugs Similar race condition found in the Linux Security Modules framework [Zhang et al. USENIX Security ’02] perform_request ( ) { ... if (allowed(principal,resource,access)){ perform_access (resource); } else { handle_auth_failure1() }; ... if (allowed(principal,resource’,access’)){ perform_access’(resource’); } else { handle_auth_failure2() }; } • Several similar bugs recently found in • popular enforcement tools: [Watson, WOOT ’07] • GSWTK • Systrace[Provos, USENIX Security ’03] • OpenBSD Sysjail[Johnson and Deksters ’07] Transactional Memory Introspection

16. II. TOCTTOU bugs Authorization check and resource access must be atomic perform_request ( ) { ... if (allowed(principal,resource,access)){ perform_access (resource); } else { handle_auth_failure1() }; ... if (allowed(principal,resource’,access’)){ perform_access’(resource’); } else { handle_auth_failure2() }; } Transactional Memory Introspection

17. III. Failure handling Handling authorization failures is ad hoc and error prone perform_request ( ) { ... if (allowed(principal,resource,access)){ perform_access (resource); } else { handle_auth_failure1() }; ... if (allowed(principal,resource’,access’)){ perform_access’(resource’); } else { handle_auth_failure2() }; } Transactional Memory Introspection

18. III. Failure handling • Exception-handling code accounts for a large fraction of server software • Over two-thirds of server software [IBM ’87] • Nearly 46% on several Java benchmarks [Weimer & Necula OOPSLA’04] • Exception-handling code itself is error-prone [Fetzer and Felber ’04] • SecurityException most often handled erroneously [Weimer & Necula OOPSLA’04] Transactional Memory Introspection

19. Summary of problems • Violation of complete mediation • Need to identify all the resources accessed • Example: Bug in Linux Security Modules [Zhang et al., USENIX Security ‘02] • Time-of-check to Time-of-use bugs • Examples: [Zhang et al., USENIX Security ‘02] [Watson, WOOT ‘07] • Handing authorization failures • Large fraction of server code relates to error handling [IBM survey, ’87, Weimer and Necula, ‘04 ] • Error-handling code is error-prone! [Fetzer & Felber ’04] Security enforcement crosscuts application functionality Our solution: TMI Decouples security enforcement from application functionality Transactional Memory Introspection

20. Outline • Enforcing authorization policies • Problems with existing techniques • Transactional Memory Introspection (TMI) • Programmer’s interface • Mechanics of TMI • Implementation and experiments Transactional Memory Introspection

21. Transactional memory primer • Alternative to lock-based programming • Reason about atomic sections, not locks • TM provides atomicity and isolation acquire(S1.lock) acquire(S2.lock) value = S1.pop() S2.push(value) Release(S2.lock) Release(S1.lock) transaction { value = S1.pop() S2.push(value) } Transactional Memory Introspection

22. Programmer’s interface to TMI dispatch_request ( ) { transaction [ principal ] { ... perform_request ( ); } } perform_request ( ) { ... perform_access (resource); ... perform_access’(resource’); } Transactional Memory Introspection

23. Programmer’s interface to TMI dispatch_request ( ) { transaction [ principal ] { ... perform_request ( ); } } Authorization manager: case (resource=R, access_type=A)  if (!allowed(principal, R, A)) then abort_tx perform_request ( ) { ... perform_access (resource); ... perform_access’(resource’); } allowed(principal, resource, access)? allowed(principal, resource’, access’)? Transactional Memory Introspection

24. I. Complete mediation for free dispatch_request ( ) { transaction [ principal ] { ... perform_request ( ); } } TMI automatically invokes authorization checks perform_request ( ) { ... perform_access (resource); ... perform_access’(resource’); } Transactional Memory Introspection

25. II. TOCTTOU-freedom for free dispatch_request ( ) { transaction [ principal ] { ... perform_request ( ); } } Conflicting resource accesses automatically abort transaction perform_request ( ) { ... perform_access (resource); ... perform_access’(resource’); } Transactional Memory Introspection

26. III. Error-handling for free dispatch_request ( ) { transaction [ principal ] { ... perform_request ( ); } } Unauthorized resource accesses automatically abort transaction perform_request ( ) { ... perform_access (resource); ... perform_access’(resource’); } Transactional Memory Introspection

27. Decouples functionality and security dispatch_request ( ) { transaction [ principal ] { ... perform_request ( ); } } Authorization manager perform_request ( ) { ... perform_access (resource); ... perform_access’(resource’); } Transactional Memory Introspection

28. Outline • Enforcing authorization policies • Problems with existing techniques • Transactional Memory Introspection (TMI) • Programmer’s interface • Mechanics of TMI • Implementation and experiments Transactional Memory Introspection

29. TM runtime system • The TM runtime maintains per-transaction read/write sets and detects conflicts transaction { value = S1.pop() S2.push(value) } val1 = S1.pop() val2 = S1.pop() S2.push(val2) S2.push(val1) Transactional Memory Introspection

30. Retry TM runtime system Execution Validation Commit Read and Write Sets Transaction body Contention manager Commit logic Transactional Memory Introspection

31. Transactional Memory Introspection Execution Validation Authorization Commit Read and Write Sets Auth. Manager Success Transaction body Contention manager Auth. checks Commit logic Failure Retry Abort Transactional Memory Introspection

32. Transactional Memory Introspection dispatch_request ( ) { transaction [ principal ] { ... perform_request ( ); } } Accesses checked before tx commits Present in read/write set perform_request ( ) { ... perform_access (resource); ... perform_access’(resource’); } Transactional Memory Introspection

33. Outline • Enforcing authorization policies • Problems with existing techniques • Transactional Memory Introspection • Implementation and experiments Transactional Memory Introspection

34. TMI Implementation: TMI/DSTM2 • Implemented using Sun’s DSTM2 • Object-basedsoftware TM system • TM system modified to • Trigger authorization checks on additions to read/write set and upon transaction validation • Raise AccessDeniedException upon abort • Integrate transactional I/O libraries • Fewer than 500 lines changed in DSTM2 Transactional Memory Introspection

35. Porting software to TMI/DSTM2 • Mark transactional objects with @atomic • Also require @atomic wrappers for libraries: java.util.HashMap, java.util.Vector • Reads and writes to fields of @atomic objects replaced with DSTM2 accessors • Place transaction{…} blocks around client requests • Write an authorization manager Transactional Memory Introspection

36. GradeSheet in TMI/DSTM2 Transactional Memory Introspection

37. Evaluation • Ported four Java-based servers • GradeSheet: A grade-management server • FreeCS: A chat server • WeirdX: An X window management server • Enforced a simple XACML based policy • Tar: A tar archive service • Enforced Java stack inspection policy Transactional Memory Introspection

38. Modifications needed Authorization managers were approximately 200 lines of code in each case Transactional Memory Introspection

39. When to enforce policy? dispatch_request ( ) { transaction [ principal ] { ... perform_request ( ); } } Eager perform_request ( ) { ... perform_access (resource); ... perform_access’(resource’); } allowed(principal, resource, access)? allowed(principal, resource’, access’)? Transactional Memory Introspection

40. When to enforce policy? dispatch_request ( ) { transaction [ principal ] { ... perform_request ( ); } } Lazy allowed(principal, resource, access)? allowed(principal, resource’, access’)? perform_request ( ) { ... perform_access (resource); ... perform_access’(resource’); } Transactional Memory Introspection

41. When to enforce policy? dispatch_request ( ) { transaction [ principal ] { ... perform_request ( ); } } Parallel perform_request ( ) { ... perform_access (resource); ... perform_access’(resource’); } allowed(principal, resource, access)? allowed(principal, resource’, access’)? Transactional Memory Introspection

42. Performance overheads of TMI 10x -15.8% Transactional Memory Introspection

43. Performance overheads of STM • Software transactional memory imposes a significant overhead Hardware TMs reduce runtime overheads of TM runtime systems Transactional Memory Introspection

44. Take-home message We can utilize the mechanisms of Software Transactional Memory to greatly improve security policy enforcement Transactional Memory Introspection

45. Thank you! Reference: Enforcing Authorization Policies using Transactional Memory Introspection Proc. ACM CCS, October 2008 Vinod Ganapathy Rutgers University vinodg@cs.rutgers.edu http://www.cs.rutgers.edu/~vinodg