Operating System Security, Con’t CS 136 Computer Security Peter Reiher October 20, 2011

1. Operating System Security, Con’tCS 136Computer Security Peter ReiherOctober 20, 2011

2. Outline • Interprocess communications protection • File protection and disk encryption • Protecting other OS resources • Logging and auditing

3. Protecting Interprocess Communications • Operating systems provide various kinds of interprocess communications • Messages • Semaphores • Shared memory • Sockets • How can we be sure they’re used properly?

4. IPC Protection Issues • How hard it is depends on what you’re worried about • For the moment, let’s say we’re worried about one process improperly using IPC to get info from another • Process A wants to steal information from process B • How would process A do that?

5. Gimme your secret Message Security Process A Process B That’s probably not going to work Can process B use message-based IPC to steal the secret?

6. How Can B Get the Secret? • He can convince the system he’s A • A problem for authentication • He can break into A’s memory • That doesn’t use message IPC • And is handled by page tables • He can forge a message from someone else to get the secret • But OS tags IPC messages with identities • He can “eavesdrop” on someone else who gets the secret

7. Can an Attacker Really Eavesdrop on IPC Message? • On a single machine, what is a message send, really? • A message is copied from a process buffer to an OS buffer • Then from the OS buffer to another process’ buffer • Sometimes optimizations skip some copies • If attacker can’t get at processes’ internal buffers and can’t get at OS buffers, he can’t “eavesdrop” • Need to handle page reuse (discussed earlier)

8. Other Forms of IPC • Semaphores, sockets, shared memory, RPC • Pretty much all the same • Use system calls for access • Which belong to some process • Which belongs to some principal • OS can check principal against access control permissions at syscall time • Ultimately, data is held in some type of memory • Which shouldn’t be improperly accessible

9. So When Is It Hard? • Always possible that there’s a bug in the operating system • Allowing masquerading, eavesdropping, etc. • Or, if the OS itself is compromised, all bets are off • What if it’s not a single machine? • What if the OS has to prevent cooperating processes from sharing information?

10. Distributed System Issues • What if your RPC is really remote? • Goal of RPC is to make remote access transparent • Looks “just like” local • The hard part is authentication • The call didn’t come from your own OS • How do you authenticate its origin?

11. The Other Hard Case Process A Process B Process A wants to tell the secret to process B But the OS has been instructed to prevent that A necessary part of Bell-La Padula, e.g. Can the OS prevent A and B from colluding to get the secret to B?

12. OS Control of Interactions • OS can “understand” the security policy • Can maintain labels on files, process, data pages, etc. • Can regard any IPC or I/O as a possible leak of information • To be prohibited if labels don’t allow it

13. Covert Channels • Tricky ways to pass information • Requires cooperation of sender and receiver • Generally in active attempt to deceive system • Use something not ordinarily regarded as a communications mechanism

15. Covert Channels in Computers • Generally, one process “sends” a covert message to another • But could be computer to computer • How? • Disk activity • Page swapping • Time slice behavior • Use of a peripheral device • Limited only by imagination

16. Handling Covert Channels • Relatively easy if you know what the channel is • Put randomness/noise into channel to wash out message • Hard to impossible if you don’t know what the channel is • Not most people’s problem

17. File Protection • Files are a common example of a typically shared resource • If an OS supports multiple users, it needs to address the question of file protection • Simple read/write access control • What else do we need to do?

18. Standard Access Control for Files • Basic application of typical access control mechanisms • Usually ACLs • Issues of complete mediation come up • Checked on open vs. on use • But what about the raw disk?

19. The Disk and File Systems • Most file systems are stored on disks • Disks are also devices • OS can access them below the file system interface • Can be moved to other machines • How do we protect file data under these circumstances?

20. Encrypted File Systems • Data stored on disk is subject to many risks • Improper access through OS flaws • But also somehow directly accessing the disk • If the OS protections are bypassed, how can we protect data? • How about if we store it in encrypted form?

21. An Example of an Encrypted File System Issues for encrypted file systems: When does the cryptography occur? Ks Where does the key come from? Sqzmredq #099 sn lx rzuhmfr zbbntms Transfer $100 to my savings account What is the granularity of cryptography?

22. When Does Cryptography Occur? • Transparently when a user opens a file? • In disk drive? • In OS? • In file system? • By explicit user command? • Or always, implicitly? • How long is the data decrypted? • Where does it exist in decrypted form?

23. Where Does the Key Come From? • Provided by human user? • Stored somewhere in file system? • Stored on a smart card? • Stored in the disk hardware? • Stored on another computer? • Where and for how long do we store the key?

24. What Is the Granularity of Cryptography? • An entire disk? • An entire file system? • Per file? • Per block? • Consider both in terms of: • How many keys? • When is a crypto operation applied?

25. What Are You Trying to Protect Against With Crypto File Systems? • Unauthorized access by improper users? • Why not just access control? • The operating system itself? • What protection are you really getting? • Data transfers across a network? • Why not just encrypt while in transit? • Someone who accesses the device not using the OS? • A realistic threat in your environment?

26. Full Disk Encryption • All data on the disk is encrypted • Data is encrypted/decrypted as it enters/leaves disk • Primary purpose is to prevent improper access to stolen disks • Designed mostly for laptops

27. HW Vs. SW Full Disk Encryption • HW advantages: • Probably faster • Totally transparent, works for any OS • Setup probably easier • HW disadvantages: • Not ubiquitously available today • More expensive (not that much, though) • Might not fit into a particular machine • Backward compatibility

28. Example of Hardware Full Disk Encryption • Seagate’s Momentus 7200 FDE.2 line • Hardware encryption for entire disk • Using AES • Key accessed via user password, smart card, or biometric authentication • Authentication information stored internally on disk • Check performed by disk, pre-boot • .15 Gbytes/sec sustained transfer rate • Primarily for laptops

29. Example of Software Full Disk Encryption • Microsoft BitLocker • Doesn’t encrypt quite the whole drive • Unencrypted partition holds bootstrap • Uses AES for cryptography • Key stored either in special hardware or USB drive • Microsoft claims “single digit percentage” overhead • One independent study claims 12%

30. Protecting Other Resources • Devices • The processor itself

31. Protecting Devices • Some handled through file system protection/authentication • Others by only allowing kernel to access them • User access mediated by the kernel • Assumes kernel will be careful in what it allows users to do • Sometimes issue with alternate access • E.g., Firewire interfaces

32. Protecting the Processor • Mostly about protecting processes from each other • But also about ensuring user-level processes can’t execute privileged instructions • Most typically, instructions OS needs to do its work • Syscalls used to explicitly change mode

33. Presumptions for Processor Protection • Assume context switch code clears out all info from old process • Registers, etc. • Model is that processor is clean and devoted only to current user • Switch to privileged mode is explicit • Don’t allow arbitrary user code to run in this mode

34. Logging and Auditing • An important part of a complete security solution • Practical security depends on knowing what is happening in your system • Logging and auditing is required for that purpose • Often (partially) built into OS

35. Logging • No security system will stop all attacks • Logging what has happened is vital to dealing with the holes • Logging also tells you when someone is trying to break in • Perhaps giving you a chance to close possible holes

36. Access Logs • One example of what might be logged for security purposes • Listing of which users accessed which objects • And when and for how long • Especially important to log failures

37. Other Typical Logging Actions • Logging failed login attempts • Can help detect intrusions or password crackers • Logging changes in program permissions • A common action by intruders • Logging scans of ports known to be dangerous

38. Problems With Logging • Dealing with large volumes of data • Separating the wheat from the chaff • Unless the log is very short, auditing it can be laborious • System overheads and costs

39. Log Security • If you use logs to detect intruders, smart intruders will try to attack logs • Concealing their traces by erasing or modifying the log entries • Append-only access control helps a lot here • Or logging to hard copy • Or logging to a remote machine

40. Local Logging vs. Remote Logging • Should you log just on the machine where the event occurs? • Or log it just at a central site? • Or both?

41. Local Logging • Only gives you the local picture • More likely to be compromised by attacker • Must share resources with everything else machine does • Inherently distributed • Which has its good points and bad points

42. Remote Logging • On centralized machine or through some hierarchical arrangement • Can give combined view of what’s happening in entire installation • Machine storing logs can be specialized for that purpose • But what if it’s down or unreachable? • A goldmine for an attacker, if he can break in

43. Desirable Characteristics of a Logging Machine • Devoted to that purpose • Don’t run anything else on it • Highly secure • Control logins • Limit all other forms of access • Reasonably well provisioned • Especially with disk

44. Network Logging • Log information as it crosses your network • Analyze log for various purposes • Security and otherwise • Can be used to detect various problems • Or diagnose them later

45. Logging and Privacy • Anything that gets logged must be considered for privacy • Am I logging private information? • If so, is the log an alternate way to access it? • If so, is the log copy as well protected as the real copy?

46. An Example • Network logs usually don’t keep payload • Only some header information • You can tell who talked to whom • And what protocol they used • And how long and much they talked • But not what they said

47. Auditing ` • Security mechanisms are great • If you have proper policies to use them • Security policies are great • If you follow them • For practical systems, proper policies and consistent use are a major security problem

48. Auditing • A formal (or semi-formal) process of verifying system security • “You may not do what I expect, but you will do what I inspect.” • A requirement if you really want your systems to run securely

49. Auditing Requirements • Knowledge • Of the installation and general security issues • Independence • Trustworthiness • Ideally, big organizations should have their own auditors

50. When Should You Audit? • Periodically • Shortly after making major system changes • Especially those with security implications • When problems arise • Internally or externally