Secure Operating Systems

Secure Operating Systems Lesson 3: OS Structures

What Services Does an OS provide? • An environment for the execution of programs… which means what? • Right: a set of APIs that allow a programmer to leverage services controlled by the OS

User-facing • The GUI • Program Execution • I/O Operations • File-system manipulation • Comms • Error Detection

Under the Hood • Resource Allocation • Accounting • Protection and Security

The OS Interface • Of course, the GUI • But also the command interpreter and shell • Compare and contrast Unix and Windows on this • The Interface to programs is the API • This requires a change in domain • The API isn’t (usually) the same as the underlying system call • Let’s walk through a simple fopen() • Why is this an OS function? • How does it relate to the OS? Why?

Example: CreateFileA?W • HANDLE WINAPI CreateFile( _In_ LPCTSTR lpFileName, _In_ DWORD dwDesiredAccess, _In_ DWORD dwShareMode, _In_opt_ LPSECURITY_ATTRIBUTES lpSecurityAttributes, _In_ DWORD dwCreationDisposition, _In_ DWORD dwFlagsAndAttributes, _In_opt_ HANDLE hTemplateFile);

How the Call Works • Three ways • Often, via the stack • Or via registers • Or via a pointer to a block of memory

The Confused Deputy • We’ll look at this in much more detail later, but the OS can be thought of as being deputized by an application to do work for hire • This is a risky proposition

System Programs/Utilities • May not need more privilege • Handle things like: • File management • Status information • File modification • Programming language support • Program loading and execution • Comms– Message passing and Shared Memory

OS Design Goals • Perhaps where we fail most is when we set our design goals: poor goals lead to a bad design, and no wonder it doesn’t work • Performance – especially worst case – is often overlooked • We tend to “hand wave” over requirements in the OS

Mechanisms and Policies • Mechanisms are how something gets done • Policies define what to do… • Easily to confuse; when we mix them, we end up with inflexible systems

OS Structure • Simple layers, like MSDOS

Or Layers (like Unix)

Microkernels • A large kernel is dangerous from a security perspective (why?) • Possible solution is the microkernel • Move everything you can from the lowest domain • The kernel focuses on message passing • Makes it easier to expand • But performance can be problematic

Modular Kernels • A standard form for kernel loadable modules • Runtime loading allows you to add when you need things… • Problem: if you load my code into your kernel… yes… ahem…

OS Debugging • A tricky business (think about it!) • VMs can help • Core Dump: dump as much state as able at a critical error point • OS debugging has a number of challenges related to complexity, concurrence and attribution • Typically, we do some type of remote debugging

Assignment • Read Chapter 2 of OSC • Read “Protection” by Butler Lampson • A word to the wise – don’t behind on the reading, because there’s a lot of it. I’ll set less assignments based on that, but the material will be on the final.

Questions & Comments • What do you want to know?

Secure Operating Systems