The Nachos Instructional OS and CS170 Projects

1. The Nachos Instructional OS and CS170 Projects CS 170, Tao Yang, Fall 2010

2. What is Nachos OS? • Allow students to examine, modify and execute operating system software. • A skeletal OS that supports threads and user-level processes, and simulates the general low-level facilities of typical machines, including interrupts, virtual memory and interrupt-driven device I/O. • Difference between Nachos and a hosting OS: Nachos is an OS running on a virtual machine, executed as a single Unix process in a hosting OS. • whereas a real OS runs on bare machines. • Over 9K lines of C++ code. • Good news: Can understand its basic mechanisms by reading about 1-2K lines of code.

3. System Layers Nachos Thread 1 Thread 2 Thread N Nachos Kernel (Threads, File System, Code execution/memory mapping, System calls/Interrupt) Simulated MIPS Machine (CPU, Memory, Disk, Console) Base Operating System (Linux for our class)

4. Nachos code directory • machine --- Basic machine specification (MIPS simulator). • threads --- threads management (Project 1). • userprog -- binary code execution and system calls (Project 2). • vm -- virtual memory (empty, Project 3). • filesys -- file system (used in Projects 2/3) • test -- binary test code (short user test programs, Projects 2/3). • network -- networking protocol (not used in our projects). • bin -- utilities/tools (binary format conversion)

5. Project 0 (Oct 6) • Obtain and install Nachos source code. • Copy the source code from ~cs170/nachosSept20.tar.gz • Compile the source code using gmake and Check if it works • Run threads demo under the threads subdirectory (just run kernel test threads). • Run user program demo under the userprog subdirectory. • Submit the code to exercise code submission process. Also tell us your group.

6. Nachos Threads • Nachos threads execute and share the same code, share the same global variables. • The Nachos scheduler maintains a ready list, containing all threads that are ready to execute. • Each thread is in one of four states: READY, RUNNING, BLOCKED, JUST_CREATED. • Each thread object maintains a context block. • Thread object supports the following operations: • Thread(char *debugName). Create a thread. • Fork(VoidFunctionPtr func, int arg). Let a thread execute a function. • Yield(). Suspend the calling thread and select a new one for execution. • Sleep(). Suspend the current thread, change its state to BLOCKED, and remove it from the ready list • Finish()

7. Sample Example of Nacho Threads main () { Thread *t1 = new Thread("forked thread1"); Thread *t2 = new Thread("forked thread2"); t1->Fork(SimpleThread, 1); t2->Fork(SimpleThread, 2); SimpleThread(3); } SimpleThread(int i) { printf(“Hello %d\n”, i); currentThread->Yield(); } Create 2 new threads. Start to fork and execute a function in each child thread. Parent also executes the same function Function executed by threads

8. Nachos Thread States and Transitions running (user) When running in user mode, the thread executes within the machine simulator. Proj 2 covers this. In Proj 1 we are only concerned with the states in this box. Machine::Run, ExceptionHandler interrupt or exception Thread::Yield running (kernel) Thread::Sleep Scheduler::Run blocked ready Scheduler::ReadyToRun

9. Thread Switching • Switching involves suspending the current thread, saving its state, and then restoring the state of the new thread. • Following code involved in execution: the old code, the new code, and the code that performs switching. • Switch(oldThread, newThread): • Save all registers in oldThread's context block. • Save the program address to be used when the old thread is resumed. • Load new values into the registers from the context block of the new thread. • Once the saved PC of the new thread is loaded, Switch() is no longer executing.

10. Scheduler object for thread scheduling • A scheduler decides which thread to run next by scanning the ready list. • The scheduler is invoked whenever the current thread gives up the CPU. • The current Nachos scheduling policy is round-robin: new threads are appended to the end of the ready list, and the scheduler selects the front of the list. • The Scheduler object has the following operations: • ReadyToRun(Thread *thread). Make thread ready to run and place it on the ready list. • Thread *FindNextToRun() • Run(Thread *nextThread)

11. Semaphore object for thread synchronization • Disable and re-enable interrupts to achieve mutual exclusion (e.g., by calling Interrupt::SetLevel()). • Operations for a Semaphore object: • Semaphore(char* debugName, int initialValue) • P(): Decrement the semaphore's count, blocking the caller if the count is zero. • V() :Increment the semaphore's count, releasing one thread if any are blocked waiting on the count.

12. Key steps when Nachos executes After you type ``nachos'' under threads subdirectory: • It is executing as a single Unix process. • The main() calls • Initialize() to start up interrupt handling, create a scheduler for managing the ready queue. • ThreadTest() (to be explained for Project 1). • currentThread->Finish() to let other threads continue to run.

13. Key Calling graph when Nachos executes under thread directory StackAllocate() in thread.cc All files are in threads directory. Thread:Fork () in thread.cc Initialize() in system.cc SWITCH () in switch.s FindNextToRun () in scheduler.cc Thread:Yield () in thread.cc main() in main.cc ThreadRoot () in switch.s ReadyToRun () in scheduler.cc ThreadTest () in threadtest.cc func() such as SimpleThread() in ThreadTest.cc Run () in scheduler.cc currentThread->Finish () in threadtest.cc

14. Project 1: threads & synchronization • Work under threads subdirectory. • Modify ThreadTest() to do simple threads programming (spawning multiple threads). • Implement locks and condition variables (missing from the file synch.cc). • Design am application using the implemented synchronization primitives. • Workload: • Read Nachos code and add few hundred lines of code. • Implemented synchronization primitives will be used in Projects 2 and 3.

15. Project 1: Files involved Key files • main.cc, threadtest.cc -- a simple test of our thread routines. • thread.h thread.cc -- Nachos thread data structure and operations. • scheduler.h scheduler.cc -- The thread ready list. • synch.h synch.cc -- synchronization routines. Other related files. • synchlist.h, synchlist.cc -- synchronized access to lists using locks/conditions (useful examples for your programming). • list.h list.cc -- generic list management. • system.h, system.cc -- Nachos startup/shutdown routines. • utility.h utility.cc -- some useful definitions and debugging routines. • interrupt.h interrupt.cc -- manage interrupts. • time.h timer.cc -- clock emulation. • switch.h, switch.s -- assembly code for thread switching. • stats.h stats.cc -- collect interesting statistics.

16. Machine • Source code: under machine subdirectory. • Roughly approximates the MIPS architecture • Machine has registers, memory, a CPU and a clock. • Simulated clock used for event scheduling and execution (interrupts). • Can execute an arbitrary program with a sequence of MIPS instructions:

17. Machine: Code execution Steps • Load instructions into the machine's memory. • Initialize registers (including the program counter PCReg). • Tell the machine to start executing instructions. • The machine then fetches the instruction PCReg points at, decodes it, and executes it. • The process is repeated until an illegal operation is performed or an interrupt is generated. • When a trap or interrupt takes place, an interrupt service routine deals with the condition.

18. Two modes of executions • User mode (executing MIPS instructions of a user program) • Example: Halt code in test directory • Nachos executes user-level processes by loading them into the simulator's memory, initializing the simulator's registers and then running the simulator. • User-programs can only access the memory associated with the simulated machine.

19. Two modes of executions • Kernel mode • kernel executes when Nachos first starts up and loads a user program. • or when a user-program executes an instruction that causes a ``hardware’’ trap • illegal instruction, • page fault (access memory address which does not exist) • system call

20. Machine Object: Implement a MIPS machine • an instance created when Nachos starts up. • Supported public variables: • Registers: 40 registers. • Memory: Byte-addressable. • Virtual memory: use a single linear page table or a software-managed TLB.

21. Machine Object: Supported operations • Machine(bool debug). • Translate(int virtAddr, int* physAddr, int size, bool writing). • OneInstruction(). • Run() • ReadRegister(int num) • WriteRegister(int num, int value) • ReadMem(int addr, int size, int* value) • WriteMem(int addr, int size, int value)

22. Machine Object: Supported operations • Machine(bool debug). • Translate(int virtAddr, int* physAddr, int size, bool writing). • OneInstruction(). • Run() • ReadRegister(int num) • WriteRegister(int num, int value) • ReadMem(int addr, int size, int* value) • WriteMem(int addr, int size, int value)

23. Interrupt Object • Maintain an event queue together with a simulated clock. • Supported operations: • Schedule(VoidFunctionPtr handler, int arg, int when, IntType type) Schedule a future event to take place at time ``when''. • Usage: schedule a yield at random interval. • SetLevel(IntStatus level). Used to temporarily disable and re-enable interrupts for mutual exclusion purposes. Two levels are supported: IntOn and IntOff. • OneTick()-- • CheckIfDue(bool advanceClock). Examines if some event should be serviced. • Idle(). ``advances'' to the clock to the time of the next scheduled event

24. Interrupt::OneTick() • Software managed clock. • The clock advances 1 tick for user mode, 10 for system mode • after every restored interrupt (disable/enable Interrupt) • or after the MIPS simulator executes one instruction. • When the ready list is empty, fast-advance ticks until the next scheduled event happens.

25. Timer object • Generate interrupts at regular or random intervals • Then Nachos invokes the predefined clock event handling procedure. • Supported operation: • Timer(VoidFunctionPtr timerHandler, int callArg, bool doRandom). • Create a real-time clock that interrupts • every TimerTicks (100) time units • Or set this a random number for random mode

26. Console Object • Simulates the behavior of a character-oriented CRT device • Data can be written to the device one character at a time through the PutChar() routine. • Input characters arrive one-at-a-time. They can be retrieved by GetChar(). • Supported operations: • Console(char *readFile, char *writeFile, VoidFunctionPtr readAvail,VoidFunctionPtr writeDone, int callArg). Create a console instance.``readFile'' is the Unix file of where the data is to be read from; if NULL, standard input is assumed. • PutChar(char ch) • GetChar()

27. Disk Object • Simulates the behavior of a real disk. • The disk has only a single platter, with multiple tracks (32). • Each track contains the same number of sectors (32). • Allow only one pending operation at a time. • Contain a ``track buffer'' cache. Immediately after seeking to a new track, the disk starts reading sectors, placing them in the track buffer. • Supported operations: • Disk(char *name, VoidFunctionPtr callWhenDone, int callArg) • ReadRequest(int sectorNumber, char *data) • WriteRequest(int sectorNumber, char *data) • ComputeLatency(int newSector, bool writing)

28. data data Executing a user program halt shell user space MIPS instructions executed by the emulator ExceptionHandler() Nachoskernel MIPS emulator Machine::Run() fetch/execute examine/deposit SaveState/RestoreState examine/deposit Rn page table process page tables Machine object SP PC memory registers

29. data data data data data From C program to MIPS binary myprogram.c myprogram.o object file int j; char* s = “hello\n”; int p() { j = write(1, s, 6); return(j); } assembler libraries and other objects linker ….. p: store this store that push jsr _write ret etc. gcc compiler program myprogram.s myprogram (executable file)

30. header text data idata wdata symbol table relocation records What’s in an Object File? Header “magic number” indicates type of image. program instructions p Section table an array of (offset, len, startVA) immutable data (constants) “hello\n” program sections writable global/static data j, s j, s ,p,sbuf Used by linker; may be removed after final link step and strip. int j = 327; char* s = “hello\n”; char sbuf[512]; int p() { int k = 0; j = write(1, s, 6); return(j); }

31. Binary code format (Noff) Source code: under userprog subdirectory. • The current Nachos can run a single MIPS binary (Noff format) e.g. type ``nachos -x ../test/halt''. • A user program must be compiled using a cross-platform gcc compiler that generates MIPS code. • A Noff-format file contains (bin/noff.h) • the Noff header, describes the contents of the rest of the file • executable code segment (TEXT) • initialized data segment (DATA) • uninitialized data segment (BSS).

32. STACK HEAP BSS DATA TEXT Space usage during execution of a C program Stack grows from top-down. Heap grows bottom-up Uninitialized data Initialized data Code

33. TEXT, DATA, BSS, HEAP, STACK in C Int f3=3; /* DATA segment */ Int f1; /*BSS segment*/ def[] = "1"; /* DATA segment */ int main(void){static char abc[12], /* BSS segment */static float pi = 3.14159; /* DATA segment */int i = 3; /* Stack*/ char *cp; /*stack*/ cp= malloc(10); /*malloc allocates space from HEAP*/ f1= i+f3; /* code is in TEXT*/ strcpy(abc , "Test" ); /* “Test” is located in DATA segment */ }

34. Noff format . Each segment has the following information: • virtualAddr: virtual address that segment begins at. • inFileAddr: Pointer within the Noff file where that section actually begins. • The size (in bytes) of that segment.

35. User process for executing a program A Nachos thread is extended as a process • Each process has its own address space containing • Executable code (Code segment) • Initialized data (Data segment) • Uninitialized data (BSS) • Stack space for function calls/local variables • how big is address space? • A process owns some other objects, such as open file descriptors.

36. Steps in User process creation Currently only execute a single user program. • Create an address space. • Zero out all of physical memory (machine->mainMemory) • Read the binary into physical memory and initialize data segment. • Initialize the translation tables to do a one-to-one mapping between virtual and physical addresses. • Zero all registers, setting PCReg and NextPCReg to 0 and 4 respectively. • Set the stackpointer to the largest virtual address of the process (stack grows downward).

37. Key Calling graph when Nachos executes under userprog directory Executable file ReadAt() Space= New AddrSpace() in addrspace.cc Initialize() in system.cc Machine-> WriteRegister() Space-> InitRegisters() main() in main.cc StartProcess () in progtest.cc Space ->RestoreState() Machine-> OneInstruction() Machine->Run () in mipssim.cc Interupt-> OneTick() In Interupt.cc

38. Creating a Nachos Process (userprog/progtest.cc) Create a handle for reading text and initial data out of the executable file. void StartProcess(char *filename) { OpenFile *executable; AddrSpace *space; executable = fileSystem->Open(filename); if (executable == NULL) { printf("Unable to open file %s\n", filename); return; } space = new AddrSpace(executable); currentThread->space = space; delete executable; // close file space->InitRegisters(); space->RestoreState(); machine->Run(); ASSERT(FALSE); } Create an AddrSpace object, allocating physical memory and setting up the process page table. Set address space of current thread/process. Initialize registers, load pagetable, and begin execution in user mode.

39. Creating a Nachos Address Space (userprog/addrspace.cc) AddrSpace::AddrSpace(OpenFile *executable) { NoffHeader noffH; unsigned int i, size; executable->ReadAt((char *)&noffH, sizeof(noffH), 0); // how big is address space? size = noffH.code.size + noffH.initData.size + noffH.uninitData.size + UserStackSize; // we need to increase the size to leave room for the stack numPages = divRoundUp(size, PageSize); size = numPages * PageSize; pageTable = new TranslationEntry[numPages]; for (i = 0; i < numPages; i++) { pageTable[i].virtualPage = i; // for now, virtual page # = phys page # pageTable[i].physicalPage = i; pageTable[i].valid = TRUE; } .... Read the header of binary file Compute address space need Setup a page table for address translation

40. Initializing a Nachos Address Space Zero out memory allocated bzero(machine->mainMemory, size); // copy in the code and data segments into memory if (noffH.code.size > 0) { noffH.code.virtualAddr, noffH.code.size); executable->ReadAt(&(machine->mainMemory[noffH.code.virtualAddr]), noffH.code.size, noffH.code.inFileAddr); } if (noffH.initData.size > 0) { noffH.initData.virtualAddr, noffH.initData.size); executable->ReadAt(&(machine->mainMemory[noffH.initData.virtualAddr]), noffH.initData.size, noffH.initData.inFileAddr); } Copy code segment to memory Copy initialized data segment to memory

41. System Calls & Exception Handling • User programs invoke system calls by executing the MIPS ``syscall'' instruction • ``syscall'' generates a hardware trap into the Nachos kernel. • A trap is implemented by invoking RaiseException() with arguments indicating the trap cause. • RaiseException() calls ExceptionHandler() to take care of the • specific problem. • The system call number is stored in Register 2 and the return address is in Register 31. Assembly code for Halt(): Halt: addiu $2,$0,SC_Halt syscall j $31 .end Halt

42. When typing “nachos –x halt” • The main thread starts by running function StartProcess() in file progtest.cc. This thread is used to run halt binary. • StartProcess() allocates a new address space and loads the halt binary. It also initializes registers and sets up the page table. • Call Machine::Run() to execute the halt binary using the MPIS emulator. • The halt binary invokes the system call Halt(), which causes a trap back to the Nachos kernel via functions RaiseException() and ExceptionHandler(). • The exception handler determines that a Halt() system call was requested from user mode, and it halts Nachos.

43. Nachos –x halt using halt.c in test directory Machine-> RaiseException(SyscallException) Machine-> OneInstruction() Machine->Run () in mipssim.cc ExceptionHandler(SyscallException) Interrupt-> Halt() In Interupt.cc

44. Proj 2: Multiprogramming&System Calls • Modify source code under userprog subdirectory. • ~1200 lines of code. • The crossplatform compiler is under ~cs170/gcc. • This compiler on x86 machines produces a.out with the coff format. • Use utility coff2noff (under nachos’ bin directory) to convert it as Noff. • Check the makefile under test subdirectory on how to use gcc and coff2noff. • System calls to be implemented: • Multiprogramming: Fork(), Yield(), Exit(), Exec() and Join(). • File and console I/O: Creat(), Open(), Read(), Write(), and Close().

45. 0 0 0 text text text data data data data data data BSS BSS BSS user stack user stack user stack args/env args/env args/env kernel area kernel area kernel area Multi-Processes and the Kernel 2n-1 2n-1 Nachos kernel 2n-1

46. To run multiple processes in Proj 2 Nachos should • Provide the physical memory management; • Set up a address translation table with linear page tables; • Save/restore address-space related state during process switching (AddrSpace::SaveUserState() and AddrSpace:RestoreUserState() are called).

47. Address translation with linear page tables • A virtual address is split into page number and page offset components • Machine->pageTable points to the page table to be used. • Machine->pageTableSize -- the actual size of the page table. • Process switching requires to set the above pointer properly for each user. • Physical page 0 starts at machine-> mainMemory. • Each page has 128-bytes. The actual number of physical pages is NumPhysPages (32).

48. A Simple Page Table Each process/VAS has its own page table. Virtual addresses are translated relative to the current page table. process page table PFN 0 PFN 1 PFN i In this example, each VPN j maps to PFN j, but in practice any physical frame may be used for any virtual page. PFN i + offset page #i offset The page tables are themselves stored in memory; a protected register holds a pointer to the current page table. user virtual address physical memory page frames

49. Project 2: Files involved • Key files. • progtest.cc -- test routines to run user code. • addrspace.h addrspace.cc -- create an address space and load the program from disk. • syscall.h -- the system call interface. • exception.cc -- the handler for system calls and other user-level exceptions such as page faults. • filesys.h, openfile.h console.h -- interface to the Nachos file system and console. • Other related files: • bitmap.h bitmap.cc -- manipulate bitmpas (useful for keeping track of physical page frames). • translate.h, translate.cc -- translation tables. • machine.h, machine.cc -- emulates main memory, processor, etc. • mipsim.cc -- emulates MPIS R2/3000 instructions. • console.cc -- emulates a terminal using UNIXfiles.

50. Nachos File System • Two versions (check openfile.h and filesys.h): • A ``stub'' version is simply a front-end to the Unix filesystem for project 2 and part 1 of project 3. • Implemented on top of a raw disk. Will be extended in part 2 of Project 3 • Function layers (from top to down): • FileSystem object • Created by fileSystem = new FileSystem(format) in system.cc • OpenFile objects (many) • Created by every call to open/access a file in a directory. • SynchDisk object • Created by synchDisk = new SynchDisk("DISK") in system.cc • The Disk object. • Created by disk = new Disk(name,..) in SynchDisk::SynchDisk()