CS U480: Systems & Networks 1. Introduction Donghui Zhang

1. CS U480: Systems & Networks1. IntroductionDonghui Zhang

2. Syllabus • Instructor: Donghui Zhang • Class page linked from my home page: http://www.ccs.neu.edu/home/donghui

3. Your Background • CSU380: Computer organization • System architecture: Processors, memory, I/O devices • Processor architecture: ALU, instruction execution • Assembly-level programming • C/C++ programming • Familiarity with the C language and the standard C library • Please consult with me if you are unsure your background is sufficient

4. Why Programming • For CS: must. • A pilot must know how to fly an airplane. • For IS: also important. • A project manager needs to have technical background. • Bill Gates used to be a superb programmer. • Guest speaker: Prof. Hafner.

5. Why C (and not Java) • Talked with Mr. Feuer and Prof. Hafner. • We provide guidance to what you should learn. (If you say “give us A without any exam or project”…) • Most OS are implemented in C. • Java hides many low-level details. • This is a place to strengthen your C skills.

6. What is an Operating System? An operating system is a program that acts as an intermediary between a user of a computer and the computer hardware. • Goals: • Execute user programs and make solving user problems easier. • Make the computer system convenient to use. • Performance measures: • Throughput: The total amount of work done over a period of time. • Turnaround: The total time it takes to complete a job.

7. Concept Map Applications GUI Console OS CPU ALU Registers Memory Peripherals Disks Keyboard Mouse Display

8. Software Layers Application c = getc() Library read(…) Operating System kb_driver_read(…) Device driver read_device(…)

9. Compilation vs. Execution High-level language program Compiler Assembly language Shared Libraries Executable Assembler Operating System Machine code Device Drivers Libraries of Machine code Link editor Memory • During compilation, statements in higher-level languages are converted into machine code • During execution, machine code is interpreted by a processor

10. Evolution of the OS • In the beginning • Whirlwind at MIT • PDP-1 from DEC • Altair from MITS • Program stored in a hardware patch board or toggled in using switches • Program can access all of memory • Program starts at location zero • Loading programs by hand is slow and error-prone and painful, so . . . n Program 0 Memory

11. Evolution of the OS: The Loader • Create a very short program to load longer programs • Toggle in bootstrap loader • Primitive loader may load in program or a more complex loader to read from cards or tape • Loading from cards is very slow, often took longer to load than to run, wasting (expensive) processor cycles, so . . . n Loader Program 0 Memory

12. Evolution of the OS: Batch Processing n Batch Loader • Jobs are spooled on tape • While one batch of jobs is running, card reader writes next batch of jobs on tape • Jobs are read sequentially from tape into memory • Tape is still relatively slow compared to processing Program 0 Memory

13. The Concept of a Job • With the introduction of programming languages, a translator is needed to convert a program into machine code • When a program is stored on cards in the programming language, e.g., assembler or Fortran, it must be translated before it can be run • Example: To run a program written in Fortran • Load (machine code for) Fortran compiler • Run compiler: Compiler reads program, writes machine code • Load (machine code for) program • Run program

14. Evolution of the OS: Multiprogramming • Load next program while previous program is running • But now both programs can’t begin at address zero (in fact, starting address isn’t known in advance) • Solutions? • Programs that perform lots of I/O waste (expensive) processor cycles n Loader Program 2 Program 1 0 Memory

15. Evolution of the OS: Multitasking • Share execution time among programs • When one program starts I/O, let the other run. • What is needed to switch from one program to another? • A program bug in one program can overwrite another program, or the system programs and data, so . . . n Loader + scheduler System data Program 2 Program 1 0 Memory

16. Evolution of the OS: Protection n Loader, scheduler, memory mgr System data Program 2 Program 3 Program 1 0 Memory • Run programs in separate address spaces • System programs need to cross address spaces. • System runs in privileged mode. • The more programs in memory, the less memory available for each program, so . . .

17. Evolution of the OS: Virtual Memory n Loader, scheduler, memory mgr, resource mgr 0 Disk Memory • Address space implemented in both main and secondary memory • Broken into pages • A program’s address space is a set of pages. • The address space for a program may be larger than main memory! • Pages are swapped in and out of main memory as needed.

18. OS Diversity • Great diversity of programmable hardware • Super computers: simulation, scene generation, data mining • Servers: database, web, video • Personal: desktop, laptop • Embedded: PDA, phone, media device • Nature of the OS depends on • Application mix • Hardware capability • Real-time requirements • With the proliferation of embedded systems, most processors do not run a general purpose OS

19. OSes are complex programs developed over many years by many people • They confront common problems that reappear in other contexts • The problems have been formalized • A variety of solutions have been proposed and implemented • Choosing a solution requires evaluating tradeoffs of space, time, and complexity • OSes are a rich source of well-designed sample programs • We will study OSes by exploring common components • Understand the motivation for each component • Understand the tradeoffs for each implementation

20. Common System Components • Process Management • Main-Memory Management • File System • I/O System • Network Management Let’s take a high-level tour of these components

21. Process Management • A process is a program in execution • To accomplish its task, a process needs certain resources: • CPU time • Memory • Files • I/O devices. • The OS is responsible for the following activities in connection with processes: • Process creation and deletion • Process suspension and resumption • Mechanisms for: • Process synchronization • Inter-process communication

22. From Program to Process High-level language program Compiler Assembly language Shared Libraries Executable Assembler Operating System Machine code Device Drivers Libraries of Machine code Link editor Memory Process Program

23. Memory Management • Main memory is a large array of words • Each word (or, often byte) has its own address • Data in memory is shared by the CPU and I/O devices • Main memory is (usually) volatile • It loses its contents in the case of system failure. • The OS is responsible for the following activities in connection with memory management: • Keep track of which parts of memory are currently being used and by whom. • Decide which processes to load when memory space becomes available. • Allocate and deallocate memory space as needed.

24. Memory Management (cont.) n Loader, scheduler, memory mgr System data Program 2 Program 3 Program 1 0 Memory

25. Secondary-Storage Management • Secondary storage is (usually) a large array of blocks • Each block has its own address • Data is moved between main memory and secondary storage in units of blocks • Secondary storage is non-volatile and can be very large • Disks are the most common in general purpose systems • Memory cards and stick are common on portable devices • The OS is responsible for the following activities in connection with secondary storage management: • Free space management • Storage allocation • For disks, scheduling of block transfers

26. Secondary-Storage Management (cont.) n Loader, scheduler, memory mgr, resource mgr 0 Disk Memory

27. File Management • A file is a collection of related information • Files are stored within a file system • From the view of most systems, a file is an array of bytes • The OS is responsible for the following activities in connections with file management: • File creation and deletion • Directory creation and deletion • Support of primitives for manipulating files and directories • Mapping files onto secondary storage. • File backup on stable (nonvolatile) storage media

28. I/O System Management • Input/Output refers to the movement of data between main memory and peripheral devices • Devices vary widely in their operation and behavior • Devices are partitioned into classes to factor common behavior • A device driver translates between general OS operations and device-specific commands • I/O managements consists of • A buffer-caching system • A general device-driver interface • Drivers for specific hardware devices

29. I/O System Management (cont.)

30. Network Management • Networking allows distinct computer systems to exchange data • Communication takes place using a protocol • Networked computers vary widely in their degree of coupling: • They may share a common OS and processes may be visible across systems • They may share nothing except a communication port • Networking allows users to access to non-local resources, allowing: • Computation speed-up, through special purpose hardware or parallel processing • Availability of data from other systems • Enhanced reliability through redundancy

31. Network Management (cont.)