Operating Systems

1. Operating Systems Prof. Neeraj SuriDinu Sarbu, Brahim Ayari, Andréas Johansson www.deeds.informatik.tu-darmstadt.de Dinu Andreas Brahim

2. Introduction • Do we really need an OS? • What does an OS do? • What issues need considering? • What is OS trustworthiness? • Administrative stuff… • DEEDS profile

3. OS History • Initially there was no OS (or cell phones or iPods’s ) • 1st Generation: M/C level programming done directly on HW: no disks, no peripherals, single user • 2nd Generation: batch programming: card readers, tapes  single program at a time • 3rd Generation: multi-programming (time sharing), multiplexing (each job to its memory partition) + scheduling of each partition • Now: distributed, networked, virtual machines, multi-user, multi-core, smart cards …

4. The Role of the OS • OS as resource manager • OS as virtual/extended machine 4 mil+ (a) pipelined (b) superscalar CPU Pentium+: > 6 pipeline levels

7. I/O and Interrupts • Polling/busy waiting • Interrupt driven • DMA (a) I/O triggers for CPU access (b) CPU handling of triggers/interrupts

8. Users Multi-Processing Processes • Process tree • A created two child processes, B and C • B created three child processes, D, E, and F

9. Processes + Limited Resources? • potential deadlock (b) actual deadlock sequencing of activities!!!!

10. Operating System Functionality • OS is a resource allocator • Manages all (HW, applications etc) resources • Decides between conflicting requests for efficient and fair resource use • each program gets its time with the computing resource • each program gets its space with the computing resource • OS is a control program • Controls execution of programs to prevent errors and improper use of the computer – ordering, sequencing, …

11. Operating System Definition • No universally accepted definition  • “Everything a vendor ships when you order an operating system” is (a widely varied!) good approximation • “The one program running at all times on the computer” is the kernel. Everything else is either a system program (ships with the operating system) or an application program • OS: program (or set of programs) providing • execution environment for user applications • interface between HW resources and user with resource arbitration

13. Users  Processes • Process tree • A created two child processes, B and C • B created three child processes, D, E, and F

14. Process Management • A process is a program in execution. It is a unit of work within the system. Program is a passive entity, process is an active entity. • Process needs resources to accomplish its task • CPU, memory, I/O, files • Initialization data • Process termination requires reclaim of any reusable resources • Single-threaded process has one program counter specifying location of next instruction to execute • Process executes instructions sequentially, one at a time, until completion • Multi-threaded process has one program counter per thread • Typically system has many processes, some user, some operating system running concurrently on one or more CPUs • Concurrency by multiplexing the CPUs among the processes / threads

15. Process Management Activities OS responsible for process management: • Creating and deleting both user and system processes • Suspending and resuming processes • Providing mechanisms for process synchronization • Providing mechanisms for process communication • Providing mechanisms for deadlock handling

16. Memory Management • All data in memory before and after processing • All instructions in memory in order to execute • Memory management determines what is in memory when • Optimizing CPU utilization and computer response to users • Memory management activities • Keeping track of which parts of memory are currently being used and by whom • Deciding which processes (or parts thereof) and data to move into and out of memory • Allocating and deallocating memory space as needed

17. Storage Management • OS provides uniform, logical view of information storage • Abstracts physical properties to logical storage unit - file • Each medium is controlled by a device (i.e., disk drive, tape drive) • Varying properties include access speed, capacity, data-transfer rate, access method (sequential or random) • File-System management • Files usually organized into directories • Access control to determine who can access what & when (RW!) • OS activities include • Creating and deleting files and directories • Primitives to manipulate files and dirs • Mapping files onto secondary storage • Backup files onto stable (non-volatile) storage media

18. Mass-Storage Management • Usually disks used to store data that does not fit in main memory or data that must be kept for a “long” period of time. • Speed of operation hinges on disk subsystem and its algorithms • OS activities • Free-space management • Storage allocation • Disk scheduling • Some storage need not be fast • Tertiary storage includes optical storage, magnetic tape • Still must be managed • Varies between WORM (write-once, read-many-times) and RW (read-write)

19. I/O Subsystem • OS hides ops of hardware devices from the user – drivers? • I/O subsystem responsible for • Memory management of I/O including buffering (storing data temporarily while it is being transferred), caching (storing parts of data in faster storage for performance), spooling (the overlapping of output of one job with input of other jobs) • General device-driver interface • Drivers for specific hardware devices

20. I/O Structures • Synchronous: After I/O starts, control returns to user program only upon I/O completion. • Wait instruction idles the CPU until the next interrupt • Wait loop (contention for memory access). • At most one I/O request is outstanding at a time, no simultaneous I/O processing. • Asynchronous: After I/O starts, control returns to user program without waiting for I/O completion. • System call – request to OS allows user wait for I/O finish • Device-status table contains entry for each I/O device indicating its type, address, and state. • Operating system indexes into I/O device table to determine device status and to modify table entry to include interrupt.

21. Device-Status Table

22. User-Kernel Comm. : System Calls read (fd, buffer, nbytes)

23. System Calls (Process/File/Mem Mgmt)

24. OS Structures? • Monolithic: all user/kernel functions inside a single kernel (pro: all OS procedures can call on each other, visible to all. con: all procedures need to be compiled and linked to each other – static!) • Layered • Virtual machines • Client-Server/Microkernel based: non-basic services float as servers with a basic kernel for primitive functions: IPC based!

25. Operating System Structure Simple structuring model for a monolithic system

26. Operating System Structure Structure of the THE layered operating system (E. W. Dijkstra, 1968)

27. Operating System Structure Structure of VM/370 with CMS… JVM’s?

28. Operating System Structure The flat client-server model - as a communication co-ordinator

29. Computing Environments • Distributed Client-Server Computing • Dumb terminals supplanted by smart PCs • Many systems now servers, responding to requests generated by clients • Compute-server provides an interface to client to request services (i.e. database) • File-server provides interface for clients to store and retrieve files

30. Operating System Structure The client-server model in a distributed system

31. Distributed/Networked OS’s? • Flexible: resources/functionality can be added • Sharing of data, msg, devices • Cheaper? price/performance • Faster? • Reliable/Dependable? (Efficient, fair, available) • Responsive? (RT.. in-time or …) • Secure, networked, load balancing? Trustworthiness

32. Application- Process User- Process Utilities Shells, Editor, Compiler Libraries e.g. libx.a User- Process Application Application Application Programming Interface (API) e.g. open, close, read, write, fork, exec, kill Login Memory- Management File System Access Control Process/Thread Management Authentication Passwords Interrupt- System OS kernel Device Drivers Networking TCP/IP, PPP, ... I/O-System Hardware (centralized or distributed)

33. Course Outline • Processes and Threads • Deadlocks/Distributed Concurrency Management  • Memory Management • Input/Output • File Systems  • Resource Allocation/Scheduling • Distributed/Networked OS: Models, RPC, … • Checkpointing, Recovery Blocks and Rejuvenation • Drivers and System Profiling  • Security • Verification/Validation  • Embedded/Real-Time OS’s 

34. Relevant Literature • Modern Operating Systems; 2nd Edition 2001+, A. Tanenbaum, Prentice Hall • Operating System Concepts, 7th Edition 2005, Silberschatz et al, John Wiley and Sons • How to Break Security,  2003, J. Whittaker and H. Thompson, Addison-Wesley • Papers!!! • Slides are available on the homepage • Username: oscourse, password: trustworthy

35. Course Structure • Credit points: • SWS: 5 (2+3) • "Credits": 7,5 • Track in the Diplom program: Informatik II • Schedule: • Lecture   (Wed, 11.40-13.20, S2/02 C120) • Exercise/Office hours (Thu, 11.40-13.20, S2/02 C120) • First: next week! • Lab instructions (TBA, S2/02 E212)  • Exam • March 12th, 2007, 9-11a.m. • Register!

36. Lab • Linux modules! • Investigate how the theoretical topics of the lecture apply to real OS’s • Optional, but one exam problem will require lab experience! • 2 lab assignments: • Introduction to device drivers (modules) • Build a module • OR: if you consider yourself experienced enough with Linux driver programming, contact us for a special assignment! • Poolroom E212 • Username and Password will be handed out

37. Related Seminars • Mobile Embedded Systems Tu, Oct 17th, S2|02 – E215 @ 13.30-15.10 • Secure/Trusted Operating Systems Thu, Oct 19th, S2|02 - E215 @ 11.40-13.20 www.deeds.informatik.tu-darmstadt.de

38. Dependable Embedded System & Software Group (DEEDS) Neeraj Suri Dept. of Computer Science TU Darmstadt, Germany Dependable Embedded Systems & SW Group www.deeds.informatik.tu-darmstadt.de

39. What isn’t a/an (Embedded) Computing System? • monitoring • dissemination • response • motes • sensors • FBW • XBW • UAV’s

40. Lets Define Emergent Embedded Systems (ES’s)… • Involve computing elements • … that are part of a larger system • … whose primary task is not standalone computation • … are often resource constrained • Only HW? Only SW? Standalone? Closed? NO…open system-of-systems, networked … • ES utility comes from both functionality & its being trusted (D+S+R) for usage: • dependable (D) • secure (S) • responsive (R) • monitoring • dissemination • response

41. What all we do and with whom… • Distributed/Networked OS • Testing • Robustness/Security Profiling • Internet level Client-Server Replication Strategies • Mobile Sensor Networks • Distributed Transactions • QoS: responsive, secure, dependable comm • Distributed Systems • Protocols: Agreement, Diagnosis • Integration • Power • Microsoft Research, Intel, SAP, IBM, Audi, Airbus, Daimler, Ericsson, European Commission, US-NSF…

42. … Application 1 Application p System Services Driver 1 Driver 2 OS kernel Driver 3 Driver n Hardware Layer Operating Systems Testing • Why test the OS? • Building block of many computing systems • Still causing most of SW-related system failures • Focus: device drivers • Challenges • Drivers are COTS components No access to source code • Located in OS kernel space Difficult to access • Unrestrictedly interaction with kernel structures Deep impact on system reliability • Research aims • Profile driver behavior at runtime • Tune testing methods to the operational profile of the driver • Improve OS/driver robustness

43. App 1 App 2 Operating System Driver 3 D1 D2 Operating System Error/Security Profiling • Techniques for robustness evaluation of OS’s • Robustness is about handling the unforeseeable • What happens when something goes wrong? • Which parts of the OS are affected? • Are errors in some modules more severe than others? • Can we do something about it? • Focus in problems in device drivers • Key components, tend to fail often • “Do you want to send an error report to Microsoft?”

44. Byzantine Fault-Tolerant Replication ? • Agreement Certificate: • f + 1 identical • responses • Execute • in this order Internet TCP/IP

45. Dependable Wireless Sensor Networks • Wireless Sensor Networks (WSN) • Applications • Tracking & Monitoring • Measurement • Data Transport • Responsiveness • Delivery reliability • Timeliness • Reliability modeling • Sustainability of WSN • Maintenance of deployed WSN • Failure detection and diagnosis

46. Wired Network GPRS WLAN UMTS Mobile Database Systems • Transaction management • ACID properties • Logging and recovery • Data replication • Challenges • Heterogeneity • Wireless network (WLAN, UMTS, …) • Mobile nodes (laptops, PDAs, …) • Perturbations • Unpredictable disconnections • Node/Communication failures • Infrastructure-based vs. ad-hoc • Mobile ad-hoc networks (MANETs) • Wireless Sensor Networks (WSNs)

47. Distributed Diagnosis • Why diagnosis • Maintenance: shall I change a component? (DECOS – Audi, etc.) • Fault tolerance: is a process faulty? Shall I trust it? • Challenge: unstable systems • Smaller electronic elements • Transient faults are more likely to happen • Spread of wireless networks • Access collisions, temporary disconnections • Diagnosis shall support the autonomic abilities of the systems • Self-healing, self-reconfiguring, etc…

48. SW-HW Integration: Design and Optimization • Integration/Mapping of mixed criticality applications • Satisfying constraints and • Optimizing multiple objectives • Design and optimization criteria • Dependability, real-time, • Power, resources utilization, cost • Developing heuristics/algorithm • Mapping feasibility and optimization • Fault-tolerance scheme • Assessment methodologies • Target applications: Dependable RT ES (automotive, avionics, control, seaborne)

49. Energy Efficient Dependable Systems • Trends • Heterogeneous systems • Increased dependence upon technology • Mobility  low voltage  smaller noise margins  more transient errors • Increased complexity • Integration/communication between systems • Energy Efficient Fault Tolerance • Evaluate • Characterize • Optimize/trade-off • Dimensions • Design-time vs. run-time • Time vs. space • System level vs. components level • Service degradations and reconfiguration