CS 2510 Graduate Operating Systems

1. CS 2510Graduate Operating Systems Jack Lange Assistant Professor University of Pittsbugh

2. Course Objectives • Understand basic OS principals • Understand historical background of OS design • Gain practical experience with current systems • x86 and Linux • Gain exposure to OS research • Gain experience in reviewing and evaluating research papers • Gain experience in presenting research papers

3. Administrivia • Instructor: Jack Lange • Email: jacklange@cs.pitt.edu • Office:Sennott Square #5407 • Office Hours: Monday 2-4PM • Teaching Assistant: Brian Kocoloski • Email: briankoco@cs.pitt.edu • Office:Sennott Square #6804 • Office Hours: TBD

4. Communication • Course homepage • http://www.cs.pitt.edu/~jacklange/teaching/cs2510-f12/ • Announcements, clarifications, corrections • Additional resources for projects • Google Group • http://groups.google.com/group/pitt-cs2510-f12 • pitt-cs2510-f12@googlegroups.com • Private discussion group • Open venue for class discussions and questions • Based on email (Pitt addresses) • Email me if you want to use a different one

5. Class Details • Inverse of other graduate courses • Lectures on Tuesdays • Provide background • Supplemental readings assigned (Check Schedule!) • Presentations on Thursdays • Student presents selected research paper • Other students read and provide a review of the paper before hand

6. Grading • Grading • Final (30%) • Presentations (10%) • Paper reviews (10%) • 4 Projects (50%) • Late policy • Submit by midnight of the due date • 10% penalty for every day late

7. Projects • Work Individually • C is required • Highly Recommended: OS or having some familiarity with Unix systems programming, preferably in C or C++ • All projects are in C • BUILDING software is 50% of the grade of this class 1-9

8. Reading papers • Mandatory readings are assigned throughout semester • Must submit a review for each assigned paper by midnight on the day before class • Each student must ask 1 relevant question during each presentation

9. Reviewing papers • Paper title and its author(s). • Brief one-line summary. • At least one paragraph of the most important ideas: perhaps a combination of their motivations, observations, interesting parts of the design, or clever parts of their implementation. • At least one paragraph of the largest flaws; maybe an experiment was poorly designed or the main idea had a narrow scope applicability. Being able to assess weaknesses as well as strengths is an important skill for this course and beyond. • At least one last paragraph where you state the relevance of the ideas today, potential future research suggested by the article, etc. • Reviewing and evaluating research papers is 10% of final grade

10. Presentations • Presenting research is a critical skill for graduate students • Large part of the class will be reading, evaluating and presenting research papers • Each student must do 1 • Present paper and answer questions about it • Develop own presentation (Cannot use author’s slides) • Be able to clearly present research ideas and think on your feet • Presentations will be graded (10% of final grade) • Evaluation will be based on presentation quality as well as technical knowledge

11. What makes a good presentation • What is the problem? • Why should people care about the problem? • How do you intend to address it (high level)? • How did you address it (w/ lowleveldetails of interesting components)? • Why is your approach superior to others? • What are the shortcomings of your approach? • Did your approach work? • Evaluation will be based on how well you answer these questions!

12. What makes a bad presentation • Lack of context • Reading from slides • Lots of text or data • Only focusing on technical details • Ignoring high level points

13. Why learn about Operating Systems? • Tangible reasons • Build or modify a real operating system • Administer and use system well • Tune application performance • Intangibles • Intrinsic curiosity • Understand how much of you computer system works • Gain/apply knowledge in other areas of Computer Science • Computer architecture and devices • Synchronization in programming languages • Data structures and algorithms • Performance analysis • Challenge of designing large, complex systems

14. What is an operating System • Not easy to define: Users Applications Operating System Hardware • OS: • Everything that isn’t an application or hardware • OS: • Software that abstract hardware into a useful form for applications • Standard Library • Resource Coordinator

15. First Function: Standard Library • Advantages of standard library • Allow applications to reuse common facilities • Make different devices look the same • Provide higher level abstractions • Challenges • What are the right abstractions?

16. Second Function: Resource Coordinator • Resource: “Anything valuable” • (e.g. CPU, memory, disk, network) • Advantages of resource coordinator • Virtualize resource so multiple users/applications can share • Protect applications from one another • Provide efficient and fair access to resources • Challenges • What mechanisms? • What policies?

17. What Functionality in OS? • No single right answer • Desired functionality depends on outside factors Technology Changes Expectations Users & Applications Operating System Computer Architecture • OS must adapt • Change abstractions provided to users • Change algorithms to implement those abstractions • Change low-level implementation to deal with hardware • Current operating systems driven by its evolution • Two distinct cases in history • Case 1: Computers are expensive • Case 2: Computers are cheap

18. History of the OS • Commercial systems (1950s) and HPC systems (early 1990s) • Enormousand expensive • Goal: Get the system working • Single operator/programmer/user runs and debugs at a time • OS functionality • Standard library -> No coordination of resources • Monitor that is always resident; transfer control to programs OS User Job Memory • Problem: Inefficient use of hardware • Performance Metrics • Throughput and Utilization

19. Batch Processing • Batch: Group of jobs submitted to machine together • Operator collects jobs; orders efficiently; runs one at a time • Role of OS: Same as before • Advantages • Amortize setup costs over many jobs • Keeps machine busy during a single users idle time • Disadvantage • User must wait for results until batch collected and submitted • If bug, receive memory and register dump; submit job again • Improve system throughput and utilization, but lose interactivity • Still the prevalent supercomputing model

20. Spooling • Problem: Mechanical I/O devices much slower than CPU • Read 17 cards/sec vs. execute 1000s instructions/sec • Spooling: Overlap I/O (e.g. card reading and line printing) with execution • Read punch cards to disk • Compute (reading and writing to disk) • Write output from disk to printer • New OS functionality • Buffering, DMA, interrupt handling • Advantage: Improves throughput and utilization • Disadvantage: Single job must wait during I/O for data

21. Multiprogrammed Batch Systems • Spooling provides pool of ready jobs • Keep multiple jobs resident in memory • OS chooses which job to run • When job waits for I/O, switch to another resident job OS User Job 1 Memory User Job 2 • New OS functionality • Job scheduling policies • Memory management and protection (virtual memory) • Advantage: Improves throughput and utilization • Disadvantage: Still not interactive User Job 2

22. History of the OS: Phase 2 • Introduction of inexpensive, fast devices • Keyboards and monitors  text editors and interactive debuggers • New set of performance trade-offs • Goal: True interactivity (via improved response time) • Time-sharing: Switch between jobs to give appearance of dedicated machine • Advantage • Users easily submit jobs and get immediate feedback • New OS functionality • More complex job scheduling, memory management • Concurrency control and synchronization

23. Personal Computers • Computers became cheap (late 70s, early 80s) • Dedicated machine per user • “Improved” functionality from OS • Remove time-sharing of multiple jobs • No protection • No virtual memory • OS becomes subroutine again • Conclusion: OS functionality changes with hardware and users OS User Job Memory

24. State of current systems • Large • 100k’s to millions of lines of code • 100-1000 person-years of work • Complex • Performance is important • Conflicting needs of different users • Poorly understood • System outlives any of its builders • Cannot remove all bugs • Behavior is hard to predict, tuning is done by guessing • Current trends • Multiprocessors • Networked systems • Cloud based systems

25. Distributed System Architecture

26. Distributed System Architecture

27. Distributed Memory Architecture

28. Message Passing • Explicit communication operations • Specify sender and receiver • Serialize data into a message • Transmit message over communication channel • Communication Channels • Sockets • E.g. Ethernet • RDMA (Remote direct memory access) • E.g. Infiniband • Shared memory • Becoming more prevalent on many-core architectures • The future OS architecture?

29. What we’ll learn • Understand basic OS principals • Understand historical background of OS design • Gain practical experience with current systems • x86 and Linux • Gain exposure to OS research • Gain experience in reviewing and evaluating research papers • Gain experience in presenting research papers

30. First assignment • Pick a paper and day for presentation • Send me (jacklange@cs.pitt.edu) an email with choice of date and email address I should use for communication • Presentation days will be assigned according to First Come First Serve (FCFS)