System Programming Lecture 1 What is System Programming?

1. System ProgrammingLecture 1What is System Programming? February 23, 2011 Engr. Naveed Khan Baloch http://groups.yahoo.com/group/sp_0708

2. Who am I? Engr. Naveed Khan Baloch Lecturer • CPED, UET Taxila • Office hours: Monday 09AM-10AM, Tuesday 10AM-11AM • Background: • MSc Computer Engineering • Research areas: • Current: Remote sensing, Satellite imaging, GIS • Other: Embedded Systems, RFID, Web Development, Image acquisition, Pattern Matching

3. Goals for Today • What is an Operating System (Review) • What is System Programming? • Why study System Programming? • Oh, and “How does this class operate?”

4. Technology Trends: Moore’s Law Moore’s Law 2X transistors/Chip Every 1.5 years Called “Moore’s Law” Gordon Moore (co-founder of Intel) predicted in 1965 that the transistor density of semiconductor chips would double roughly every 18 months. Microprocessors have become smaller, denser, and more powerful.

5. Societal Scale Information Systems The world is a large parallel system Microprocessors in everything Vast infrastructure behind them Massive Cluster Clusters Gigabit Ethernet InternetConnectivity Scalable, Reliable, Secure Services Databases Information Collection Remote Storage Online Games Commerce … MEMS for Sensor Nets

6. People-to-Computer Ratio Over Time Today: Multiple CPUs/person! Approaching 100s? From David Culler

7. New Challenge: Slowdown in Joy’s law of Performance 3X From Hennessy and Patterson, Computer Architecture: A Quantitative Approach, 4th edition, Sept. 15, 2006  Sea change in chip design: multiple “cores” or processors per chip • VAX : 25%/year 1978 to 1986 • RISC + x86: 52%/year 1986 to 2002 • RISC + x86: ??%/year 2002 to present

8. ManyCore Chips: The future is here “ManyCore” refers to many processors/chip 64? 128? Hard to say exact boundary How to program these? Use 2 CPUs for video/audio Use 1 for word processor, 1 for browser 76 for virus checking? Parallelism must be exploited at all levels • Intel 80-core multicore chip (Feb 2007) • 80 simple cores • Two floating point engines /core • Mesh-like "network-on-a-chip“ • 100 million transistors • 65nm feature size • Frequency Voltage Power Bandwidth Performance • 3.16 GHz 0.95 V 62W 1.62 Terabits/s 1.01 Teraflops • 5.1 GHz 1.2 V 175W 2.61 Terabits/s 1.63 Teraflops • 5.7 GHz 1.35 V 265W 2.92 Terabits/s 1.81 Teraflops

9. Computer System Organization Computer-system operation One or more CPUs, device controllers connect through common bus providing access to shared memory Concurrent execution of CPUs and devices competing for memory cycles

10. Increasing Software Complexity From MIT’s 6.033 course

11. Example: Some Mars Rover Requirements • Serious hardware limitations/complexity: • 20Mhz powerPC processor, 128MB of RAM • cameras, scientific instruments, batteries, solar panels • Many independent processes work together • Can’t hit reset button very easily! • Must reboot itself if necessary • Always able to receive commands from Earth • Further, all software may crash occasionally • Automatic restart with diagnostics sent to Earth • Periodic checkpoint of results saved? • Certain functions time critical: • Need to stop before hitting something • Must track orbit of Earth for communication

12. OS Tool: Virtual Machine Abstraction Application Operating System Hardware • Software Engineering Problem: • Turn hardware/software quirks  what programmers want/need • Optimize for convenience, utilization, security, reliability, etc… • For Any OS area (e.g. file systems, virtual memory, networking, scheduling): • What’s the hardware interface? (physical reality) • What’s the application interface? (nicer abstraction) Virtual Machine Interface Physical Machine Interface

13. Virtual Machines • Software emulation of an abstract machine • Make it look like hardware has features you want • Programs from one hardware & OS on another one • Programming simplicity • Each process thinks it has all memory/CPU time • Each process thinks it owns all devices • Different Devices appear to have same interface • Device Interfaces more powerful than raw hardware • Bitmapped display  windowing system • Ethernet card  reliable, ordered, networking (TCP/IP) • Fault Isolation • Processes unable to directly impact other processes • Bugs cannot crash whole machine • Protection and Portability • Java interface safe and stable across many platforms

14. Virtual Machines (con’t): Layers of OSs • Useful for OS development • When OS crashes, restricted to one VM • Can aid testing programs on other OSs

15. What does an Operating System do? • Silerschatz and Gavin: “An OS is Similar to a government” • Begs the question: does a government do anything useful by itself? • Coordinator and Traffic Cop: • Manages all resources • Settles conflicting requests for resources • Prevent errors and improper use of the computer • Facilitator: • Provides facilities that everyone needs • Standard Libraries, Windowing systems • Make application programming easier, faster, less error-prone • Some features reflect both tasks: • E.g. File system is needed by everyone (Facilitator) • But File system must be Protected (Traffic Cop)

16. What is an Operating System,… Really? Most Likely: Memory Management I/O Management CPU Scheduling Communications? (Does Email belong in OS?) Multitasking/multiprogramming? What about? File System? Multimedia Support? User Interface? Internet Browser? 

17. Operating System Definition (Cont.) • No universally accepted definition • “Everything a vendor ships when you order an operating system” is good approximation • But varies wildly • “The one program running at all times on the computer” is the kernel. • Everything else is either a system program (ships with the operating system) or an application program

18. Altair 8080 What if we didn’t have an Operating System? • Source CodeCompilerObject CodeHardware • How do you get object code onto the hardware? • How do you print out the answer? • Once upon a time, had to Toggle in program in binary and read out answer from LED’s!

19. Simple OS: What if only one application? • Examples: • Very early computers • Early PCs • Embedded controllers (elevators, cars, etc) • OS becomes just a library of standard services • Standard device drivers • Interrupt handlers • Math libraries

20. Assignment # 1 • Download the source code of Ubunto (Linux) from internet and make a list of 10 modules in the code with there function.

21. MS-DOS Layer Structure

22. More complex OS: Multiple Apps • Full Coordination and Protection • Manage interactions between different users • Multiple programs running simultaneously • Multiplex and protect Hardware Resources • CPU, Memory, I/O devices like disks, printers, etc • Facilitator • Still provides Standard libraries, facilities

23. Example: Protecting Processes from Each Other • Problem: Run multiple applications in such a way that they are protected from one another • Goal: • Keep User Programs from Crashing OS • Keep User Programs from Crashing each other • [Keep Parts of OS from crashing other parts?] • (Some of the required) Mechanisms: • Address Translation • Dual Mode Operation • Simple Policy: • Programs are not allowed to read/write memory of other Programs or of Operating System

24. Virtual Addresses Physical Addresses CPU MMU Address Translation • Address Space • A group of memory addresses usable by something • Each program (process) and kernel has potentially different address spaces. • Address Translation: • Translate from Virtual Addresses (emitted by CPU) into Physical Addresses (of memory) • Mapping often performed in Hardware by Memory Management Unit (MMU)

25. Data 2 Code Data Heap Stack Code Data Heap Stack Stack 1 Heap 1 Code 1 Stack 2 Data 1 Heap 2 Code 2 OS code OS data OS heap & Stacks Example of Address Translation Prog 1 Virtual Address Space 1 Prog 2 Virtual Address Space 2 Translation Map 1 Translation Map 2 Physical Address Space

26. Dual Mode Operation • Hardware provides at least two modes: • “Kernel” mode (or “supervisor” or “protected”) • “User” mode: Normal programs executed • Some instructions/ops prohibited in user mode: • Example: cannot modify page tables in user mode • Attempt to modify  Exception generated • Transitions from user mode to kernel mode: • System Calls, Interrupts, Other exceptions

27. Applications User Mode Standard Libs Kernel Mode Hardware UNIX System Structure

28. Course Administration • Instructor: Naveed Khan Baloch (Lecturer CPED) UET Taxila Office Hours: M 9-10, Tu 10-11 • Labs: CE-07/VIP Lab-08 • Group: www.groups.yahoo.com/group/SP_0708 • Course Email: naveed.khan@uettaxila.edu.pk

29. Grading • Rough Grade Breakdown • Quizzes 10%Assignments 10%MID Semester Exam 20%Lab Reports 20%End Semester Exam 40%

30. Class Schedule • Class Time: Wednesday • (08:40 AM to 10:20 AM) 07 Computer • (11:10 AM to 10:20 PM) 08 Computer • Venue: Computer Department

31. Attention 30 min. Break 30 min. Break 25 min. “In Conclusion, ...” Time Typical Lecture Format • 5-Minute Review • 30-Minute Lecture • 5- Minute Administrative Matters • 30-Minute Lecture • 5-Minute Break (water, stretch) • 25-Minute Lecture

32. Input Output Process What is System Programming..?? • Computer programming can be categorized into two categories .i.e • Application Programming • Systems Programming

33. Application Three Layered Approach • A system programmer may use a three layered approach for systems programming.

34. Methods of I/O • In the three layered approach if we are following the first approach we need to program the hardware. • The hardware can be programmed to perform I/O in three ways: • Programmed I/O • Interrupt driven I/O • Direct Memory Access

35. I/O Device I/O Controller CPU I/O Controllers • No I/O device is directly connected to the CPU. • I/O controller is located between the CPU and the I/O device.

36. Not only Programmable Hardware devices….. • Systems programming is not just the study of programmable hardware devices. To develop effective system software one needs to the internals of the operating system as well.

37. What more… • Processes • Inter-Process Communication • Interrupts/Signals • Network protocols (Communication) • Device Drivers (DOS/Windows)

38. Textbook • Text: Unix Network Programming (The Sockets Networking API) • Volume 1, 3rd Edition By W. Richard Stevens • Linux System Programming By Robert Love. (O’ REILLY) • Linux Device Drivers By Alessandro Rubini & Jonathan Corbet (O’ REILLY) • Online supplements • See “Tutorial” link on course website

39. Reference Books • Beej’s Guide to Network Programming • Beej’s Guide to C Programming • C++ The Complete Reference By Herbert Schildt • 4th Edition

40. Thanks