COT 4600 Operating Systems Fall 2010

1. COT 4600 Operating Systems Fall 2010 Dan C. Marinescu Office: HEC 439 B Office hours: Tu-Th 3:30-4:30 PM

2. Last time: Interpreters. Communication links. Today: Naming; Use by: value/name Binding and indirection Generic naming model Name mapping algorithms Comparing names; name discovery Practical design of naming scheme Contexts Name overloading Next time UNIX File System Lecture 7 – Tuesday September 14, 2010 Lecture 7

3. Naming • The three abstractions manipulate objects identified by name. • How could object A access object B: • Make a copy of object B and include it in A  use by value • Safe  there is a single copy of B • How to implement sharing of object B? • Pass to A the means to access B using its name  use by reference • Not inherently safe  both A and C may attempt to modify B at the same time. Need some form of concurrency control. Lecture 7

4. Binding and indirection • Names allow the system designer to: • organize the modules of a system and to define communication patterns among them • defer for a later time • to create object B referred to by object A • select the specific object A wishes to use • Indirection  decoupling objects from their physical realization through names. • Binding  linking the object to names. Examples: • A compiler constructs • a table of variables and their relative address in the data section of the memory map of the process • a list of unsatisfied external references • A linker binds the external references to modules from libraries Lecture 7

5. Generic naming model • Naming scheme  strategy for naming. Consists of: • Name space  the set of acceptable names; the alphabet used to select the symbols from and the syntax rules. • Universe of values  set of objects/values to be named • Name mapping algorithm  resolves the names, establishes a correspondence between a name and an object/value • Context  the environment in which the model operates. • Example: searching for John Smith in the White Pages in Orlando (one context) or in Tampa (another context). • Sometimes there is only one context  universal name space; e.g., the SSNs. • Default context Lecture 7

6. Figure 2.10 from the textbook Lecture 7

7. Operations on names in the abstract model • Simple models: value  RESOLVE (name, context) • The interpreter: • Determines the version of the RESOLVE (which naming scheme is used) • Identifies the context • Locates the object • Example: the processor • Complex models support: • creation of new bindings: status  BIND(name, value, context) • deletion of old bindings: status  UNBIND(name, value) • enumeration of name space: list  ENUMERATE(context) • comparing names status: result  COMPARE(name1,name2) Lecture 7

8. Name mapping • Name to value mapping • One-to-One  the name identifies a single object • Many-to-One  multiple names identify one objects (aliasing) • One-to-Many  multiple objects have the same name even in the same context. • Stable bindings  the mapping never change. Examples: • Social Security Numbers • CustomerId for customer billing systems Lecture 7

9. Name-mapping algorithms • Table lookup • Phone book • Port numbers  a port the end point of a network connection • Recursive lookup: • File systems – path names • Host names – DNS (Domain Name Server) • Names for Web objects - URL – (Universal Resource Locator) • Multiple lookup  searching through multiple contexts • Libraries • Example: the classpath is the path that the Java runtime environment searches for classes and other resource files Lecture 7

10. 1. Table lookup Figure 2.11 from the textbook Lecture 7

11. How to determine the context • Context references: • Default  supplied by the name resolver • Constant  built-in by the name resolver • Processor registers (hardwired) • Virtual memory (the page table register of an address space) • Variable  supplied by the current environment • File name (the working directory) • Explicit  supplied by the object requesting the name resolution • Per object • Looking up a name in the phone book • Per name  each name is loaded with its own context reference (qualified name). • URL • Host names used by DNS Lecture 7

12. Dynamic and multiple contexts • Context reference static/dynamic. • Example: the context of the “help” command is dynamic, it depends where you are the time of the command. • A message is encapsulated (added a new header, ) as flows down the protocol stack: • Application layer (application header understood only in application context) • Transport layer (transport header understood only in the transport context) • Network layer (network header understood only in the network context) • Data link layer (data link header understood only in the data link context) Lecture 7

13. 2. Recursive name resolution • Contexts are structured and a recursion is needed for name resolution. • Root  a special context reference - a universal name space • Path name  name which includes an explicit reference to the context in which the name is to be resolved. • Example: first paragraph of page 3 in part 4 of section 10 of chapter 1 of book “Alice in Wonderland.” • The path name includes multiple components known to the user of the name and to name solver • The least element of the path name must be an explicit context reference • Absolute path name  the recursion ends at the root context. • Relative path name  path name that is resolved by looking up its mot significant component of the path name Lecture 7

14. Example • AliceInWonderland.Chapter1.Section10.Part4.Page3.FirstParagraph Most significant   Least significant Lecture 7

15. 3. Multiple lookup • Search path  a list of contexts to be searched Example: the classpath is the path that the Java runtime environment searches for classes and other resource files • User-specific search paths  user-specific binding • The contexts can be in concentric layers. If the resolver fails in a inner layer it moves automatically to the outer layer. • Scope of a name  the range of layers in which a name is bound to the same object. Lecture 7

16. Comparing names • Questions • Are two names the same?  easy to answer • Are two names referring to the same object (bound to the same value)?  harder; we need the contexts of the two names. • If the objects are memory cells are the contents of these cells the same? Lecture 7

17. Name discovery • Two actors: • The exporter  advertizes the existence of the name. • The prospective user  searches for the proper advertisement.Example: the creator of a math library advertizes the functions. • Methods • Well-known names • Broadcasting • Directed query • Broadcast query • Introduction • Physical randezvoue Lecture 7

18. Practical design of naming schemes • Transition from abstract models to practical ones. • Name conflict  multiple modules have the same name. • How to avoid name conflicts when modules are developed independently often by different individuals? • The theoretical model tells us that we must specify a context for name resolution; but this is not so straightforward!! • Example: there are two versions of module A; one is used by module B and the other by module C. Conflict when module B uses C. Lecture 7

19. Single context  ambiguityWORD_PROCESSOR  (INITIALIZE,SPELL_CHECK)SPELL_CHECH (INITIALIZE) (but a different version of it) Figure 3.1 Lecture 7

20. Two distinct contexts  how does the interpreter choose the context?It needs a basis for the contexts. Figure 3.3 Lecture 7

21. Add a context reference to each module telling the interpreter which context to use for that module? Not feasible to tinker with someone else’s modules. Figure 3.4 Lecture 7

22. Have separate contexts but establish a link between them; the link points to the new context for the shared object Figure 3.5 Lecture 7

23. An elegant solution • We need a systematic rather than ad-hoc ways to deal with the problem because programs contain many references to objects. • Solution  associate the name of the object not with the object itself but with a structure consisting of pairs (original object, context). • Some programming languages implement such a structure called closure and use static references. • File systems rarely apply this solutions. Lecture 7

24. Metadata and name overloading • Metadata  information about an object that • is useful to know about the object but cannot be found inside the object • may be changed without changing the object, e.g., the last date a file was referenced. • Examples: • A user-friendly name; e.g., quadratic_solver  /user/local/bin/ linpack/quad.exe • The type of an object: e.g., Lecture9.ppt • Where to place metadata? • In the same place with the data; e.g., the Unix file system stores metadata in inodes • In a separate place  process control block stored in the kernel space. • Overloading the name  e.g., Lecture9.ppt • Overloading  adding metadata to a name • Contradicts the principle that names should only be used to reference objects • Creates a tension between the need to keep the name unchanged and the need to modify overloading information. Lecture 7

25. Names and overloading • Pure names  names with no overloading. • Fragile names  overloaded names which violate the idea of modular design. E.g.., adding location information on the file name. • Opaque name to a module  the name has no overloading the module knows how to interpret. • A name may pass through several modules before reaching a module which knows how to interpret it. Lecture 7

26. Addresses • Address  • A name used to locate an object • Not a pure name it is overloaded with metadata • Parsing an address provides a guide to the location • Often chosen from a compact set of integers • Does address adjacency correspond to physical adjacency? • True in some cases; e.g., sectors on a disk • False in other cases; e.g., the area code of a phone number • Can we apply arithmetic operations to addresses? • Yes in some cases; e.g., memory references • No in other cases; e.g; telephone numbers (actually the phone numbers do not forms a dense set!!) • Remember that overloading the causes name fragility Lecture 7

27. Changing addresses • Changing addresses not hidden by a level of indirection is tricky. • Solutions • Search for all addresses and change them • Make each user do a search for the object and if the search returns “object not found” detect that the address has been changed and supply the new address. • If possible bind the object to both the old and the new name • If the name is bond to an active agent place a forwarding scheme to the old address. • The optimal solution  hide an address under a level of indirection. We’ll discuss DNS, domain name services that map host names to IP addresses. Lecture 7

28. Unique names • Problems arise when names must be generated at a high rate., e.g., for online banking transactions (billions a week). • Danger of name collision. • Solution • Using a fine timestamp read a fine digital clock (say with a resolution of microseconds) and convert the binary representation of the timestamp to a string of characters. • Use a random number generator with a very large name space. • For objects with a binary representation (e.g., files, images) use the object itself. • Hashing algorithms such as SHA (Secure Hash Algorithm) avoid the problem of long names. HSA produces names of fixed length. Lecture 7

29. Hierarchical naming schemes • Think about naming in the Internet with hundred millions of hosts. • Unfeasible with a central authority. • Domain names • E.g., boticelli.cs.ucf.edu • How to relate a hierarchical naming scheme used by Internet with the flat naming schemes used for MAC addresses? • MAC addresses do not have any overloading • ARP • RARP • DHCP Lecture 7

30. Computer System Organization • Operating Systems (OS)  software used to • Control the allocation of resources (hardware and software) • Support user applications • Sandwiched between the hardware layer and the application layer • OS-bypass: the OS does not hide completely the hardware from applications. It only hides dangerous functions such as • I/O operations • Management function • Names  modularization Lecture 7

31. Figure 2.16 from the textbook Lecture 7

32. The hardware layer • Modules representing each of the three abstractions (memory, interpreter, communication link) are interconnected by a bus. • The bus  a broadcast communication channel, each module hears every transmission. • Control lines • Data lines • Address lines • Each module • is identified by a unique address • has a bus interface • Modules other than processors need a controller. Lecture 7

33. Figure 2.17 from the textbook Lecture 7

34. Bus sharing and optimization • Communication  broadcast • Arbitration protocol  decide which module has the control of the bus. Supported by hardware: • a bus arbiter circuit • distributed among interfaces – each module has a priority • daisy chaining • Split-transaction  a module uses the arbitration protocol to acquire control of the bus • Optimization: • hide the latency of I/O devices • Channels  dedicated processors capable to execute a channel program (IBM) • DMA (Direct Memory Access) • Support transparent access to files: • Memory Mapped I/O Lecture 7

35. Optimization • Direct Memory Access (DMA): • supports direct communication between processor and memory; the processor provides the disk address of a block in memory where data is to be read into or written from. • hides the disk latency; it allows the processor to execute a process while data is transferred • Memory Mapped I/O: • LOAD and STORE instructions access the registers and buffers of an I/O module • bus addresses are assigned to control registers and buffers of the I/O module • the processor maps bus addresses to its own address space (registers) • Supports software functions such as UNIX mmap which map an entire file. • Swap area: disk image of the virtual memory of a process. Lecture 7

36. DMA Transfer Lecture 7 36

37. B. The software layer: the file abstraction • File: memory abstraction used by the application and OS layers • linear array of bits/bytes • properties: • durable  information will not be changed in time • has a name  • allows access to individual bits/bytes  has a cursor which defines the current position in the file. • The OS provides an API (Application Programming Interface) supporting a range of file manipulation operations. • A user must first OPEN a file before accessing it and CLOSE it after it has finished with it. This strategy: • allows different access rights (READ, WRITE, READ-WRITE) • coordinate concurrent access to the file • Some file systems • use OPEN and CLOSE to enforce before-or-after atomicity • support all-or-nothing atomicity e.g., ensure that if the system crashes before a CLOSE either all or none of WRITEs are carried out Lecture 7

38. Open and Read operations Lecture 7 38

39. Unix and Unix File System • Developed in early 1970s at Bell Labs for PDP computers made by Digital • Lineage  MULTICS developed at MIT in 1960s • Versions: • Berkeley Unix (BSD) • GNU/Linux • Darwin – part of MAC OS • Unix file system – hierarchical data organization: blocks  files  directories  file systems • the objects: • files – linear arrays of blocks of data; each file has a cursor giving the current position • directories – collections of files; tree structure • metadata - useful information about the file, not contained in the file (e.g., owner, access modes, last modified date, length, etc.) • supports: • creation, deletion, renaming of files and directories • reading data from and writing data to a file • reading and writing metadata describing a file Lecture 7

40. API for the Unix File System OPEN(name, flags, model)  connect to a file Open an existing file called name, or Create a new file with permissions set to mode if flags is set. Set the file pointer (cursor) to 0. Return the file descriptor (fd). CLOSE(fd)  disconnect from a file Delete file descriptor fd. READ(fd, buf,n)  read from file Read n bytes from file fd into buf; start at the current cursor position and update the file cursor (cursor = cursor + n). WRITE(fd, buf,n)  write to file Write n bytes to the file fd from buf; start at the current cursor position and update the file cursor (cursor = cursor + n). SEEK(fd, offset,whence)  move cursor of file Set the cursor position of file fd to offset from the position specified by whence (beginning, end, current position) Lecture 7

41. API for the Unix File System (cont’d) FSYNC(fd)  make all changes to file fd durable. STAT(name) read metadata CHMOD, CHOWN  change access mode/ownership RENAME(from_name,to_name)  change file name LINK(name, link_name) create a hard link UNLINK(name) remove name from directory SYMLINK(name, link_name) create a symbolic link MKDIR(name) create directory name RMDIR(name) delete directory name CHDIR(name)  change current directory to name CHROOT  Change the default root directory name MOUNT(name,device)  mount the file system name onto device UNMOUNT(name)  unmount file system name Lecture 7

42. Some of the Challenges • Naming storage objects • Computers can accommodate easily numbers which requires a fixed-length field. • Humans remember names of different length. • Dealing coherently with logical and physical storage objects • Applications deal with logical objects (e.g., files) of different size • Data is stored on disk in blocks of fixed size • Ensuring atomicity in face of system failures • Concurrency Control and file sharing Lecture 7

43. Naming layers • Unix file system uses a number of layers to hide the implementation of the storage abstraction from the users. • User-oriented names • 1. Symbolic link layer integrates multiple file systems with symbolic names • 2. Absolute path name layer provides a root for the naming hierarchies • 3. Path name layer  organizes files into naming hierachies • Machine-user interface • 4. File name layer  Supplies human-oriented names for files • Machine-oriented names • 5. Inode number layer  provides machine-oriented names for files • 6. File layer  organizes blocks into files • 7. Block layer  Provides the physical address of data blocks Lecture 7

44. 7. Block layer • The storage device (disk)  a linear array of cells/blocks • Block  fixed-size allocation unit (e.g., 512 bytes, 2,048 bytes); occupies several disk sectors. • Block name  integer from a compact set, the offset from the beginning of the device • Boot block  usually contains a boot program; Has a well-known name, 0. • Super block  provides a description of the layout of the file system on the disk. Has a well-known name,1. • Bitmap to keep track of free blocks and of defective blocks. Lecture 7

45. Disk Figure 2.23 Lecture 7

46. Disk layout Figure 2.20 Lecture 7

47. Figure 2.22 Lecture 7

48. 6. File layer • A file consists of multiple blocks. • inode (index node) container for the metadata about the file; context for naming the blocks of the file structureinode integerblock_number[N] // the number of the block in the file integersize // file size in bytes • Name mapping algorithm: procedureINDEX_TO_BLOCK_NUMBER (inodeinstancei, integer index) returns integer return i_block_numbers[index] • Indirect blocks  blocks that contain block numbers rather than data  to accommodate large file. • Doubly indirect blocks  blocks that contain block numbers of indirect blocks • Example: UNIX V6 • the first N entries in i_block_numbersare indirect blocks and the N+1 is a doubly indirect block • the block size is 512 bytes; • an index consists of 2 bytes  an indirect block may contain 256 block numbers • the maximum file size is: (N-1) x 256 + 256 x 256 blocks Lecture 7

49. 5. Inode number layer • inodes are named and passed around by name (inode number) • inode numbers are a compact set • inode_table • maps inode names to the starting block of the inode • stored at a known-location (e.g., at the beginning of the storage device) • Inode manipulation functions: allocate, dealocate, add to list of free inodes, remove from list of free inodes • Name mapping algorithm  procedure that returns the block that contains the byte at offset in a file identified by inode_number: procedureINODE_NUMBER_TO_BLOCK(integeroffset, integer inode _number) returnsblock inodeinstance i INODE_NUMBER_TO_INODE [inode,number] offset_in_block offset/blocksize b_number  INDEX_TO_BLOCK_NUMBER(i, offset_in_block) return BLOCK_NUMBER_TO_BLOCK [b_number] Lecture 7

50. 4. File name layer • Maps human-oriented names to machine-oriented names. • Hides the metadata required by file management from the user. • Directory  • durable object providing the context for binding between a file name and an inode number • it is a file • the inode data structure is extended with a type field to indicate if the inode is for a file or for a directory. • To create a file: • allocate an inode • initialize the metadata for the file • bind the file name to the inode number Lecture 7