Computer Organization & Design

1. Computer Organization & Design

2. Temporal locality The principle stating that if a data location is referenced then it will tend to be referenced again soon. spatial locality The locality principle stating that if a data location is referenced, data locations with nearby addresses will tend to be referenced soon. block The minimum unit of information that can be either present or not present in the two-level hierarchy. hit rate The fraction ofmemory accesses found in a cache. If the data requested by the processor appears in some block in the upper level, this is called a hit

3. miss rate The fraction of memory accesses not found in a level of the memory hierarchy. hit time The time required to access a level of the memory hierarchy, including the time needed to determine whether the access is a hit or a miss. miss penalty The time required to fetch a block into a level of the memory hierarchy from the lower level, including the time to access the block, transmit it from one level to the other, and insert it in the level that experienced the miss.

6. direct-mapped cache A cache structure in which each memory location is mapped to exactly one location in the cache.

7. Before the request, the cache contains a collection of recent references XI, X2, ••• , Xn -1' and the processor requests a word Xn that is not in the cache. This request results in a miss, and the word Xn is brought from memory into cache. The tags contain the address information required to identify whether a word in the cache corresponds to the requested word. The tag needs only to contain the upper portion of the address, corresponding to the bits that are not used as an index into the cache. valid bit A field in the tables of a memory hierarchy that indicates that the associated block in the hierarchy contains valid data. (Block address) modulo (Number of cache blocks in the cache)

11. Handling Cache Misses The control unit must detect a miss and process the miss by fetching the requested data from memory. If the cache reports a hit, the computer continues using the data as if nothing had happened.

12. I. Send the original PC value to the memory. 2. Instruct main memory to perform a read and wait for the memory to complete its access. 3. Write the cache entry, putting the data from memory in the data portion of the entry, writing the upper bits of the address (from the ALU) into the tag field, and turning the valid bit on. 4. Restart the instruction execution at the first step, which will refetch the instruction, this time finding it in the cache.

13. write-through A scheme in which writes always update both the cache and the memory, ensuring that data is always consistent between the two. write buffer A queue that holds data while the data are waiting to be written to memory. write-back A scheme that handles writes by updating values only to the block in the cache, then writing the modified block to the lower level of the hierarchy when the block is replaced. A cache miss on an instruction fetch requires that the processor ``stall'' or wait until the instruction is available from main memory. Read-stallcycles = Reads/Program X Read miss rate X Read miss Penalty For writes, the expression is similar, except that the effect of the write buffer must be added in: Write-stall cycles = Memory accesses X Cache miss rate X Cache miss Penalty

14. Write Through contd.. First fetch the words of the block from memory. After the block is fetched and placed into the cache, overwrite the word that caused the miss into the cache block. Write the word to main memory using the full address. Every write causes the data to be written to main memory. These writes will take a long time, likely at least 100 processor clock cycles, and could slow down the processor Use a Write Buffer - A write buffer stores the data while it is waiting to be written to memory. After writing the data into the cache and into the write buffer, the processor can continue execution. A write to main memory completes, the entry in the write buffer is freed

16. Cache Prformance CPU time can be divided into the clock cycles that the CPU spends executing the program and the clock cycles that the CPU spends waiting for the memory system. Assume that the costs of cache accesses that are hits are part of the normal CPU execution cycles. CPU time = (CPU execution clock cycles + Memory-stall clock cycles) x Clock cycle time memory-stall clock cycles come primarily from cache misses Memory-stall clock cycles can be defined as the sum of the stall cycles coming from reads plus those coming from writes: Memory-stall clock cycles = Read-stall cycles + Write-stall cycles

17. The read-stall cycles can be defined in terms of the number of read accesses per program, the miss penalty in clock cycles for a read, and the read miss rate: Read-stall cycles = Reads/Program X Read miss rate X Read miss penalty Write-stall cycles = (Writes/Program X Write Miss Rate X Write Miss Penalty) + Write buffer stalls combine the reads and writes by using a single miss rate and the miss penalty: Memory-stall clock cycles = Memory accesses X Miss rate X Miss penalty Program Memory-stall clock cycles = Instructions/Program X Misses/Instrn X Miss Penalty

18. The write buffer stalls depend on the timing of writes, and not just the frequency, it is not possible to give a simple equation to compute such stalls. Cache Coherence * With multiple caches, one CPU can modify memory at locations that other CPUs have cached. # For example: # CPU A reads location x, getting the value N . # Later, CPU B reads the same location, getting the value N . # Next, CPU A writes location x with the value N - 1 . # At this point, any reads from CPU B will get the value N , while reads from CPU A will get the value N - 1 .

19. Reducing Cache Misses by More Rexible Placement of Blocks A block can be placed in any location in the cache. Such a scheme is called fully associative because a block in memory may be associated with any entry in the cache. To find a given block in a fully associative cache, all the entries in the cache must be searched because a block can be placed in anyone. fully associative cache A cache structure in which a block can be placed in any location in the cache. set-associative cache Acache that has a fixed number oflocations (at least two) where each block can be placed.

22. Locating a block in the Cache

23. Virtual Memory virtual memory A technique that uses main memory as a "cache" for secondary storage. The principle of locality enables virtual memory as well as caches, and virtual memory allows us to efficiently share the processor as well as the main memory. To allow multiple programs to share the same memory, we must be able to protect the programs from each other, ensuring that a program can only read and write the portions of main memory that have been assigned to it. page fault An event that occurs when an accessed page is not present in main memory. virtual address An address that corresponds to a location in virtual space and is translated by address mapping to a physical address when memory is accessed.

24. address translation Also called address mapping. The process by which a virtual address is mapped to an address used to access memory. Virtual memory also simplifies loading the program for execution by providing relocation. Relocation maps the virtual addresses used by a program to different physical addresses before the addresses are used to access memory. This relocation allows us to load the program anywhere in main memory. physical address An address In main memory

25. protection A set of mechanisms for ensuring that multiple processes sharing the processor, memory, or I/O devices cannot interfere, intentionally or unintentionally, with one another by reading or writing each other's data. These mechanisms also isolate the operating system from a user process.

30. translation-Iookaside buffer (TLB) A cache that keeps track of recently used address mappings to avoid an access to the page table.

31. I. Support at least two modes that indicate whether the running process is a user process or an operating system process, variously called a supervisor process, a kernel process, or an executive process. 2. Provide a portion of the processor state that a user process can read but not write. This includes the user/supervisor mode bit, which dictates whether the processor is in user or supervisor mode, the page table pointer, and the TLB. To write these elements the operating system uses special instructions that are only available in supervisor mode. 3. Provide mechanisms whereby the processor can go from user mode to supervisor mode, and vice versa. The first direction is typically accomplished by a system call exception, implemented as a special instruction (syscall in the MIPS instruction set) that transfers control to a dedicated location in supervisor code space. As with any other exception, the program counter from the point of the system call is saved in the exception PC (EPC), and the processor is placed in supervisor mode. To return to user mode from the exception, use the retllrn from exception (ERET) instruction, which resets to user mode and jumps to the address in EPC

33. Implementing Protection with Virtual Memory Support at least two modes that indicate whether the running process is a user process or an operating system process, variously called a supervisor process, a kernel process, or an executive process. Provide a portion of the processor state that a user process can read but notwrite. This includes the user/supervisor mode bit, which dictates whether the processor is in user or supervisor mode, the page table pointer, and the TLB. Provide mechanisms whereby the processor can go from user mode to supervisor mode, and vice versa. The first direction is typically accomplished by a system call exception, implemented as a special instruction (syscall in the MIPS instruction set) that transfers control to a dedicated location in supervisor code space

35. Memory Hierarchy

36. Charactristics Behavior: Input (read once), output (write only, cannot be read), or storage (can be reread and usually rewritten). • Partner: Either a human or a machine is at the other end of the I/O device, either feeding data on input or reading data on output. • Data rate: The peak rate at which data can be transferred between the I/O device and the main memory or processor. It is useful to know what maximum demand the device may generate.

37. nonvolatile Storage device where data retains its value even when power is removed. track One of thousands of concentric circles that makes up the surface of a magnetic disk. sector One of the segments that make up a track on a magnetic disk; a sector is the smallest amount of information that is read or written on a disk. Seek – The process of positioning a read/write head over the proper track on a disk

39. Bus Basics A bus generally contains a set of control lines and a set of data lines. The control lines are used to signal requests and acknowledgments, and to indicate what type of information is on the data lines. The data lines of the bus carry information between the source and the destination This information may consist of data, complex commands, or addresses. bus transaction A sequence of bus operations that includes a request and may include a response, either ofwhich may carry data. A transaction is initiated by a single request and may take many individual bus operations. A bus transaction includes two parts: sending the address and receiving or sending the data

40. Buses are traditionally classified as processor-memory buses or 110 bl/ses. Processor- memory buses are short, generally high speed, and matched to the memory system so as to maximize memory-processor bandwidth. I/O buses, by contrast, can be lengthy, can have many types of devices connected to them, and often have a wide range in the data bandwidth of the devices connected to them backplane bus A bus that is designed to allow processors, memory, and I/O devices to co exist on a single bus.

41. synchronous bus A bus used to interconnect devices that comprise a computer system where the timing of transactions between devices is under the control of a synchronizing clock signal. A device connected to a synchronous bus must guarantee to respond to a command within a period set by the frequency of the clock signal or a transmission error will occur. Such buses are usually employed in closely controlled processor backplane environments where device characteristics and interdevice signal delays are known. synchronous bus A bus used to interconnect devices that comprise a computer system where the timing of transactions between devices is under the control of a synchronizing clock signal. A bus that includes a clock in the control lines and a fixed protocol for communicating that is relative to the clock. asynchronous bus A bus that uses a handshaking protocol for coordinating usage rather than a clock; can accommodate a wide variety of devices of differing speeds.

42. The responsibilities of the operating system arise from three characteristics of I/O systems: Multiple programs using the processor share the I/O system. 2. I/O systems often use interrupts (externally generated exceptions) to communicate information about I/O operations. Because interrupts cause a transfer to kernel or supervisor mode, they must be handled by the operating system (OS). 3. The low-level control of an I/O device is complex because it requires managing a set of concurrent events and because the requirements for correct device control are often very detailed.

43. Three types of communication I. The OS must be able to give commands to the I/O devices. These commands include not only operations like read and write, but also other operations to be done on the device, such as a disk seek. 2. The device must be able to notify the OS when the I/O device has completed an operation or has encountered an error. For example, when a disk completes a seek, it will notify the os. 3. Data must be transferred between memory and an I/O device. For example, the block being read on a disk read must be moved from disk to memory.

44. Giving Commands to I/O Devices memory-mapped I/O An I/O scheme in which portions of address space are assigned to I/O devices and reads and writes to those addresses are interpreted as commands to the I/O device. Two methods are used to address the device: memory-mapped I/O and special I/O instructions. I/O Instructions A dedicated instruction that is lIsed to give a command to an va device and that specifies both the device number and the command word (or the location of the command word in memory). ** Device Controllers are used for both operations

45. The process of periodically checking status bits to see if it is time for the next I/O operation, as in the previous example, is called polling. Polling is the simplest way for an I/O device to communicate with the processor. The I/O device simply puts the information in a Status register, and the processor must come and get the information. interrupt-driven I/O An I/O scheme that employs interrupts to indicate to the processor that an I/O device needs attention. direct memory access (DMA) A mechanism that provides a device controller the ability to transfer data directly to or from the memory without involving the processor.

46. handshaking protocol A series of steps used to coordinate asynchronous bus transfers in which the sender and receiver proceed to the next step only when both parties agree that the current step has been completed. The responsibilities of the operating system arise from three characteristics of I/O systems: I. Multiple programs using the processor share the I/O system. 2. I/O systems often use interrupts (externally generated exceptions) to communicate information about I/O operations. Because interrupts cause a transfer to kernel or supervisor mode, they must be handled by the operating system (OS). 3. The low-level control of an I/O device is complex because it requires managing a set of concurrent events and because the requirements for correct device control are often very detailed.

47. The OS guarantees that a user's program accesses only the portions of an I/O device to which the user has rights. For example, the OS must not allow a program to read or write a file on disk if the owner of the file has not granted access to this program. In a system with shared I/O devices, Protection could not be provided if user programs could perform I/O directly. • The OS provides abstractions for accessing devices by supplying routines that handle low-level device operations. The OS handles the interrupts generated by I/O devices, just as it handles the exceptions generated by a program• • The OS tries to provide equitable access to the shared I/O resources, as well as schedule accesses in order to enhance system throughput.

48. Communicating with the Processor The process of periodically checking status bits to see if it is time for the next I/O operation, as in the previous example, is called polling. Polling is the simplest way for an I/O device to communicate with the processor. The I/O device simply puts the information in a Status register, and the processor must come and get the information. The processor is totally in control and does all the work An I/O interrupt is asynchronous with respect to the instruction execution. That is, the interrupt is not associated with any instruction and does not prevent the instruction completion. In addition to the fact that an I/O interrupt has occurred, we would like to convey further information such as the identity of the device generating the interrupt.

49. direct memory access (DMA) A mechanism that provides a device controller the ability to transfer data directly to or from the memory without involving the processor. I/ O Performance Measures Transaction processing : A type of application that involves handling small short operations (called transactions) that typically require both I/O and computation. Transaction processing applications typically have both response time requirements and a performance measurement based on the throughput of transactions.

50. most popular of such benchmarks has five phases, using 70 files: • MakeDir: Constructs a directory subtree that is identical in structure to the given directory subtree • Copy: Copies every file from the source subtree to the target subtree • ScanDir: Recursively traverses a directory subtree and examines the status of every file in it • ReadAIl: Scans every byte of every file in a subtree once • Make: Compiles and links all the files in a subtree