1 / 84

Chapter 2. Data Storage

Chapter 2. Data Storage. Outline. Memory hierarchy Hardware: Disks Access Times Example - Megatron 747 Optimizations Disk failure RAIDs. Users. DBMS’s. Operating Systems. Hardware - Data Storage. As Virtual Memory. Disk. File System. The Memory Hierarchy. DBMS. Programs,

burke-sanford
Download Presentation

Chapter 2. Data Storage

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Chapter 2. Data Storage Chapter 2

  2. Outline Memory hierarchy Hardware: Disks Access Times Example - Megatron 747 Optimizations Disk failure RAIDs Chapter 2

  3. Users DBMS’s Operating Systems Hardware - Data Storage Chapter 2

  4. As Virtual Memory Disk File System The Memory Hierarchy DBMS Programs, Main-memory DBMS’s Tertiary Storage Main memory Cache Chapter 2

  5. Cache • The cache is an integrated circuit or part of the processor’s chip • Holding data or machine instructions • Copy from main-memory • If data being expelled from the cache has been modified, then the new value must be copied into the main memory. • Typical performance • Capacities up to a megabyte • Access time: 10 nanoseconds (10-8 seconds) • Moving data bet. Cache and main memory: 100 nanoseconds (10-9 seconds) Chapter 2

  6. Main Memory • Everything that happens in the computer is resident in main memory • Capacity: around 100 Mbyte to 10 Gbyte • Random access • Typical access time is 10-100 nanoseconds Chapter 2

  7. Virtual Memory • Is a part of disk • In a 32-bit address machine • Virtual memory grows up to 232 bytes (4 Gbyte) • Data is moved between disk and main memory in entire blocks, which are also called pages in main memory • Main-memory database systems Chapter 2

  8. file buffer Secondary Storage (1) • Slower, more capacious than main memory • Random access • magnetic, optical, magneto-optical disks • Disk read/write are done by moving a chuck of bytes called blocks (or pages) Chapter 2

  9. Secondary Storage (2) • Accessing a block: 10-30 milliseconds • Recently, one disk unit can store data ranging from 10 to 32 Gbytes • A machine can have several disk units Chapter 2

  10. Tertiary Storage (1) • Have been developed to hold data volumes measured in terabytes • Compared with secondary storage, it offers • Higher read/write times • Larger capacities and smaller cost per byte • Not random access in general Chapter 2

  11. Tertiary Storage (2) • Kinds of tertiary storage devices • Ad-hoc tape storage • Optical-disk juke boxes: CD-ROMs • Tape silo: an automated version ad-hoc tape storage • Capacities • CD: 2/3 Gbytes, 2.3 Gbytes • Tapes: 50 Gbytes • Access time: about 1000 times slower than secondary memory Chapter 2

  12. Volatile and Nonvolatile • Volatile vs. nonvolatile storage • Flush memory • A form of main memory • Nonvolatile • Becomes economical • RAM disk • A battery-backed main memory Chapter 2

  13. Y (10 y bytes) 13 Tertiary 12 11 Secondary 10 9 Main zip disk 8 7 floppy disk Cache 6 5 2 1 0 -1 -2 -3 -4 -5 -6 -7 -8 -9 X (10 seconds) Access Time vs. Capacity Chapter 2

  14. Moore’s Law • Gordon Moore observed that the followings double every 18 months • The speed of processors, i.e., the number of instructions executed per second and the ratio of the speed to cost of a processor • The cost of main memory per bit and the number of bits that can be put on one chip • The cost of disk per bit and the number of bytes that a disk can hold • Not applicable to • Main memory access time, disk access time Chapter 2

  15. A typical disk … Terms: Platter, Head, Actuator Cylinder, Track Sector, Block, Gap Disks Chapter 2

  16. Disks: A Top View • Cylinder, Track, Sector, Gap • Gaps often represents about 10% of the total tracks • A entire section cannot be used if portion of it gets destroyed • Typically a block consists of one or more sectors. top view Chapter 2

  17. Processor Bus Main Memory Disk Controller Disks The Disk Controller • Controls one or more disk drives • controlling the mechanical actuator • selecting a surface or a sector on that surface • Transferring bits via a data bus Chapter 2

  18. Disk Storage Characteristics (as of 1999) • Rotation speed of the disk assembly • 5400 RPM (one rotation every 11 milliseconds) • Number of platters per unit • Typical disk drive: 5 platters (10 surfaces) • Floppy/zip disk: 1 platter (2 surfaces) • Number of tracks per surface • Have as many as 10,000 tracks • 3.5 inch diskette : 40 tracks • Number of bytes per track • Common disk: 105 or more bytes • 3.5 inch diskette: 150K Chapter 2

  19. Characteristics Have 4 platters (8 surfaces) 8192 (213) tracks per surface On average 256 (28) sectors per track 512 (29) bytes per sector Diameters of tracks outermost track is 3.5 inches innermost track is 1.5 inches Track consists of two parts gap: 10 % data: 90% Megatron 747 Disk (1) Chapter 2

  20. Megatron 747 Disk (2) • The capacity of the disk • 8 surfaces * 8192 tracks * 256 sectors * 512 bytes = 8G bytes • A single track on average • 256 sectors * 512 bytes = 128K bytes = 1 Mbits • A cylinder is of 1 Mbytes on average • If a block is 4096 bytes (212) • A block uses 8 sectors (= 4096 bytes / 512 bytes) • A track consists of 32 blocks (= 256 sectors / 8) Chapter 2

  21. If each track had the same number (i.e. 256) of sectors, then the density of bits around the tracks would be greater Length of the outermost track 0.9 * 3.5 *  ≒ 9.9 inch 1 megabit / 9.9 ≒ 100,000 bits per inch Length of the innermost track 0.9 * 1.5 *  ≒ 4.2 inch 1 megabit /4.2 ≒ 250,000 bits per inch Each track in Megatron 747 has the different numbers of sectors outer: 320 sectors middle: 250 sectors inner: 192 sectors The outermost track 1,801,800 bit / 9.9 ≒ 182,000 bpi The innermost track 47,880 bit / 4.2 ≒ 114,000 bpi Megatron 747 Disk (3) Chapter 2

  22. The Latency of The Disk • Disk access time • seek time • rotational delay • transfer time • others block x in memory I want block X disk access time Chapter 2

  23. In range 3 or 20x Time x 1 Max Cylinders Traveled Seek Time • The time to position the head assembly at the proper cylinder • 0(zero): already to be at the proper cylinder • Otherwise: move to be at the proper cylinder Chapter 2

  24. Head Here Block I Want Rotational latency Time • The time for disk to rotate the first of the sectors containing the block • One rotation takes 10 ms, so rotational latency on average 5 ms. Chapter 2

  25. Transfer Time/Other delays • Transfer Time • the time to read/writes the data on the appropriate disk surface • 10 Mbytes per second • Other delays (here, those are neglected) • taken by the processor and disk controller • due to contention for the disk controller • other delays due to contention Chapter 2

  26. Modifying Blocks • Not possible to modify a block on disk directly • Sequence of procedures • Read block (time: rt) • Modify in memory (time: mt) • Write block (time: wt) • Verify (time: vt) if appropriate • Total time • rt + mt + wt + vt Chapter 2

  27. Example 2.3 (1) • Let us examine the time to read a 4096-byte block from the Megatron 747 disk • Characteristic • 4 platters (8 surfaces), 1 surface = 8192 tracks • 1 track = 256 sectors, 1 sector = 512 bytes • Disk rotates at 3840 RPM, one rotation = 1/64 of a second • To move the head assembly • 1ms (to start and stop)+ 1ms for every 500 cylinders • Heads move one track in 1.002 ms • To move heads from innermost to outermost track • 1 + (8192 / 500) = 17.4 ms Chapter 2

  28. Example 2.3 (2) • Minimum time (the best case) • No seek time, no rotational latency, only transfer time • Note: 1 track = 256 sectors, 1 sector = 512 bytes • 4096 bytes / 512 bytes = 8 sectors (including 7 gap) • gaps/sectors occupy 10%/90% of track • A track has 256 gaps and 256 sectors • 36 * 7/256 + 324 * 8/256 = 11.109 degrees • (11.109/360)/64 = 4.8e-4 seconds = 0.5 ms Chapter 2

  29. Example 2.3 (3) • Maximum time (the worst case) • full seek time and rotational latency, plus transfer time • full seek time: 17.4 ms • full rotational time: 1/64 of a second = 15.6 ms • transfer time: 0.5 ms • 17.4 + 15.6 + 0.5 = 33.5 ms Chapter 2

  30. 4096 Average travel 2048 0 0 4096 8192 Starting track Example 2.3 (4) • Average Time • Transfer time: 0.5 ms • Average rotational time: half of the full rotation = 7.8 ms • Average seek time • average distance traveled = 1/3 of the disk = 2730 cylinders • 1+ 2730/500 = 6.5ms • 0.5 + 7.8 + 6.5 = 14.8 ms Chapter 2

  31. RAM model vs. I/O model computation • I/O model computation • Dominance of I/O cost • Remember, 105 - 106 in-memory operations take the same time as one disk I/O • Should minimize the number of block accesses • Data Structure vs. File Processing Chapter 2

  32. Using Secondary Storage Effectively • In general database • Whole databases are much too large to fit in main memory • Key parts of databases are buffered in main memory • Disk I/O’s occur frequently • Main memory sorts (such as “Quick sort”) are inadequate Chapter 2

  33. Merge Sort Chapter 2

  34. Two-Phase, Multiway Merge-Sort (1) • Phase 1 • Sort main-memory-sized pieces of the data • Fill all available main memory with blocks • Sort the records in main memory • Write the sorted records Chapter 2

  35. Two-Phase, Multiway Merge-Sort (2) • Phase 2 • Merge all the sorted sublists into a single sorted list • Find the smallest key among the first remaining elements of all the lists • Move the smallest element to the first available position of the output block • If output block is full, write it to disk and reinitialize the same buffer • Repeat until all input blocks become exhausted. Chapter 2

  36. Input buffers, one for each sorted list Pointers to first unchosen record Select smallest unchosen for output Output buffers Main-memory Organization Chapter 2

  37. Merge Sort Example (1) • Assumption • 10,000,000 tuples, 1 tuple = 100 bytes • So, 1 Gbyte data • 50 Mbytes memory available • 4096 byte blocks, so each block contains 40 records • Total # of blocks: 250,000 • # of blocks in main memory: 12,800 (= 50*220 / 212) • Number of sublists • 19 sublists (12,800 blocks) + 1 sublists (6,800 blocks) • Each block read or write: 15 ms Chapter 2

  38. Merge Sort Example (2) • Computation • First phase • Read each of the 250,000 blocks once • Write 250,000 new blocks • Total time • (250,000 * 15 ms) * 2 = 7500 seconds = 125 minutes • Second phase • Similar with the first phase • Total time: 125 minutes Chapter 2

  39. Improving the Access Time of Secondary Storage • Place blocks on the same cylinder • Divide the data among several small disks • Mirroring disks • Use a disk-scheduling algorithm • Prefetch blocks to main memory in anticipation of their later use Chapter 2

  40. Organizing Data by Cylinders • Use several adjacent cylinders • Read all the blocks on a single track or on a cylinder consecutively • Neglect all but the first seek time and the first rotational latency Chapter 2

  41. Example 2.9 (1) • Recall examples 2.3 and 2.7 • Original data may be stored on consecutive cylinders • Total # of cylinders: 1000 (= 1Gbytes / 1M bytes) • Main memory can hold 50 cylinders (i.e. 50M) • To read 50 cylinder data into main memory • 6.5 ms for average seek time • 49 ms for 49 one-cylinder seeks (1 ms each) • 6.4 seconds for transfer of 12,800 blocks • (12,800 * 0.5 ms) / 1000 = 6.4 seconds • So, 6.5 + 49 + 6,400 = 6455.5 ms Chapter 2

  42. Example 2.9 (2) • First phase • Read • ((6.5 ms + 49 ms + 6.4 seconds) * 20 times) = 2.15 minutes • Write: The same as reading • Total time: 4.3 minutes • Second phase • Still takes about 125 minutes (WHY ?) Chapter 2

  43. Using Multiple Disks in place of One • Use several disks with their independent heads • Transfer data at a higher rate • Roughly speaking, total time could be divided by the number of disks Chapter 2

  44. Example 2.10 (1) • Replace one 747 by four 737’s which have one platter and two surfaces • Assumption • Divide the given records among the four disks • Occupy 1000 adjacent cylinders on each disk • Fill ¼ of main memory each disk • Recall previous examples • Average seek time and rotational latency: 0 • Number of full memory blocks: 12,800 • ¼ memory size: 3,200 blocks Chapter 2

  45. Example 2.10 (2) • Computation • First phase • Transfer time: 3200 * 0.5 ms = 1.6 seconds • Read: (6.5 ms + 49 ms + 1.6 seconds) * 20 = 33 sec. • Write: similar with reading • Total time: about 1 minute Chapter 2

  46. Example 2.10 (3) • Second phase • Apply delicate techniques (?) to reduce disk I/O time • Start comparisons among the 20 lists as soon as the first element of the block appears in main memory • Use four output buffers • … • Total time: about 1 hours (?) Chapter 2

  47. Mirroring Disks • Two or more disks hold identical copies of data • Survive a head crash by either disk • If we make n copies of a disk, we can read any n blocks in parallel. • Using mirror disks does not speed up writing, but neither does it slow writing down (to some extent) Chapter 2

  48. Scheduling Requests by the Elevator Algorithm • Disk controller choose which of several requests to execute first, to increase throughput • Elevator Algorithm • Proceed in the same direction until the next cylinder with blocks to access is encountered • When no requests ahead in direction of travel, reverse direction Chapter 2

  49. Example 2.11 Finishing times for block accesses using the elevator algorithm Finishing times for block accesses using the first-come-first-served algorithm Arrival times for six block-access requests Chapter 2

  50. Prefetching Data on Track- or Cylinder-sized Chunks • Can we predict the order in which blocks will be requested from disk ? • For example, • Devote two block buffers to each list when merged (when there is plenty of memory) • When a buffer is exhausted, switch to the other buffer for the same list Chapter 2

More Related