Download
1 / 139
1.41k likes | 1.6k Views
Chapter 13. Disk Storage, Basic File Structures, and Hashing. Introduction. Memory Hierarchies and Storage Devices (1/6). Primary storage This category includes storage media that can be operated on directly by the computer CPU.
E N D
Chapter 13 Disk Storage, Basic File Structures, and Hashing
Memory Hierarchies and Storage Devices (1/6) • Primary storage • This category includes storage media that can be operated on directly by the computer CPU. • Primary storage usually provides fast access to data but is of limited storage capacity. • Secondary storage • The secondary storage devices usually have a larger capacity, cost less, and provide slower access to data than do primary storage devices. • Data in secondary storage cannot be processed directly by the CPU; it must first be copied into primary storage.
Memory Hierarchies and Storage Devices (2/6) • Primary storage • Cache memory • Static RAM • Main memory • DRAM • Advantage • Low cost • Drawbacks • Volatility • Lower speed compared with static RAM
Memory Hierarchies and Storage Devices (3/6) • Main memory databases • In some cases, entire databases can be kept in main memory (with a backup copy on magnetic disk). • These are particularly useful in real-time applications that require extremely fast response times.
Memory Hierarchies and Storage Devices (4/6) • Flash memory • Another form of memory between DRAM and magnetic disk storage. • Using EEPROM (Electrically Erasable Programmable Read-Only Memory) technology. • Advantages • Nonvolatile • Fast access speed • Disadvantage • An entire block must be erased and written over at a time.
Memory Hierarchies and Storage Devices (5/6) • Optical storage • CD-ROM • Contains prerecorded data that cannot be overwritten. • Holds about half a gigabyte of data per disk and last much longer than magnetic disks. • WORM (Write-Once-Read-Many) • Allows data to be written once and read any number of times without the possibility of erasing. • Optical juke box memories • Use an array of CD-ROM platters, which are loaded onto drives on demand. • Although optical juke boxes have capacities in the hundreds of gigabytes, their retrieval times are in the hundreds of milliseconds, quite a bit slower than magnetic disks. • DVD (Digital Video Disk) • Allows four to fifteen gigabytes of storage per disk.
Memory Hierarchies and Storage Devices (6/6) • Magnetic tapes are used for archiving and backup storage of data. • Tape jukeboxes—which contain a bank of tapes that are catalogued and can be automatically loaded onto tape drives—are becoming popular as tertiary storage to hold terabytes of data.
Storage of Databases (1/5) • Most databases are stored permanently (or persistently) on magnetic disk secondary storage, for the following reasons: • Generally, databases are too large to fit entirely in main memory. • The circumstances that cause permanent loss of stored data arise less frequently for disk secondary storage than for primary storage. • The cost of storage per unit of data is an order of magnitude less for disk than for primary storage.
Storage of Databases (2/5) • The techniques used to store large amounts of structured data on disk are important for database designers, the DBA, and implementers of a DBMS. • Database designers and the DBA must know the advantages and disadvantages of each storage technique when they design, implement, and operate a database on a specific DBMS. • Physical database design • DBMS system implementers must study data organization techniques so that they can implement them efficiently and thus provide the DBA and users of the DBMS with sufficient options.
Storage of Databases (3/5) • Typical database applications need only a small portion of the database at a time for processing. • Whenever a certain portion of the data is needed, it must be located on disk, copied to main memory for processing, and then rewritten to the disk if the data is changed. • The data stored on disk is organized as files of records.
Storage of Databases (4/5) • Primary file organizations determine how the records of a file are physically placed on the disk, and hence how the records can be accessed. • A heap file (or unordered file) places the records on disk in no particular order by appending new records at the end of the file. • A sorted file (or sequential file) keeps the records ordered by the value of a particular field (called the sort key). • A hashed file uses a hash function applied to a particular field (called the hash key) to determine a record’s placement on disk. • B-trees use tree structures.
Storage of Databases (5/5) • A secondary organization or auxiliary access structure allows efficient access to the records of a file based on alternate fields than those that have been used for the primary file organization.
Hardware Description of Disk Devices (1/8) • A disk is single-sided if it stores information on only one of its surfaces and double-sided if both surfaces are used. • To increase storage capacity, disks are assembled into a disk pack, which may include many disks and hence many surfaces. • Information is stored on a disk surface in concentric circles of small width, each having a distinct diameter. Each circle is called a track. • For disk packs, the tracks with the same diameter on the various surfaces are called a cylinder because of the shape they would form if connected in space.
Hardware Description of Disk Devices (2/8) • Because a track usually contains a large amount of information, it is divided into smaller sectors. • The division of a track into sectors is hard-coded on the disk surface and cannot be changed. • One type of sector organization calls a portion of a track that subtends a fixed angle at the center as a sector. • Several other sector organizations are possible, one of which is to have the sectors subtend smaller angles at the center as one moves away, thus maintaining a uniform density of recording. • Not all disks have their tracks divided into sectors.
Hardware Description of Disk Devices (3/8) • The division of a track into equal-sized disk blocks (or pages) is set by the operating system during disk formatting (or initialization). • Block size is fixed during initialization and cannot be changed dynamically. • A disk with hard-coded sectors often has the sectors subdivided into blocks during initialization. • Blocks are separated by fixed-size interblock gaps, which include specially coded control information written during disk initialization.
Hardware Description of Disk Devices (4/8) • Transfer of data between main memory and disk takes place in units of disk blocks. • The hardware address of a block—a combination of a surface number, track number (within the surface), and block number (within the track)—is supplied to the disk input/output (I/O) hardware. • The address of a buffer—a contiguous reserved area in main storage that holds one block—is also provided. • For a read command, the block from disk is copied into the buffer; whereas for a write command, the contents of the buffer are copied into the disk block. • Sometimes several contiguous blocks, called a cluster, may be transferred as a unit.
Hardware Description of Disk Devices (5/8) • The actual hardware mechanism that reads or writes a block is the disk read/writehead, which is part of a system called a disk drive. • A disk or disk pack is mounted in the disk drive, which includes a motor that rotates the disks. • A read/write head includes an electronic component attached to a mechanical arm. • Disk packs with multiple surfaces are controlled by several read/write heads—one for each surface. • All arms are connected to an actuator attached to another electrical motor, which moves the read/write heads in unison and positions them precisely over the cylinder of tracks specified in a block address.
Hardware Description of Disk Devices (6/8) • Disk drives for hard disks rotate the disk pack continuously at a constant speed. • Once the read/write head is positioned on the right track and the block specified in the block address moves under the read/write head, the electronic component of the read/write head is activated to transfer the data. • Some disk units have fixed read/write heads, with as many heads as there are tracks. These are called fixed-head disks, whereas disk units with an actuator are called movable-headdisks.
Hardware Description of Disk Devices (7/8) • A disk controller, typically embedded in the disk drive, controls the disk drive and interfaces it to the computer system. • The controller accepts high-level I/O commands and takes appropriate action to position the arm and causes the read/write action to take place. • The total time needed to locate and transfer an arbitrary block, given its address, is the sum of the seek time, rotational delay, and block transfer time.
Hardware Description of Disk Devices (8/8) • The time needed to locate and transfer a disk block is in the order of milliseconds. • Locating data on disk is a major bottleneck in database applications. • The file structures attempt to minimize the number of block transfers needed to locate and transfer the required data from disk to main memory.
Magnetic Tape Storage Devices (1/3) • Magnetic tapes are sequential access devices; to access the nth block on tape, we must first scan over the preceding n - 1 blocks. • Data is stored on reels of high-capacity magnetic tape, somewhat similar to audio or video tapes. • A tape drive is required to read the data from or to write the data to a tapereel.
Magnetic Tape Storage Devices (2/3) • A read/write head is used to read or write data on tape. • Data records on tape are also stored in blocks—although the blocks may be substantially larger than those for disks, and interblock gaps are also quite large.
Magnetic Tape Storage Devices (3/3) • Tape access is slow and tapes are not used to store on-line data, except for some specialized applications. • Tapes serve a very important function—that of backing up the database. • Tapes can also be used to store excessively large database files. • Finally, database files that are seldom used or outdated but are required for historical record keeping can be archived on tape.
With the exponential growth in the performance and capacity of semiconductor devices and memories, faster microprocessors with larger and larger primary memories are continually becoming available. • To match this growth, it is natural to expect that secondary storage technology must also take steps to keep up in performance and reliability with processor technology.
A major advance in secondary storage technology is represented by the development of RAID, which originally stood for Redundant Arrays of Inexpensive Disks. • Lately, the "I" in RAID is said to stand for Independent.
The main goal of RAID is to even out the widely different rates of performance improvement of disks against those in memory and microprocessors. • A concept called data striping is used, which utilizes parallelism to improve disk performance. • Data striping distributes data transparently over multiple disks to make them appear as a single large, fast disk. • Striping improves overall I/O performance by allowing multiple I/Os to be serviced in parallel, thus providing high overall transfer rates. • Data striping also accomplishes load balancing among disks. • Moreover, by storing redundant information on disks using parity or some other error correction code, reliability can be improved.
Improving Reliability with RAID (1/3) • For an array of n disks, the likelihood of failure is n times as much as that for one disk. • Keeping a single copy of data in such an array of disks will cause a significant loss of reliability. • An obvious solution is to employ redundancy of data so that disk failures can be tolerated. • The disadvantages are many: additional I/O operations for write, extra computation to maintain redundancy and to do recovery from errors, and additional disk capacity to store redundant information.
Improving Reliability with RAID (2/3) • One technique for introducing redundancy is called mirroring or shadowing. • Data is written redundantly to two identical physical disks that are treated as one logical disk. • When data is read, it can be retrieved from the disk with shorter queuing, seek, and rotational delays. • If a disk fails, the other disk is used until the first is repaired. • Disk mirroring also doubles the rate at which read requests are handled, since a read can go to either disk. • The transfer rate of each read, however, remains the same as that for a single disk.
Improving Reliability with RAID (3/3) • Another solution to the problem of reliability is to store extra information that is not normally needed but that can be used to reconstruct the lost information in case of disk failure. • The incorporation of redundancy must consider two problems: (1) selecting a technique for computing the redundant information, and (2) selecting a method of distributing the redundant information across the disk array. • The first problem is addressed by using error correcting codes involving parity bits, or specialized codes such as Hamming codes. • For the second problem, the two major approaches are either to store the redundant information on a small number of disks or to distribute it uniformly across all disks.
Improving Performance with RAID (1/2) • The disk arrays employ the technique of data striping to achieve higher transfer rates. • Disk striping may be applied at a finer granularity by breaking up a byte of data into bits and spreading the bits to different disks. • Bit-level data striping can be generalized to a number of disks that is either a multiple or a factor of eight. • In a m-disk array, bit n goes to the disk which is (n mod m). • Each disk participates in each I/O request and the total amount of data read per request is m times as much.
Improving Performance with RAID (2/2) • The granularity of data interleaving can be higher than a bit; for example, blocks of a file can be striped across disks, giving rise to block-level striping. • With block-level striping, multiple independent requests that access single blocks (small requests) can be serviced in parallel by separate disks, thus decreasing the queuing time of I/O requests. • Requests that access multiple blocks (large requests) can be parallelized, thus reducing their response time. • In general, the more the number of disks in an array, the larger the potential performance benefit.
RAID Organizations and Levels (1/4) • Different RAID organizations were defined based on different combinations of the two factors of granularity of data interleaving (striping) and pattern used to compute redundant information. • In the initial proposal, levels 1 through 5 of RAID were proposed, and two additional levels—0 and 6—were added later.
RAID Organizations and Levels (2/4) • RAID level 0 has no redundant data and hence has the best write performance since updates do not have to be duplicated. • However, its read performance is not as good as RAID level 1, which uses mirrored disks. • In the latter, performance improvement is possible by scheduling a read request to the disk with shortest expected seek and rotational delay. • RAID level 2 uses memory-style redundancy by using Hamming codes, which contain parity bits for distinct overlapping subsets of components. • Level 2 includes both error detection and correction, although detection is generally not required because broken disks identify themselves.
RAID Organizations and Levels (3/4) • RAID level 3 uses a single parity disk relying on the disk controller to figure out which disk has failed. • Levels 4 and 5 use block-level data striping, with level 5 distributing data and parity information across all disks. • Finally, RAID level 6 applies the so-called P + Q redundancy scheme using Reed-Soloman codes to protect against up to two disk failures by using just two redundant disks.
RAID Organizations and Levels (4/4) • Rebuilding in case of disk failure is easiest for RAID level 1. • Other levels require the reconstruction of a failed disk by reading multiple disks. • Level 1 is used for critical applications such as storing logs of transactions. • Levels 3 and 5 are preferred for large volume storage, with level 3 providing higher transfer rates.
Buffering of Blocks (1/2) • When several blocks need to be transferred from disk to main memory and all the block addresses are known, several buffers can be reserved in main memory to speed up the transfer. • While one buffer is being read or written, the CPU can process data in the other buffer. • This is possible because an independent disk I/O processor (controller) exists that, once started, can proceed to transfer a data block between memory and disk independent of and in parallel to CPU processing.
Buffering of Blocks (2/2) • Double buffering permits continuous reading or writing of data on consecutive disk blocks, which eliminates the seek time and rotational delay for all but the first block transfer. • Moreover, data is kept ready for processing, thus reducing the waiting time in the programs.
Records and Record Types (1/2) • Data is usually stored in the form of records. • Each record consists of a collection of related data values or items, where each value is formed of one or more bytes and corresponds to a particular field of the record. • A collection of field names and their corresponding data types constitutes a record type or record format definition.
Records and Record Types (2/2) • In recent database applications, the need may arise for storing data items that consist of large unstructured objects, which represent images, digitized video or audio streams, or free text. These are referred to as BLOBs (Binary Large Objects). • A BLOB data item is typically stored separately from its record in a pool of disk blocks, and a pointer to the BLOB is included in the record.
