1 / 37

Logging in Flash-based Database Systems

Logging in Flash-based Database Systems. Lu Zeping 2009-06-27. Outline . Characteristics of flash memory Log records in different kinds of Flash devices In Flash chips - IPL In SSD In USB – FlashLogging Future work Recovery processing Experiments of PTLog. Outline .

wayne
Download Presentation

Logging in Flash-based Database Systems

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Logging in Flash-based Database Systems Lu Zeping 2009-06-27

  2. Outline • Characteristics of flash memory • Log records in different kinds of Flash devices • In Flash chips - IPL • In SSD • In USB – FlashLogging • Future work • Recovery processing • Experiments of PTLog

  3. Outline • Characteristics of flash memory • Log records in different kinds of Flash devices • In Flash chips - IPL • In SSD • In USB – FlashLogging • Future work • Recovery processing • Experiments of PTLog

  4. Characteristics of flash memory • No in-place update × Out – place update

  5. Characteristics of flash memory • No mechanical latency

  6. Characteristics of flash memory • Asymmetric speed of read/write

  7. The flash memory organization Device : 1G Block : 1024 Page : 64 2K Bytes 64 Bytes Page Register

  8. The flash memory organization Device : 1G Block : 1024 Page : 64 2K Bytes 64 Bytes Page Register

  9. Outline • Characteristics of flash memory • Log records in different kinds of Flash devices • In Flash chips - IPL • In SSD • In USB – FlashLogging • Future work • Recovery processing • Experiments of PTLog

  10. Design of IPL • IPL：In-Page Logging in-memory data page (8KB) Database Buffer in-memory log sector (512B) S. W. Lee, and B. Moon. Design of Flash-Based DBMS: An In-Page Logging Approach. SIGMOD 2007

  11. Design of IPL 15 data pages (8KB each) Erase unit (Block): 128KB Flash Memory …… log area (8KB): 16 sectors …… …. …. …. S. W. Lee, and B. Moon. Design of Flash-Based DBMS: An In-Page Logging Approach. SIGMOD 2007

  12. IPL Write in-memory data page (8KB) Database Buffer in-memory log sector (512B) Erase unit (Block): 128KB Flash Memory 15 data pages (8KB each) …… log area (8KB): 16 sectors …… …. …. …. S. W. Lee, and B. Moon. Design of Flash-Based DBMS: An In-Page Logging Approach. SIGMOD 2007

  13. IPL Read in-memory data page (8KB) Database Buffer in-memory log sector (512B) Erase unit (Block): 128KB Flash Memory 15 data pages (8KB each) …… log area (8KB): 16 sectors …… …. …. …. S. W. Lee, and B. Moon. Design of Flash-Based DBMS: An In-Page Logging Approach. SIGMOD 2007

  14. IPL Read in-memory data page (8KB) Database Buffer in-memory log sector (512B) Erase unit (Block): 128KB Flash Memory 15 data pages (8KB each) …… log area (8KB): 16 sectors …… …. …. …. S. W. Lee, and B. Moon. Design of Flash-Based DBMS: An In-Page Logging Approach. SIGMOD 2007

  15. IPL Merge Merge Flash Memory …… …… …. …. …… …… …. …. …. …. …. …. S. W. Lee, and B. Moon. Design of Flash-Based DBMS: An In-Page Logging Approach. SIGMOD 2007

  16. IPL Merge Merge Flash Memory …… …… …. …. …… …… …. …. …. …. …. …. S. W. Lee, and B. Moon. Design of Flash-Based DBMS: An In-Page Logging Approach. SIGMOD 2007

  17. Outline • Characteristics of flash memory • Log records in different kinds of Flash devices • In Flash chips - IPL • In SSD • In USB – FlashLogging • Future work • Recovery processing • Experiments of PTLog

  18. SSD • SSD = Flash memory + FTL（Flash Translation Layer）

  19. To be used in…… • Accessing data in secondary storage is the main source of bottlenecks in database processing. • Database table, index, log, temporary table and rollback segment could be created on physically separate storage devices .(SSD+ HDD) • Log records are always appended to the end of log.

  20. TPC-B benchmark performance We used TPC-B to stress-test the log storage part of the commercial database server. Due to the group commit mechanism, the average size of a single log write was slightly different between the two storage media. B. Moon, C. Park, S. W. Lee. A Case for Flash Memory SSD in Enterprise Database Applications. SIGMOD 2008.

  21. TPC-B benchmark performance In the case of SSD, the bottleneck in transaction throughput is the processing speed of CPU , while it was not in the case of disk. B. Moon, C. Park, S. W. Lee. A Case for Flash Memory SSD in Enterprise Database Applications. SIGMOD 2008.

  22. Outline • Characteristics of flash memory • Log records in different kinds of Flash devices • In Flash chips - IPL • In SSD • In USB – FlashLogging • Future work • Recovery processing • Experiments of PTLog

  23. USB flash memory advantages • USB ports are widely available. • USB bus bandwidth is good for logging. • The hot-plug capability allows easy replacement of drives. • The price of an individual drive is low. Shimin Chen. FlashLogging: Exploiting Flash Devices for Synchronous Logging Performance. SIGMOD'09.

  24. Flash devices and disk performance Shimin Chen. FlashLogging: Exploiting Flash Devices for Synchronous Logging Performance. SIGMOD'09.

  25. Four key challenges • Efficiently exploiting an array of flash drives. • Coping with large variance of write latencies. • Efficient recovery processing. • Combining USB flash drives with disks for better logging and recovery performance. Shimin Chen. FlashLogging: Exploiting Flash Devices for Synchronous Logging Performance. SIGMOD'09.

  26. Exploiting an array of flash drives What’s wrong with the conventional array organization? …… Devices：N In-memory log buffer the stripe unit size is S Data at address A are found on the k-th device, where k = ⌊A/S⌋ mod N. Shimin Chen. FlashLogging: Exploiting Flash Devices for Synchronous Logging Performance. SIGMOD'09.

  27. Unconventional array design …… Worker Worker Worker Request queue In-memory log buffer LSN：Log Sequence Number Shimin Chen. FlashLogging: Exploiting Flash Devices for Synchronous Logging Performance. SIGMOD'09.

  28. Coping with outlier Writes • Sequential writes have large variance • Outlier prediction is difficult. Shimin Chen. FlashLogging: Exploiting Flash Devices for Synchronous Logging Performance. SIGMOD'09.

  29. Outlier Detection and Hiding Measure the average sequential write latencies for different request sizes for a device. Outlier hiding area Non-decreasing LSN? Worker Worker Reissue the same request to a different ready device. Longer than twice the average latency? In-memory log buffer

  30. Efficient recovery processing Two main steps of recovery processing • Locating the log record corresponding to the LSN in the last checkpoint. (ii) Scanning the log sequentially until the crash point. Shimin Chen. FlashLogging: Exploiting Flash Devices for Synchronous Logging Performance. SIGMOD'09.

  31. Flash devices and disk performance Shimin Chen. FlashLogging: Exploiting Flash Devices for Synchronous Logging Performance. SIGMOD'09.

  32. FlashLogging design …… Worker Worker Worker Request queue In-memory log buffer Shimin Chen. FlashLogging: Exploiting Flash Devices for Synchronous Logging Performance. SIGMOD'09.

  33. Outline • Characteristics of flash memory • Log records in different kinds of Flash devices • In Flash chips - IPL • In SSD • In USB – FlashLogging • Future work • Recovery processing • Experiments of PTLog

  34. Several recovery technologies

  35. Outline • Characteristics of flash memory • Log records in different kinds of Flash devices • In Flash chips - IPL • In SSD • In USB – FlashLogging • Future work • Recovery processing • Experiments of PTLog

  36. Experiments of PTLog • Berkeley DB • LinuxNandsim nand_base.c、 nand_bbt.c、nand_ids.c、 nand_ecc.c 、 nandsim.c 、diskonchip.c、cafe_nand.c 、s3c2410.c

  37. Thank you！ Q&A？

More Related