Design of Flash-Based DBMS: An In-Page Logging Approach

1. Design of Flash-Based DBMS: An In-Page Logging Approach Sang-Won Lee and Bongki Moon Presented by Chris Homan

2. Introduction • Flash Memory • Introduced 1987 • Mp3s, PDAs, mobile phones, digital cameras… • Advantages over disk • Smaller size • Lighter weight • Better shock resistance • Lower power consumption • Less noise • Faster read performance • Non-volatile and retains contents when powered off. • Already replaced disk in some computers. • Future mobile and embedded systems expected to carry out more larger, complex, data centric tasks.

3. How it works • Uses Flash Translation Layer (FTL) to make linear flash memory appear like disk to upper layers. • Two basic types: • NOR – full memory mapped random access interface, dedicated address and data lines. • NAND – no dedicated address line, IO-Like interface (similar to disk). • No data item updated in-place without erasing large block containing it (called an erase unit).

4. Problems of Conventional Design • Update in-place for DBMS • Clearly would prove quite costly with erasing before writing even for just 1 record. • Erase unit has a lifespan of 100,000 times before becoming statistically unreliable. • Wear-leveling • Distributes erase cycles evenly across memory. • No erasures as long as a free sector is available. • Optimizes write to detriment of read performance. • Large amount of garbage collection. • Bad for databases with a write-intensive workload!

5. Goals • Take advantage of new features of flash memory such as uniform random access speed and asymmetric read/write speed. • Overcome erase-before-write limitation of flash memory. • Minimize changes made to overall DBMS architecture.

6. In-Page Logging • To minimize the erase-before-write limitation, log changes to the database on the per-page basis. • If sequential logging were used, then log records could be written sequentially even if the data was scattered (requiring a sequential scan of the log to apply changes to a page). • Since there is no penalty for scattered writes, co-locate a data page and the log records pertaining to it to the same memory location (In-Page Logging).

7. Design • Only change Buffer and Storage Manager of DBMS. • When updating, the operation is the same as a traditional DBMS for the in-memory copy. • IPL Buffer Manager adds physiological log record on per-page basis to in-memory log sector associated with in memory copy of the data page. • These are written to Flash memory when log sector becomes full or when a dirty data page is evicted from the buffer pool. • Similar to write-caching.

8. Design (continued…) • No need to rewrite data, just write out log sectors. • Saves write operations. • Read reads data item and applies log record to it. • Each unit of flash memory divided into two parts by IPL Storage Manager • Data pages • Log sectors pertaining to those data pages

9. Merging • What happens when Flash memory log sectors become full? • Merging • Log sectors in erase unit are full • Allocate new erase unit • Compute new version of data page by applying log records to it • Write new version of data page to new erase unit • Erase and free old erase unit • FTL handles updating the logical-to-physical mapping of the data. • Merge is expensive, but is done less times than write and erase operations under in-place updating or sequential logging.

10. Evaluation • Flash Memory vs. Disk • TPC-C Benchmark

11. Recovery • Conventional system-wide logging must be adopted to keep track of start/end of transactions (commit or abort). • Maintain a list of dirty pages in the buffer pool in memory so that a transaction that commits or aborts (other than system failure) can locate in-memory log sectors containing log records added by the transaction.

12. Commit • No-Force Policy – only log tail is forced to stable storage, not all data pages modified by the transaction. • Force policy would lead to random disk accesses for an increase volume of data. • Write corresponding in-memory log sectors to Flash memory when a transaction commits. • No REDO action needed at system restart since redefined IPL read operation applies log records to data on the fly and the committed log records would be in Flash memory.

13. Abort • An aborting transaction (not by system failure) has log records that remain in in-memory log sectors that can be located via the list of dirty pages in memory. • Once found these are removed from their respective in-memory log sector, and de-applied to corresponding data pages in the buffer pool.

14. Abort (continued…) • What if a Merge operation already happened? • Selective Merge • Committed – apply log record and write new data • Aborted – ignore log record • Active – write log record to new log sector • If log sector remains full (such as all the transactions are active still), allow an overflow log sector in another erase unit. • No explicit UNDO operation – changes will just be merely ignored and not merged when a merge operation is encountered on its respective data page. • If upon recovery a transaction was found to be active (no commit or abort), add an abort record for the transaction to the system-wide log.

15. Conclusions • Flash memory will replace disks in computers. • Normal traditional database servers will suffer seriously deteriorated write performance due to erase-before-write limitation of Flash memory. • IPL has shown potential for considerably improving performance for OLTP apps by exploiting advantages of Flash memory. • IPL also supports a nice recovery mechanism for transactions.