Chapter 9 Log Manager

1. Chapter 9 Log Manager Dr. Kien A. Hua

2. Log Manager • Log knows everything. It is the temporal database • The online durable data are just their current versions • The log has their complete histories. • Uses of the Log: • Transaction recovery • Auditing • Performance analysis • Accounting • The log can easily become a performance problem, and it can get very large. • Intelligent algorithms are needed.

3. Log Sequence Numbers (LSNs) (1) • A log file consists of a sequence of log records • Each log record has a unique identifier, or key, called its log sequence number (LSN) • The LSN is composed of the record’s file number and the relative byte offset of the record within that file. typedef struct { long file; /* number of log file in log directory */ long rba; /* relative byte address of (first byte) record in file */ } LSN;

4. Log Sequence Numbers (LSNs) (2) Property: If log record A for an object is created “after” log record B for that object, then LSN (A) > LSN (B) This monotonicity property is used by the write-ahead log (WAL) protocol. Note: If two objects send their log records to different logs, then their LSNs are incomparable.

5. Log Manager: Overview (1) The log manager provides an interface to the log table, which is a sequence of log records. create table log_table( lsn LSN, prev_lsn LSN, /* for scanning the log backward */ timestamp TIMESTAMP, /* for time domain addressing */ resource_mg RMID, /* this RM will handle the UNDO-REDO work */ trid TRID, /* creator of this record */ tran_prev_lsn LSN, /* avoid scanning the log backward during transaction UNDO */ body varchar, /* UNDO-REDO information generated by the RM */ primary key(lsn), foreign key (prev_lsn) references log_table (lsn), foreign key (tran_prev_lsn) references log_table (lsn), ) entry sequenced; /* inserts go at end of tile */

6. Log Manager: Overview (2) Example: find the records written by a RM. select * from log_table where resource_mgr = :rmid order by lsn descending;

7. File System and Archive System • The log manager provides read and write access to the log table for all the other RMs, and for the TM. • In a perfect world, the log manager simply writes log records, and no one ever reads them. • In the event of failures, the log is used to return each logged object to its most recent consistent state. • The log manager maps the log table onto a growing collection of sequential files. • As the log table fills one file, another is allocated. • Only recent records are kept online. • Log records more than a few hours old are stored in less expensive tertiary storage managed by the archive system.

8. Why Have a Log Manager? Can’t we maintain the log table using SQL operations? At restart, almost none of the system is functioning. The log manager must be able to find, read, and write the log without much help from the SQL system. • The log manager must maintain and use a special catalog listing the physical log files. • It must use low-level interfaces to read write these catalogs and files.

9. Multiple Logs • In systems with very high update rates, the bandwidth of the log can become a bottleneck. • Such bottlenecks can be eliminated by creating multiple logs and by directing the log records of different objects to different logs. • In some situations, a particular RM keeps its own log table for portability reasons. • Distributed systems are likely to have one or more logs per network node. • They maintain multiple logs for performance and for node autonomy. • With a local log, each node can recover its local transactions without involving the other nodes.

10. Group Commit (1) Log Insert: • The program acquires the log lock. • It fixes the log page in the buffer pool. • It allocates space for the log record in the page, and fills in the record. • It unfixes the page in the buffer pool, and unlocks the semaphore. • The movement of data to durable storage is coordinated by an asynchronous process called the log flush daemon.

11. Group Commit (2) Group Commit: The log daemon wakes up once every t ms and does all the log writing that has accumulated in the buffer pool (batch processing log writes). advantage: I/O overhead is reduced disadvantage: It makes transaction last longer and delays releasing locks.