ICS 214B: Transaction Processing and Distributed Data Management

ICS 214B: Transaction Processing and Distributed Data Management Course Project

Client Client Client Client Workflow Controller Transaction Manager Resource Manager Resource Manager Resource Manager Resource Manager Flights Hotels Cars Customers Distributed Travel Reservation System Notes 03

Project Plan • Two steps: • Build centralized travel reservation system • Components: Resource Manager • Add distributed functionality • Components: Workflow Controller, Transaction Manager Notes 03

Client Client Client Client interface ResourceManager Part 1: Simple Travel Resource Manager • start(); • queryFlightPrice(); • reserveFlight(); • queryCarPrice(); • reserveCar(); • … • commit(); Resource Manager Flights, Hotels, Cars, Customers Notes 03

Resource Manager Functionality • Data Manipulation • Query and Update flight, car, and hotel data • Make reservations • Transactions with ACID properties • Technical/Testing Support Notes 03

Data Definition (fixed) • Stores the following tables: • FLIGHTS(flightNum, price, numSeats, numAvail) • HOTELS(location, price, numRooms, numAvail) • CARS(location, price, numCars, numAvail) • CUSTOMERS(custName) • RESERVATIONS(custName, resvType, resvKey) • Simple (unrealistic) assumptions • One hotel, rental car agency per location • One airline • All seats in a flight cost the same • All rooms and cars in a location cost the same Notes 03

Data Manipulation Operations • Flights • Add flights • Add seats to flight • Remove seats from a flight • Cancel flight • Query number of seats available on a flight • Query price of a flight • Reserve seat in flight for a given customer • Similar operations for Cars and Hotels • Described in interface ResourceManager.java Notes 03

Important concepts review • We assume you’ve completed ICS214(A) or equivalent • Here we present a short overview just to refresh your memory Notes 03

Important concepts review Consistent DB Consistent DB’ T Notes 03

Big assumption: If T starts with consistent state + T executes in isolation  T leaves consistent state Notes 03

What can go wrong? • Transaction bug • DBMS bug • Hardware failure e.g., disk crash alters balance of account • System crash • CPU halts, memory wiped out, disk intact • Concurrency e.g.: T1: give 10% raise to programmers T2: change programmers  systems analysts Notes 03

Key problem: Unfinished transaction Example Constraint: A=B T1: A  A  2 B  B  2 Notes 03

failure! 16 16 16 T1: Read (A,t); t  t2 Write (A,t); Read (B,t); t  t2 Write (B,t); Output (A); Output (B); A: 8 B: 8 A: 8 B: 8 memory disk Notes 03

Need atomicity: execute all actions of a transaction or none at all Notes 03

Implementing Atomicity • In ICS214 (Fall 2002): • Logging • In this programming project, use • Shadow paging • Why? • Shadow paging is simpler to implement • But performance is lower than logging Notes 03

Shadow Paging Overview • Each file is managed via a page table P • Each transaction T updates the file via a private page table • Commit T by replacing the public page table by a private one • Example: suppose DB has two files, “a” and “b” Notes 03

Pt[a,1] Pt´[a] Pt[b,1] Pt´[b] 1 2 3 1 2 3 1 2 3 1 2 3 P2a old P2a new P2b new P1a new P1a old P2b old P1b old a b a b ... ... ... ... MAIN MEMORY FOR T D I S K Master´ Master Master Pointer Notes 03

Pt´[a] Pt´[b] Pt[b,1] Pt[a,1] 1 2 3 1 2 3 1 2 3 1 2 3 P2b old P2b new P2a old P2a new P1a new P1a old P1b old a b a b ... ... ... ... D I S K Master´ Master Master Pointer Notes 03

SHADOW PAGING • It is a technique pioneered in System R • changes are made to a copy of a page (block) • When a transaction commits, the copy becomes the current page and the original is discarded Notes 03

Single transaction • Suppose transaction A starts up: • the current page table (directory) is copied to the shadow page table (shadow directory) • if the transaction updates a page, the original page is not altered, rather a copy is created and that is modified • the copy is pointed to by the current page table - the shadow page table is never modified Notes 03

Write operation When transaction Tiissues a write(A) command, the write operation is executed as follows (assuming data item A resides on page PA): • If page PAis not already in main-memory then issue input(PA) • If this is the first write operation on page PAby transaction Tithen: • allocate a new disk page (call it tPA) • copy PAinto tPA • modify the current page table so that the entry corresponding to PA now points to tPA • perform the update on the page pointed to by tPA Notes 03

Commit When transaction Ti commits: • Ensure all buffer pages in memory that have been modified are flushed to the disk. • output the current page table to disk • change the current page table to become the new page table • free the pages of the old shadow page table that are no longer necessary • read the old shadow page table and free its pages Notes 03

Crash recovery • What is required if the system crashes while a transaction is executing? • free up all modified pages • discard the current page table • reinstate the shadow page table as the current page table Notes 03

Pros and cons • appears simple for single transaction environments • complexity increases for concurrent transactions • clustering diminishes quickly • not aware of any commercial implementations Notes 03

Multiple transactions • What if two transactions update different pages of a file? • If they share their main memory copy of the page table, then committing one will commit the other’s updates too! • Use a private copy of page table, per transaction. To commit T, within a critical section: • get a private copy of the last committed value of the page table of each file modified by T • update their entries for pages modified by T • store the updated page tables on disk • write a new master record and master pointer, thereby installing the update just for T (// end of critical section) Notes 03

Shadow Paging in Practice • Reference: R. Lorie, “Physical Integrity in a Large Segmented Database”ACM Trans. on DB Sys., March 1977. • Used in the Gemstone OO DBMS. • Not good for high-performance TP systems • count disk updates per transaction • how to do record-level locking? Notes 03

Concurrency Control T1 T2 … Tn DB (consistency constraints) • Use the 3 rules (well-formed xacts, legal scheduler, 2PL) • Support shared locks Notes 03

# locks time Simple Locking System • System transparently acquires locks when transactions access data • Holds all locks until transaction commits • Called “Strict 2-phase locking” • Strict 2PL automatically avoids cascading rollbacks Notes 03

Implementing the RM • For simplicity, implement the tables using hashtables • Each row is a hashtable entry • Create a class for every kind of row e.g., Flight • Primary key is hashkey • For the RESERVATIONS table, you can merge with the CUSTOMERS table: include a List of reservations in each hashtable entry. Notes 03

Implementing the RM • Java has convenient Hashtable and Vector classes • For durability, write the database to disk • You can use Java serialization to directly write the hashtables to disk • The class java.io.ObjectOutputStream might be helpful Notes 03

Implementing the RM • Using serialized hashtables makes it for easy persistence • But does not use block-based I/O model • Low-performance • Need to adapt shadow paging for this model Notes 03

Shadowing with hashtables • Have two copies of file on disk, with master pointer pointing to last committed copy • Last committed copy also cached in memory • Each transaction has private update list: hashtable of entries it has read or written • On commit: • Merge update list into in-memory table • Table written to disk using different filename • Master pointer updated to point to new file Notes 03

MAIN MEMORY File0 In-Mem Copy D I S K A:3 B:5 C:2 A:3 B:5 C:2 ... ... Update List for T B:8 … … ... Master Pointer Notes 03

MAIN MEMORY File0 In-Mem Copy D I S K A:3 B:5 C:2 A:3 B:8 C:2 ... ... Update List for T File1 B:8 … … A:3 B:8 C:2 ... ... Master Pointer Notes 03

Concurrency Control • We provide a lock manager • Supports two operations • lock(xid, itemName, READ|WRITE) • unlockAll(xid) • Implement two-phase locking using the supplied lock manager Notes 03

Resource Manager Transactions • A client starts a transaction by calling the start() method • Returns a transaction id (xid) • Client includes xid in all data manipulation operation requests • Client calls commit(xid) or abort(xid) to finish a transaction • System crash or deadlock may forcibly abort transactions Notes 03

Client Client Client Client Workflow Controller Transaction Manager Resource Manager Resource Manager Resource Manager Resource Manager Flights Hotels Cars Customers Part 2: Distributed Travel Reservation System Notes 03

Part 2: Distributed Transactions • Add Workflow Control and Transaction Manager components • Implement 2-phase commit • To be covered later Notes 03

Code base • project directory with two packages • lockmgr: do not change • lock & unlockAll in LockManager.java • transaction: your code here • ResourceManager.java: do not change • ResourceManagerImpl.java • Client.java • Makefile Notes 03

Java RMI • RMI: Remote Method Invocation • The way Client talks to the RM • 1. Start rmiregistry • Use your own port number • 2. Start server (RM) • RM binds to a name at the registry • 3. Start Client • Client queries registry using the bind name • All taken care of in the code base Notes 03

Java synchronized constructs • Convenient critical section implementation • Synchronized block: • Associated with any object • Before entering, thread obtains exclusive lock on obj. • no two threads inside synch. blocks belonging to same obj at same time • Synchronized methods: lock on “this” Notes 03

Java synchronized statements • Synchronized block example: AAA synchronized (ht) { BBB } CCC • Synchronized method example: public synchronized void foo() { SAME AS … } public void foo() { synchronized (this) { … } } Notes 03

Java Hashtable • Table of (key,value) pairs Hashtable ht = new Hashtable(); … // Insertion ht.put(“Flight214, flt); … // Lookup flt = (Flight) ht.get(“Flight214”); Notes 03

Java serialization • Object Serialization is process of writing the state of an object to a byte stream • Serializable objects can be written out to disk and restored easily. • Needed by RMI • Hashtable implements Serializable, so: ObjectOutputStream outS = new ObjectOutputStream(new FileOutputStream(“data/RMimage1”); outS.writeObject(ht); ---------------------------------------------- ObjectInputStream inS = new ObjectInputStream(new FileInputStream(“data/RMimage1”); Hashtable ht = (Hashtable) inS.readObject(); Notes 03

Project Logistics • Getting started: class home page • http://www.ics.uci.edu/~ics214b/ • Work on ICS SUN machines • Collaboration policy • As always, each group can have up to 3 students • Due dates: • part 1: Feb 15, Saturday • part 2: March 14, Friday • Submit: source code + short README • Grading: correct functionality • We’ll use our own Client.java to test your code Notes 03

Acknowledgements • This project is based on a similar course project developed by Anand Rajaraman at Stanford, which was based on a similar project developed by Phil Bernstein at the University of Washington Notes 03

ICS 214B: Transaction Processing and Distributed Data Management