1 / 10

Publish / Subscribe Database Log Shipping over Bittorent P2P

Publish / Subscribe Database Log Shipping over Bittorent P2P. CS 848 Fall 2006 Univeristy of Waterloo Project Presentation by N. T c h e r v e n s k i. Intro. Implemented a tool to facilitate publish / subscribe of databases. Technologies used: Log shipping Bittorrent RSS.

bian
Download Presentation

Publish / Subscribe Database Log Shipping over Bittorent P2P

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Publish / Subscribe Database Log Shipping over Bittorent P2P CS 848 Fall 2006 Univeristy of Waterloo Project Presentation by N. Tchervenski

  2. Intro • Implemented a tool to facilitate publish / subscribe of databases. • Technologies used: • Log shipping • Bittorrent • RSS

  3. Motivation • Looking for an easy and quick way to create read-only replicated databases using minimum new infrastructure and minimum overhead • Instead of keeping a standby replica, can use it for queries • Log shipping can be performed on many of the popular DB systems – DB2, Oracle, MS SQL Server, Postgres, Teradata, etc. • Transferring large amounts of data can be done using P2P like Bittorrent

  4. Architecture DB Server Publishing Tool Archived Logs & backup images Archived logs directory RSS Feeds of tracker data BitTorrent seeder Internet RSS client DB Log management tool BitTorrent client Subscription Tool commands DB Replica Db restore and rollforward Archived logs directory

  5. Features • Minimum impact on the server • No need to capture data • Can be part of regular backup / replication process • Can send data to as many or as few peers as needed • Log shipping is popular – existing scripts and infrastructure can be reused • Sharing through Bittorrent is flexible – can limit upload speed, number of connections, disable IPs, etc.

  6. Current Limitations • Database backups are not cross-platform / cross-database-version portable • Moving the whole database, rather than just the data • need similarly configured machines (access control, paths, etc. ) • Delay when bringing up the database up after rollforward ( index rebuilding, etc. ). To include new logs, need to rollback and then rollforward again – this cannot be done too often. • Not suitable for databases with lots of updates • When LOAD is done (DB2), tablespace backup needs to be provided or data location be available to the remote DB • Security • Authorization to download • Bittorrent transfers can be slowed down by malicious peers sending garbage data

  7. Related Work • DPROPR - IBM DataPropagator Relational • Clients subscribe to particular rows / columns of tables • Can receive full refresh or just updates • For updates only mode, capture control tables are used

  8. Testing • Testing and implementation is done using • DB2 V9 • Linux – Ubuntu • Bittorrent client – Enchanced CTorrent

  9. Conclusion • Based on gluing together existing technologies • A way to use standby replica • Legitimate use of BitTorrent  • Hope this will stir more related research • Ideal for public databases

  10. References • [1] DPROPR Planning and Design Guide, http://www.redbooks.ibm.com/abstracts/sg244771.html • DB2 Replication Guide and Reference, ftp://ftp.software.ibm.com/ps/products/db2/info/vr82/pdf/en_US/db2e0e82.pdf • Warm Standby Servers for High Availability, http://www.postgresql.org/docs/8.2/static/warm-standby.html

More Related