1 / 8

Telegraph: A Universal System for Information

Telegraph: A Universal System for Information. Telegraph History & Plans. Initial Vision Carey, Hellerstein, Stonebraker “Regres”, “B-1” Sweat, ideas and further vision 4 of my grads committed Brewer + 2 grads committed Franklin will play obvious tie-ins with other projects. & synergies!.

isaac-thompson
Download Presentation

Telegraph: A Universal System for Information

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Telegraph: A Universal System for Information

  2. Telegraph History & Plans • Initial Vision • Carey, Hellerstein, Stonebraker • “Regres”, “B-1” • Sweat, ideas and further vision • 4 of my grads committed • Brewer + 2 grads committed • Franklin will play • obvious tie-ins with other projects

  3. & synergies! Telegraph Architecture Control, DigLib Query/Browse/Mine Mariposa, Millenium, Control Global Agoric Federation Continuously Reoptimizing Query Processor Adaptive Data Placement River, Ninja, Aetherstore, Control,STIX Storage Manager (FS, DB, Web) Ninja, GiST, IStore

  4. Storage Manager • Historic chance to start over! • new hardware realities • variable-length segments, not blocks • big main memories • extra CPUs at the devices (IStore) • revisit and clean up infrastructure for transactions • clean API supporting both log-based & version-based schemes; version-based runs today! • big SW Eng. challenge • unify DB/FS/Web server! • Clients: Ninja’s persistent hash table, query processing, web server, Linux (NT?) filesystem. • (Mohan Lakhamraju, Rob von Behren, Steve Gribble)

  5. Query Engine • Shared-nothing (cluster) • all data flow (no blocking ops) • auto load-balance to micro/macro changes in environment • adaptivity more important than raw performance!! • CONTROL! || ripple join, online reordering • (Shankar Raman) • continuously reoptimizing query plans • tie-ins with STIX (Christos/Sinclair/Russell/Hellerstein) • (Ron Avnur) • first steps in handling streaming sources

  6. Cluster Data Layout • issues: fragmentation, placement, replication on 10^6 disks. For DB/FS/Web. • goals: availability, efficiency, consistency, manageability. • Adaptivity: cooperative vs. competitive ($$) techniques? • (Mehul Shah)

  7. Global Federation • Global distribution • federated DBMS layer a la Mariposa/Cohera • address all the hard stuff they dropped! • Global data placement • as in cluster, but must be competitive. (Mehul Shah) • Global query processing (Amol Deshpande) • Agoric query optimization • distributed query processing • Global metadata • yellow pages both for services & datasets • Millenium/Ninja tie-ins?

  8. Applications • Really finding stuff in all the world’s data? • UI meets AI meets Logic (browse/mine/query) • CONTROL is key: seamless, non-blocking interaction • multi-res output and feedback during browse/query • hints, wizards, training (AI mining, user in the loop) • build on existing “scalable spreadsheet”/xform tools (Shankar Raman)

More Related