1 / 25

Continuous Queries

NiagaraCQ : A Scalable Continuous Query System for Internet Databases Jianjun Chen et al Computer Sciences Dept. University of Wisconsin-Madison SIGMOD 2000 Presented by Mukund Agrawal. Continuous Queries. A triple ( Q, A, Stop) Scope also includes future data Example

chaeli
Download Presentation

Continuous Queries

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. NiagaraCQ : A Scalable Continuous Query System for Internet DatabasesJianjun Chen et al Computer Sciences Dept. University of Wisconsin-MadisonSIGMOD 2000Presented byMukund Agrawal

  2. Continuous Queries • A triple ( Q, A, Stop) • Scope also includes future data • Example Inform me when there is a new publication related to multi-query optimization • A broad classification • Change based • Timer based

  3. NiagaraCQ • A CQ system for the Internet • Continuous Queries on XML data sets • Scalable CQ processing • Incremental group optimization • Handles both change based and timer based queries in a uniform way

  4. Outline • General strategy of incremental group optimization • Query split with materialized intermediate files • Incremental grouping of selection and join operators • System architecture • Experimental results

  5. NiagaraCQ command language • Creating a CQ CreateCQ_name XML-QL query Doaction { START start_time} { EVERY time_interval} { EXPIRE expiration_time} • Delete CQ_name

  6. Incremental group optimizationGeneral Strategy • Why can’t we regroup all queries when a new query is added ? • Use of expression signatures for grouping • Same syntax structure • Different constant values

  7. Expression Signature • Query examples Where <Quotes><Quote><Symbol>INTC</></></> element_as $g in “http://www.stock.com/quotes.xml” construct $g Where <Quotes><Quote><Symbol>MSFT</></></> element_as $g in “http://www.stock.com/quotes.xml” construct $g • Expression signatures = Quotes.Quote.Symbol in quotes.xml constant

  8. Query plans Trigger Action I Trigger Action J Select Symbol = “INTC” Select Symbol = “MSFT” File Scan File Scan quotes.xml quotes.xml

  9. Group • Group Signature • Common signature of all queries in the group • Group constant table

  10. The group plan

  11. Incremental Grouping Algo When a new query is submitted If the expression signature of the new query matches that of existing groups Break the query plan into two parts Remove the lower part Add the upper part onto the group plan else create a new group

  12. Query split with materialized intermediate files • Why not use a pipeline scheme ? • Split operator may block simple queries • Gives a single complicated execution plan • A large portion of query plan may not need to be executed at each invocation • Does not work for grouping timer based queries • Using intermediate files • Cut query plan into 2 parts at split operator • Add a file scan operator to upper part to read intermediate file

  13. The query split scheme

  14. Trade-offs • Other advantages of materialized intermediate files • Only the necessary queries are executed • Uniform handling of intermediate files and original data source files • Disadvantages • Split operator becomes a blocking operator • Extra disk I/Os

  15. Incremental grouping of selection predicates • Multiple selection predicates in a query • CNF for predicates on same data source • Incremental grouping • Choose the most selective conjunct • Evaluation of other predicates • Upper levels of continuous query • Example query Where <Quotes><Quote><Symbol>”INTC”</> <Current_Price>$p</></> element_as $g </> in “quotes.xml”, $p < 100 Construct $g

  16. Range-query groups • Problem • Intermediate files may contain duplicate tuples • Solution : Virtual intermediate files • Virtual intermediate file stores value ranges • One real intermediate file has a clustered index

  17. Incremental grouping of join operators • A join query Quotes.Quote.Change_Ratio constant in “quotes.xml” Where <Quotes><Quote><Symbol>$s</></> element_as $g </> in “quotes.xml”, <Companies><Company><Symbol>$s</></> element_as $t</> in “companies.xml” construct $g, $t

  18. Queries that contain both join and selection • Example query : Where <Quotes><Quote><Symbol>$s</> <Industry>”Computer Service”</></> element_as $g </> in “quotes.xml”, <Companies><Company><Symbol>$s</></> element_as $t</> in “companies.xml” construct $g, $t • Where to place the selection operator ? • Below the join • Above the join

  19. Grouping timer-based queries • Challenge • Sharing common computation • Event List • Stores time events sorted in time order

  20. Incremental evaluation • Invoke queries only on changed data • For each file, NiagaraCQ keeps a delta file • Incremental evaluation of join operators requires complete data files

  21. Memory Caching • Thousands of continuous queries can’t fit in memory • What should we cache ? • Grouped query plans • What about non-grouped queries ? • Favor small delta files • Front part of the event list

  22. System Architecture

  23. CQ processing

  24. Experimental Results Example query : Where <Quotes><Quote><Symbol>”INTC”</></> element_as $g </> in “quotes.xml”, construct $g

  25. Thank You

More Related